Multi-touch touchscreen incorporating pen tracking

• US 9,268,413 B2
• Filed: 07/07/2008
• Issued: 02/23/2016
• Est. Priority Date: 07/07/2008
• Status: Expired due to Fees

• Current Assignee: RPX Clearinghouse LLC (RPX Corporation)
• Original Assignee: RPX Clearinghouse LLC (RPX Corporation)
• Inventors: Doray, Bernard, To, Paul, Haller, Michael, Powell, James Robert, Brandl, Peter, Leitner, Jakob, Seifried, Thomas Josef, Ma, Moses Tao-Ling
• Primary Examiner(s): Patel, Kumar
• Assistant Examiner(s): Sadio, Insa
• Application Number:
  US12/168,678
• Publication Number:
  US 20100001962A1
• Time in Patent Office: 2,787 Days
• Field of Search: 345173-184, 178 1801- 1809
• US Class Current: 1/1
• CPC Class Codes: G06F 2203/04109 FTIR in optical digitiser, i.e. touch detection by frustrating the total internal reflection within an optical waveguide due to changes of optical properties or deformation at the touch location
• CPC Class Codes: G06F 2203/04808 Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
• CPC Class Codes: G06F 3/0317 in co-operation with a patterned surface, e.g. absolute position or relative movement detection for an optical mouse or pen positioned with respect to a coded surface
• CPC Class Codes: G06F 3/0321 by optically sensing the absolute position with respect to a regularly patterned surface forming a passive digitiser, e.g. pen optically detecting position indicative tags printed on a paper sheet constructional details of pen-shaped pointing devices G06F3/03545, G06F3/03542, G06F3/037
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

The present invention relates to a multi-touch display system that supports both multi-touch human input as well as input from a digital pen. The display system has a display panel that is configured to allow human touches along a front surface to be detected and tracked. The display panel also includes a location pattern that preferably covers the viewable areas of the display panel. The location pattern is configured to allow any location within the location pattern to be detected by analyzing a portion of the display pattern that is associated with the particular location. The digital pen is used to “write” on the display panel, wherein such a writing function involves detecting the location where writing occurs and controlling display content that is displayed on the display panel to reflect what is being written.

Claims

1. A multi-touch display system comprising:

a display panel comprising a front surface to receive human touches and a location pattern provided in association with the front surface, the location pattern readable by a digital pen from the front surface and providing unique location indicia for the front surface wherein unique locations in the location pattern are identifiable from corresponding portions of the unique location indicia, and wherein a unique identifier of the display panel is encoded in the location pattern;

touch sensing circuitry adapted to detect the human touches on the front surface of the display panel and provide corresponding touch input;

a communication interface adapted to receive pen data bearing on writing information derived in response to the digital pen capturing images of those portions of the unique location indicia where the digital pen is being used to write; and

processing circuitry associated with the touch sensing circuitry and the communication interface and adapted to control display content being displayed on the display panel in response to the touch input and the pen data.

2. The multi-touch display system of claim 1 wherein the location indicia of the location pattern is continuously variable across the entire front surface of the display panel such that the location indicia is unique at any point within the location pattern.

3. The multi-touch display system of claim 1 wherein the location indicia comprises a plurality of dots.

4. The multi-touch display system of claim 1 wherein the location pattern is an Anoto pattern.

5. The multi-touch display system of claim 1 wherein the location indicia at any point within the location pattern correspond to a unique location coordinate for the point in relation to the front surface of the display panel.

6. The multi-touch display system of claim 1 wherein the processing circuitry is adapted to control the display content by including writing substantially directly corresponding to what is being written on the display panel by the digital pen.

7. The multi-touch display system of claim 1 wherein the display panel comprises a rear surface further comprising a projector that is associated with the processing circuitry and configured to project the display content toward the rear surface of the display panel under control of the processing circuitry, the display panel configured to allow the display content being projected toward the rear surface to be viewable on the front surface.

8. The multi-touch display system of claim 1 wherein to detect the human touches on the front surface of the display panel, the touch sensing circuitry is adapted to detect reflected light that is reflected from a rear surface of the display panel in response to the human touches.

9. The multi-touch display system of claim 8 wherein reflected light is infrared light.

10. The multi-touch display system of claim 8 wherein the display panel comprises a waveguide panel and a light source adapted to inject light into the waveguide panel such that a state of total internal reflection is achieved within the waveguide panel when the front surface of the display panel is not being touched by a human and frustrated total internal reflections occur when the front surface of the display panel receives the human touches, certain of the frustrated total internal reflections being reflected light that is detectable by the touch sensing circuitry.

11. The multi-touch display system of claim 10 wherein the waveguide panel comprises a beveled perimeter edge that has a substantially flat edge surface on which the light source is mounted, the light source injecting the light into the waveguide panel through the substantially flat edge surface.

12. The multi-touch display system of claim 11 wherein the light source comprises a plurality of light emitting diodes that are mounted along the substantially flat edge surface of the beveled perimeter edge.

13. The multi-touch display system of claim 1 wherein the communication interface is a wireless communication interface that is adapted to receive the pen data via wireless communications.

14. The multi-touch display system of claim 13 wherein the wireless communication interface is adapted to communicate directly with the digital pen.

15. The multi-touch display system of claim 1 wherein the communication interface is adapted to communicate with a device or system that is capable of communicating with the digital pen.

16. The multi-touch display system of claim 1 wherein the display panel comprises a waveguide panel, a compliant layer adjacent to the waveguide panel, and a projection layer over the compliant layer, the projection layer comprising the location pattern and providing a projection surface on which an image of the display content is projected.

17. The multi-touch display system of claim 16 wherein the projection layer comprises a translucent screen layer and a pattern layer above the translucent screen layer wherein the translucent screen layer provides the projection surface and the pattern layer comprises the location pattern.

18. The multi-touch display system of claim 16 wherein the display panel further comprises a filter layer adapted to filter at least certain wavelengths of ambient light.

19. The multi-touch display system of claim 18 wherein the filter layer is provided between the compliant layer and the projection layer.

20. The multi-touch display system of claim 19 wherein the filter layer comprises:

a filtering layer to filter the at least certain wavelengths of ambient light; and

an anti-stick layer between the filtering layer and the compliant layer and configured to prevent the filtering layer from adhering to the compliant layer in response to the human touches.

21. The multi-touch display system of claim 16 wherein the waveguide panel comprises a beveled perimeter edge that has a substantially flat edge surface on which a light source is mounted, the light source injecting light into the waveguide panel through the substantially flat edge surface.

22. The multi-touch display system of claim 21 wherein an angle formed between the substantially flat edge surface and the front surface is between about 30 degrees and 60 degrees.

23. The multi-touch display system of claim 22 wherein the angle formed between the substantially flat edge surface and the front surface is around about 45 degrees.

24. The multi-touch display system of claim 16 wherein the projection layer further provides a reflection surface from which light that is provided from a light source of the digital pen and that is not absorbed by the location pattern is reflected back toward a light sensor of the digital pen.

25. The multi-touch display system of claim 1 wherein the touch sensing circuitry is adapted to detect frustrated total internal reflections.

26. The multi-touch display system of claim 1 further comprising the digital pen, which comprises a body, a stylus at one end of the body, a pressure sensor associated with the stylus, an image sensor, a light source, and a processor associated with the pressure sensor and the image sensor, wherein the processor is adapted to capture successive images of those portions of the unique location indicia where the digital pen is being used to write in response to the pressure sensor indicating that the stylus is in contact with the front surface of the display panel, the light source illuminating those portions of the unique location indicia to facilitate image capture.

27. A multi-touch display system comprising:

a display panel comprising a front surface to receive human touches and a location pattern provided in association with the front surface, the location pattern readable by a digital pen from the front surface and providing unique location indicia for the front surface wherein unique locations in the location pattern are identifiable from corresponding portions of the unique location indicia;

touch sensing circuitry adapted to detect the human touches on the front surface of the display panel and provide corresponding touch input;

a communication interface adapted to receive pen data bearing on writing information derived in response to the digital pen capturing images of those portions of the unique location indicia where the digital pen is being used to write; and

processing circuitry associated with the touch sensing circuitry and the communication interface and adapted to control display content being displayed on the display panel in response to the touch input and the pen data,

wherein to detect the human touches on the front surface of the display panel, the touch sensing circuitry is adapted to detect reflected light that is reflected from a rear surface of the display panel in response to the human touches;

wherein the display panel comprises a waveguide panel and a light source adapted to inject light into the waveguide panel such that a state of total internal reflection is achieved within the waveguide panel when the front surface of the display panel is not being touched by a human and frustrated total internal reflections occur when the front surface of the display panel receives the human touches, certain of the frustrated total internal reflections being reflected light that is detectable by the touch sensing circuitry;

wherein the waveguide panel comprises a beveled perimeter edge that has a substantially flat edge surface on which the light source is mounted, the light source injecting the light into the waveguide panel through the substantially flat edge surface; and

wherein an angle formed between the substantially flat edge surface and the front surface is between about 30 degrees and 60 degrees.

28. The multi-touch display system of claim 27 wherein the angle formed between the substantially flat edge surface and the front surface is around about 45 degrees.