Determining touch signals from interactions with a reference plane proximate to a display surface

• US 10,185,445 B2
• Filed: 05/27/2015
• Issued: 01/22/2019
• Est. Priority Date: 10/14/2013
• Status: Active Grant

• Current Assignee: Touchjet Pte., Ltd.
• Original Assignee: Touchjet Pte., Ltd.
• Inventors: Li, Jiang, Liu, Zhen
• Primary Examiner(s): Shah, Priyank J
• Application Number:
  US14/723,088
• Publication Number:
  US 20150253929A1
• Time in Patent Office: 1,336 Days
• CPC Class Codes: G03B 17/54 with projector
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20
• CPC Class Codes: H04N 9/3173 wherein the projection device is specially adapted for enhanced portability

Abstract

A reference plane generating unit generates a reference plane of light parallel to a display surface. For example, the reference plane generating unit projects a plane of infrared light parallel to a plane including the display surface. When a pointer, such a stylus or a finger, breaks the reference plane, light is reflected from the reference plane, with at least a portion of the reflected light directed toward an image capturing unit. Based on the reflected light captured by the image capturing unit, the image processing unit determines a position of the pointer in the reference plane and determines a touch signal corresponding to the determined position of the pointer. The touch signal is communicated to a processing unit, such as a mobile device, that performs an operation corresponding to the determined touch signal.

Claims

1. A system for performing one or more operations based on interaction with image data presented on a display surface comprising:

a reference plane generating unit configured to project a reference plane of light parallel to the display surface on which frames of image data are displayed, the reference plane positioned within a threshold distance of the display surface and detachably coupled to a display device including the display surface;

an image capturing unit positioned so the display surface and the reference plane are within a field of view of the image capturing unit, the image capturing unit configured to capture images including light reflected from the reference plane by a pointer breaking the reference plane of light, the image capturing unit detachably coupled to a foldable support that is also coupled to the display device including the display surface;

an image processing unit configured to store data associating different positions and trajectories of light reflected from the reference plane with different touch signals, to determine a position in the reference plane of the pointer and a trajectory of the pointer in the reference plane from the images captured by the image capturing unit including light reflected from the reference plane by the pointer, and to determine a stored touch signal associated with a position and a trajectory matching the determined position and trajectory of the pointer in the reference plane; and

a processing unit configured to maintain stored data associating touch signals with commands, to receive the touch signal from the image processing unit, to retrieve a command associated with the touch signal from the image processing unit, and to communicate a display signal execute the retrieved command.

2. The system of claim 1, wherein the reference plane generating unit comprises a coherent light source projecting light onto a diffractive optical element that projects the reference plane of light.

3. The system of claim 2, wherein the coherent light source is a laser.

4. The method of claim 2, wherein the diffractive optical element is a lens.

5. The system of claim 1, wherein the reference plane of light is in an infrared wavelength.

6. The system of claim 1, wherein the display surface is a monitor.

7. The system of claim 1, wherein the display surface is a projection screen.

8. The system of claim 1, wherein the image processing unit is configured to determine the touch signal corresponding to the determined position in the reference plane based at least in part on image data presented by a portion of the display surface in a plane parallel to the determined position in the reference plane.

9. The system of claim 1, wherein the pointer comprises a stylus.

10. The system of claim 1, wherein the image capturing unit comprises an infrared camera.

11. The system of claim 1, wherein a width of the reference plane equals or exceeds a width of the display surface and a length of the reference plane equals or exceeds a length of the reference plane.

12. The system of claim 1, wherein the image capturing unit is further configured to capture a series of images including light reflected from the reference plane by the pointer breaking the reference plane of light at different times.

13. The system of claim 12, wherein the image processing unit determines the trajectory of the pointer within the reference plane based on the series of images including light reflected from the reference plane by the pointer breaking the reference plane of light at different times.

14. A method for performing one or more operations based on interaction with image data presented on a display surface comprising:

maintaining stored data associating touch signals with commands and associating different touch signals with different positions and trajectories of light reflected from a reference plane;

projecting the reference plane of light parallel to the display surface on which frame of image date, the reference plane positioned within a threshold distance of the display surface;

capturing images of the reference plane at different times by an image capturing unit detachably coupled to a foldable support that is also coupled to a display device including the display surface, each image including light reflected from the reference plane by a pointer breaking the reference plane of light;

determining a position of the pointer in the reference plane and a trajectory of the pointer in the reference plane based on the images including the light reflected from the reference plane;

determining a touch signal associated with a position and a trajectory matching the determined position and trajectory of the pointer in the reference plane from the stored data;

retrieving a command associated with the determined touch signal from the stored data; and

executing the retrieved command associated with the determined touch signal to perform an operation corresponding to the determined touch signal.

15. The method of claim 14, wherein determining the position of the pointer in the reference plane and the trajectory of the pointer in the reference plane based on the images including the light reflected from the reference plane comprises:

determining a trajectory of the pointer within the reference plane based on positions of the pointer in the reference plane in images including light reflected from the reference plane by the pointer at different times.