Virtual optical touch screen detecting touch distance

• US 9,454,243 B2
• Filed: 04/11/2014
• Issued: 09/27/2016
• Est. Priority Date: 04/12/2013
• Status: Active Grant

• Current Assignee: Iconics, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Iconics, Inc. (Mitsubishi Electric Corporation)
• Inventors: Kresl, Vojta, Elsbree, Christopher N., Agrusa, Russell L., De Barros, Paulo G., Altman, Petr
• Primary Examiner(s): Sternbane, Larry
• Assistant Examiner(s): Azari, Sepehr
• Application Number:
  US14/251,059
• Publication Number:
  US 20140306889A1
• Time in Patent Office: 900 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386

Abstract

Apparatus and methods are provided for converting a display into a touch display. One or more optical sensors are arranged proximate the display. A processor receives a signal from the optical sensor(s) and, based on the signal, executes instructions for: (i) determining a distance between a pointer device and a surface of the display signal; (ii) determining a position of the pointer device on the surface of the display; (iii) when the distance is less than a threshold detection distance and greater than a threshold touch distance, providing an indication on the display that the position of the pointer device has been determined; and (iv) when the distance is less than the threshold touch distance, taking an action indicative of contact between the pointer device and the surface of the display.

Claims

1. An apparatus for converting a display into a touch display, the apparatus comprising:

at least one optical sensor arranged proximate the display and substantially within a plane defined by a surface of the display, the at least one optical sensor comprising at least one depth sensor and being oriented in a direction parallel to the surface of the display; and

a processor configured to receive a signal from the at least one optical sensor, the processor configured to execute instructions for;

(i) determining a first distance between at least one pointer device and a surface of the display, based on the signal;

(ii) determining a position of the at least one pointer device on the surface of the display, based on the signal; and

(iii) determining that the first distance is less than a threshold touch distance and, in response thereto, taking an action indicative of contact between the at least one pointer device and the surface of the display,

wherein the at least one depth sensor comprises a laser projector configured to determine a second distance between the at least one optical sensor and the at least one pointer device, along the surface of the display.

2. The apparatus of claim 1, wherein the at least one optical sensor further comprises at least one camera.

3. The apparatus of claim 1, wherein the at least one optical sensor comprises a first optical sensor oriented in a first direction, and a second optical sensor oriented in a second direction, the first direction being substantially orthogonal to the second direction, the first and second directions being substantially parallel to the surface of the display.

4. The apparatus of claim 3, wherein the at least one optical sensor further comprises a third optical sensor oriented in a third direction, and a fourth optical sensor oriented in a fourth direction, the third direction being substantially orthogonal to the fourth direction, the third and fourth directions being substantially parallel to the surface of the display.

5. The apparatus of claim 1, wherein the at least one pointer device comprises two or more pointer devices, and the processor is configured to determine a position of each pointer device on the surface of the display.

6. The apparatus of claim 1, wherein the at least one pointer device comprises at least one finger of a user.

7. The apparatus of claim 1, wherein determining a position comprises determining a closest point to the at least one pointer device on the surface of the display.

8. The apparatus of claim 1, wherein the threshold touch distance is less than about 1 inch.

9. The apparatus of claim 1, wherein the processor is further configured to execute instructions for determining that the first distance is less than a threshold detection distance and greater than the threshold touch distance and, in response thereto, providing an indication on the display that the position of the at least one pointer device has been determined, the indication comprising adjusting an image on the display in a vicinity of the position of the at least one pointer device on the surface of the display.

10. The apparatus of claim 9, wherein the threshold detection distance is less than about 6 inches.

11. The apparatus of claim 1, wherein taking an action indicative of contact between the at least one pointer device and the surface of the display comprises an action selected from the group consisting of:

selecting an item, translating an image, expanding an image, shrinking an image, and rotating a virtual object.

12. The apparatus of claim 1, wherein determining the first distance between the at least one pointer device and the surface of the display comprises calculating a scale factor based on the second distance between the at least one optical sensor and the at least one pointer device.

13. A method of converting a display into a touch display, the method comprising:

receiving a signal from at least one optical sensor arranged proximate the display and substantially within a plane defined by a surface of the display, the at least one optical sensor comprising at least one depth sensor and being oriented in a direction parallel to the surface of the display;

determining a first distance between at least one pointer device and a surface of the display, based on the signal;

determining a position of the at least one pointer device on the surface of the display, based on the signal; and

determining that the first distance is less than a threshold touch distance and, in response thereto, taking an action indicative of contact between the at least one pointer device and the surface of the display,

wherein the at least one depth sensor comprises a laser projector configured to determine a second distance between the at least one optical sensor and the at least one pointer device, along the surface of the display.

14. The method of claim 13, wherein the at least one optical sensor further comprises at least one camera.

15. The method of claim 13, wherein the at least one optical sensor comprises a first optical sensor oriented in a first direction, and a second optical sensor oriented in a second direction, the first direction being substantially orthogonal to the second direction, the first and second directions being substantially parallel to the surface of the display.

16. The method of claim 15, wherein the at least one optical sensor further comprises a third optical sensor oriented in a third direction, and a fourth optical sensor oriented in a fourth direction, the third direction being substantially orthogonal to the fourth direction, the third and fourth directions being substantially parallel to the surface of the display.

17. The method of claim 13, wherein the at least one pointer device comprises two or more pointer devices, and determining a position of the least one pointer device comprises determining a position of each pointer device on the surface of the display.

18. The method of claim 13, wherein the at least one pointer device comprises at least one finger of a user.

19. The method of claim 13, wherein determining a position comprises determining a closest point to the at least one pointer device on the surface of the display.

20. The method of claim 13, wherein the threshold touch distance is less than about 1 inch.

21. The method of claim 13, further comprising determining that the first distance is less than a threshold detection distance and greater than the threshold touch distance and, in response thereto, providing an indication on the display that the position of the at least one pointer device has been determined, wherein providing the indication on the display comprises adjusting an image on the display in a vicinity of the position of the at least one pointer device on the surface of the display.

22. The method of claim 21, wherein the threshold detection distance is less than about 6 inches.

23. The method of claim 13, wherein taking an action indicative of contact between the at least one pointer device and the surface of the display comprises an action selected from the group consisting of:

selecting an item, translating an image, expanding an image, shrinking an image, and rotating a virtual object.

24. The method of claim 13, wherein determining the first distance between the at least one pointer device and the surface of the display comprises calculating a scale factor based on the second distance between the at least one optical sensor and the at least one pointer device.