Method and apparatus for approximating depth of an object's placement onto a monitored region with applications to virtual interface devices

• US 20030218761A1
• Filed: 09/17/2002
• Published: 11/27/2003
• Est. Priority Date: 05/22/2002
• Status: Active Grant

• Current Assignee: Heilongjiang Golden Jumping Group Co., Ltd.
• Original Assignee: Canesta Incorporated (Microsoft Corporation)
• Inventors: Gokturk, Salih Burak, Tomasi, Carlo
• Granted Patent: US 7,006,236 B2
• Time in Patent Office: Days
• US Class Current: 356/627
• CPC Class Codes: G01S 17/06 Systems determining position data of a target
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1632 External expansion units, e.g. docking stations
• CPC Class Codes: G06F 1/1673 Arrangements for projecting a virtual keyboard
• CPC Class Codes: G06F 2200/1633 Protecting arrangement for the entire housing of the computer
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

Structured light is directed across a monitored region. An image is captured of a light pattern that forms on the object as a result of the object intersecting the structured light when the object is placed at a first position in the monitored region. A geometric characteristic is identified of the image of the light pattern. The geometric characteristic is variable with a depth of the first position relative to where the image is captured. The depth of the first position is approximated based on the measured geometric characteristic.

Claims

1. An electronic device for detecting a depth of an object'"'"'s placement on a monitored region, the electronic device comprising:

one or more light sources configured to direct structured light across the monitored region;

a light detecting device configured to capture an image of a light pattern that forms on the object as a result of the object intersecting the structured light when being placed at a first position in the monitored region; and

a processor configured to;

identify a geometric characteristic of the image of the light pattern, wherein the geometric characteristic of the image is variable with a depth of the first position relative to where the light detecting device captures the image; and

approximate the depth of the first position based on the identified geometric characteristic.

2. The electronic device of claim 1, wherein the one or more light sources are actuatable to generate one or more beams that extend across and fan over the monitored region.

3. The electronic device of claim 1, wherein the one or more light sources are actuatable to generate a first beam and a second beam to extend over the monitored region, wherein the first beam and the second beam are parallel to one another.

4. The electronic device of claim 1, wherein the one or more light sources are actuatable to generate a first beam and a second beam to extend over the monitored region, wherein the first beam and the second beam are directed to converge towards one another beyond the monitored region.

5. The electronic device of claim 1, wherein:

the one or more light sources are actuatable to generate at least a first beam and a second beam, the first beam and the second beam fanning across the monitored object;

the light detecting device is positioned to capture the image containing a first portion of the light pattern corresponding to the object intersecting the first beam, and a second portion of the light pattern corresponding to the object intersecting the second beam; and

the processor is configured to identify a gap distance on the image between the first portion of the light pattern and the second portion of the light pattern.

6. The electronic device of claim 4, wherein the processor is configured to match the gap distance to one of a plurality of predetermined values, wherein each of the plurality of predetermined values are based at least partially on the depth of the first position.

7. The electronic device of claim 4, wherein in response to identifying the gap distance, the processor is configured to calculate the depth of the first position using the gap distance.

8. The electronic device of claim 1, wherein:

the one or more light sources are actuatable to generate a first beam to project across the monitored region; and

the processor is configured to identify a dimension of the image of the light pattern formed by the object intersecting first beam.

9. The electronic device of claim 7, wherein processor is configured to identify a height of the image of the light pattern formed by the object intersecting first beam.

10. The electronic device of claim 7, wherein the processor is configured to match the dimension of the image of the light pattern to the depth of the first position.

11. The electronic device of claim 7, wherein in response to identifying the dimension, the processor is configured to calculate the depth of the first position using the dimension of the image of the light pattern.

12. The electronic device of claim 1, wherein the processor is configured to use the light pattern captured by the light detecting device in order to determine when the object intersects the structured light to make contact with a surface of the monitored region at the first position.

13. The electronic device of claim 11, wherein the processor is configured to determine that the object made contact with the surface at the first position using the geometric characteristic of the light pattern captured by the light detecting device.

14. The electronic device of claim 11, wherein the processor is configured to interpret the depth of the first position as input.

15. The electronic device of claim 11, wherein the processor is configured to interpret a coordinate of the first position as input.

16. The electronic device of claim 1, further comprising:

a projector to provide a visual indication of an input value associated with each region in a plurality of regions of the monitored region, and

wherein the processor is configured to interpret the first position as the input value that is visual indicated on a region in the plurality of regions that contains the first position.

17. The electronic device of claim 15, wherein the projector provides an image of a set of keys, so that each key in the set of keys is visually identifiable.

18. The electronic device of claim 16, wherein the projector provides the image of a QWERTY style keyboard.

19. The electronic device of claim 11, wherein the light detecting device is configured to capture a plurality of images of the light pattern that forms on the object as the object is moved from the first position to a second position across the monitored region, and where the processor is configured to approximate the depth of the second position relative to where the image of the object is captured, and wherein the processor is configured to interpret the depth of at least one of the first position and the second position as input.

20. The electronic device of claim 19, wherein the processor is configured to approximate the depth of one or more intermediate points where the object is placed as the object is moved from the first position to the second position.

21. The electronic device of claim 20, wherein the processor is configured to cause a gesture to be displayed, the gesture corresponding to a formation interconnecting the first position, the second position, and the one or more intermediate points.

22. An electronic device for approximating a depth of an objects placement on a monitored region, the electronic device comprising:

processor configured to;

receive information from a light detecting device, including information from an image of a light pattern that is formed on the object as a result of the object intersecting a structured light when being placed at a first position in the monitored region;

identify a geometric characteristic of the image of the light pattern, wherein the geometric characteristic of the image is variable with a depth of the first position relative to where the light detecting device captures the image; and

approximate the depth of the first position based on the identified geometric characteristic.

23. The electronic device of claim 22, wherein the processor is configured to:

receive information identifying a first portion of the light pattern and a second portion of the light pattern, the first portion of the light pattern is formed by the object intersecting a first beam that is directed over the monitored area, and the second portion of the light pattern is formed by the object intersecting a second beam that is directed over the monitored area;

determine the depth of the first position from a gap distance between the first portion and the second portion.

24. The electronic device of claim 23, wherein the processor is configured to:

receive information in order to identify a dimension of the light pattern; and

determine the depth of the first position using the dimension of the light pattern.

25. The electronic device of claim 22, wherein the processor is configured to determine an input at least partially from the depth of the first position.

26. The electronic device of claim 22, wherein the processor is configured to:

receive information from the light detecting device that includes information from a plurality of images, each image of a light pattern formed on the object as a result of the object intersecting the structured light while being moved from the first position to a second position; and

identify the geometric characteristic of the image of each light pattern; and

approximate the depth of at least the first position and the second position.

27. The electronic device of claim 26, wherein the processor is configured to approximate the depth of one or more intermediate points between the first position and the second position using an image captured of the object at each of the one or more intermediate points.

28. The electronic device of claim 26, wherein the processor is configured to cause a gesture to be displayed, the gesture corresponding to a formation interconnecting the first position, the second position, and the one or more intermediate points.

29. The electronic device of claim 22, further comprising a connector to couple the processor to another device containing the light detecting device.

30. An electronic device comprising:

one or more light sources to project structured light over a monitored region, the one or more light sources including a first end that is distal to a surface upon which the monitored region is provided on; and

a light detecting device that is configured to capture an image of the object intersecting the structured light when being placed at a first position in the monitored region; and

a processor that is configured to use the image to determine a depth of the first position relative to where the light detecting device captures the image;

wherein the light detecting device is placed so that a center of the light detecting device is more proximate to the surface than the first end of the one or more light sources.

31. The electronic device of claim 30, wherein the light detecting device is configured to capture the image that includes a light pattern forming on the object as a result of the object intersecting the structured light.

32. The electronic device of claim 31, wherein the processor is configured to determine the depth of the first position using the image of the light pattern.

33. The electronic device of claim 30, wherein the one or more light sources are less than three centimeters from the surface.

34. The electronic device of claim 30, wherein the one or more light sources are between one and two centimeters from the surface.

35. The electronic device of claim 30, wherein the processor is configured to determine an input based at least partially on the depth of the first position.

36. The electronic device of claim 30, wherein:

the one or more light sources include a first light source for directing a first beam over the monitored region, and a second light source for directing a second beam over the monitored region;

wherein the first light source and second light source are positioned vertically so that one of the first light source and second light source is distal from the surface.

37. The electronic device of claim 30, further comprising:

a housing containing a front panel;

a lens for the light detecting device positioned on the front panel;

and wherein the one or more light sources direct the structured light from the front panel.

38. The electronic device of claim 30, wherein the processor is configured to use a geometric characteristic of the image to determine the depth.

39. The electronic device of claim 38, wherein the geometric characteristic of the image corresponds to a gap distance between a first portion of the image of the light pattern and a second portion of the image of the light pattern, the first portion being formed by the object intersecting a first beam from the one or more light sources, the second portion being formed by the object intersecting a second beam from the one or more light sources.

40. The electronic device of claim 30, further comprising:

a projector that is configured to project an image of a virtual input device from the front panel.