Method and apparatus for approximating depth of an object's placement onto a monitored region with applications to virtual interface devices

• US 7,050,177 B2
• Filed: 09/17/2002
• Issued: 05/23/2006
• Est. Priority Date: 05/22/2002
• Status: Expired due to Fees

• Current Assignee: Heilongjiang Golden Jumping Group Co., Ltd.
• Original Assignee: Canesta Incorporated (Microsoft Corporation)
• Inventors: Gokturk, Salih Burak, Tomasi, Carlo
• Primary Examiner(s): Pham, Hoa Q.
• Application Number:
  US10/245,925
• Publication Number:
  US 20030218760A1
• Time in Patent Office: 1,344 Days
• Field of Search: 356614-623, 356/4.07, 345/157, 345173-183, 178/18.01, 178/18.02, 178/18.09, 250/221
• US Class Current: 356/614
• CPC Class Codes: G01S 17/06 Systems determining position data of a target
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1632 External expansion units, e.g. docking stations
• CPC Class Codes: G06F 1/1673 Arrangements for projecting a virtual keyboard
• CPC Class Codes: G06F 2200/1633 Protecting arrangement for the entire housing of the computer
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0426 tracking fingers with respect to a virtual keyboard projected or printed on the surface virtual keyboards on touch screens G06F3/04886
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus

Abstract

Structured light is directed across a monitored region. An image is captured of a light pattern that forms on the object as a result of the object intersecting the structured light when the object is placed at a first position in the monitored region. A geometric characteristic is identified of the image of the light pattern. The geometric characteristic is variable with a depth of the first position relative to where the image is captured. The depth of the first position is approximated based on the measured geometric characteristic.

Claims

1. A method for determining depth of an object'"'"'s placement onto a monitored region, the method comprising:

directing structured light across the monitored region, wherein the structured light is capable of creating a geometric characteristic when forming a light pattern on the object in the monitored region;

capturing an image of the light pattern that forms on the object as a result of the object intersecting the structured light when the object is placed at a first position in the monitored region;

identifying a geometric characteristic of the image of the light pattern, wherein the geometric characteristic of the image is variable with a depth of the first position relative to where the image is captured; and

approximating the depth of the first position based on the geometric characteristic.

2. The method of claim 1, wherein directing structured light over the monitored region includes generating one or more beams that extend across and fan over the monitored region.

3. The method of claim 1, wherein directing structured light across the monitored region includes generating a first beam and a second beam to extend over the monitored region, wherein the first beam and the second beam are parallel to one another.

4. The method of claim 3, wherein generating a first beam and a second beam to extend over the monitored region includes generating the first beam to be proximate to a surface of the monitored region and to be thicker than the second beam.

5. The method of claim 1, wherein directing structured light over the monitored region includes generating a first beam and a second beam to extend over the monitored region, wherein the first beam and the second beam are directed to converge towards one another beyond the monitored region.

6. The method of claim 1, wherein capturing an image of a light pattern that forms on the object includes detecting reflected light using light-sensitive pixels.

7. The method of claim 1, wherein identifying a geometric characteristic of the image of the light pattern includes measuring a dimension of the image of the light pattern.

8. The method of claim 1, wherein:

directing structured light across includes directing at least a first beam and a second beam across the monitored object;

capturing an image of a light pattern that forms on the object includes capturing the image of a first portion of the light pattern and a second portion of the light pattern, the first portion of the light pattern being formed by the object intersecting the first beam, the second portion of the light pattern being formed by the object intersecting the second beam; and

identifying a geometric characteristic of the image of the light pattern includes measuring a gap distance between the first portion of the light pattern and the second portion of the light pattern.

9. The method of claim 8, wherein approximating the depth of the first position includes mapping the gap distance to one value from a set of values, wherein each value in the set of values corresponds to a range of the measured gap distance.

10. The method of claim 1, wherein:

directing structured light across the monitored region includes generating a single beam to extend over the monitored region;

capturing an image of a light pattern that forms on the object includes capturing a light pattern formed by the object intersecting the single beam; and

identifying a geometric characteristic of the image of the light pattern includes measuring a dimension of the light pattern formed by the single beam.

11. The method of claim 10, wherein identifying a geometric characteristic of the image of the light pattern includes measuring a height of the image of the light, wherein the height of the image of the light pattern is in a direction that is substantially orthonormal to a direction of the depth of the object'"'"'s placement.

12. The method of claim 11, wherein approximating the distance based on the geometric characteristic includes mapping the height of the image of the light pattern to one value from a set of values, wherein each value in the set of values corresponds to a range of the measured height.

13. The method of claim 1, wherein the method further comprises:

determining that the object intersects the structured light and makes contact with a surface of the monitored region at the first position using the light pattern formed on the object intersecting the structured light.

14. The method of claim 1, wherein the method further comprises:

determining that the object intersects the structured light and makes contact with a surface of the monitored region at the first position using the geometric characteristic of the image of the light pattern formed on the object intersecting the structured light.

15. A method for using depth of an object'"'"'s placement onto a monitored region as input, the method comprising:

directing structured light across the monitored region, wherein the structured light is capable of creating a geometric characteristic when forming a light pattern on the object in the monitored region;

capturing an image of a light pattern that forms on the object at a first position as a result of the object intersecting the structured light;

measuring one or more geometric characteristics of the image of the light pattern, wherein the one or more geometric characteristics of the image are variable with a depth of the first position relative to where the image is captured;

approximating the depth of the first position based on the measured geometric characteristic; and

determining an input corresponding at least partially to the depth of the first position.

16. The method of claim 15, further comprising determining at least two coordinates for the first position, wherein at least a first coordinate is determined from the determined depth of the first object.

17. The method of claim 15, wherein capturing an image of a light pattern includes capturing a plurality of images, each image being of the plurality of images including a light pattern formed on the object by the object being moved from the first position on the monitored region to at least a second position on the monitored region.

18. The method of claim 15, where the method further comprises:

providing a plurality of identifiable regions on the monitored region, each identifiable region being assigned an input value.

19. The method of claim 18, wherein:

capturing an image of a light pattern that forms on the object includes capturing the image that forms on the object when the object intersects the structured light substantially at one region in the plurality of identifiable regions; and

determining an input corresponding to the first position includes;

determining the identifiable region in the plurality of regions where the object intersects the structured light, and

identifying the input value assigned to the determined identifiable region.

20. The method of claim 19, wherein identifying the input value assigned to the determined identifiable region includes:

assigning an input value to each identifiable region in the plurality of identifiable regions; and

matching the determined identifiable region to the input value assigned to that identifiable region.

21. The method of claim 18, wherein providing a plurality of identifiable regions includes providing a virtual image of the plurality of identifiable regions.

22. The method of claim 21, wherein providing a plurality of identifiable regions includes visually indicating the input value assigned to one or more of the identifiable regions on the virtual image.

23. The method of claim 18, wherein providing a plurality of identifiable regions on the monitored region includes directing another light source to illuminate a set of keys, wherein each key in the set of keys corresponds to one of the identifiable regions in the plurality of identifiable regions.

24. The method of claim 23, wherein directing another light source to illuminate a set of keys includes illuminating an alphanumeric keypad.

25. The method of claim 23, wherein directing another light source to illuminate a set of keys includes illuminating a QWERTY keyboard.

26. The method of claim 18, wherein determining an input corresponding at least partially to the depth of the first position includes determining a first input value based at least partially on the depth of the first position, and determining a second input value based at least partially on a depth of the second position relative to where the image of the light pattern is captured.

27. The method of claim 26, further comprising determining an input value for one or more of the plurality of points that are crossed by the object as the object is moved from the first position to the second position.

28. The method of claim 15, wherein capturing an image of a light pattern that forms on the object includes capturing the image of the object at a plurality of points that are crossed by the object as the object is moved from the first to a second position on the monitored region.

29. The method of claim 15, wherein

directing structured light across the monitored region includes directing at least a first beam and a second beam across the monitored object;

and

capturing an image of a light pattern that forms on the object includes capturing a first portion of the light pattern formed by the object intersecting the first beam, and capturing a second portion of the light pattern formed by the object intersecting the second beam;

measuring one or more geometric characteristics of the image of the light pattern includes measuring a gap distance between the first portion and the second portion of the light pattern.

30. The method of claim 15, wherein:

directing structured light across the monitored region includes generating a first beam to project across the monitored region; and

measuring one or more geometric characteristics of the image of the light pattern includes measuring a dimension of the light pattern formed by the object intersecting the first beam.

31. A method for determining depth of an object'"'"'s placement onto a monitored region, the method comprising:

directing structured light across the monitored region, the structured light comprising a first beam and a second beam, wherein the structured light is capable of creating a geometric characteristic when forming a light pattern on the object in the monitored region;

capturing an image of a light pattern that forms on the object as a result of the object intersecting the structured light when the object is placed at a first position in the monitored region, the image of the light pattern including a first portion corresponding to where the object intersected the first beam, and a second portion corresponding to where the object intersected the second beam;

identifying a gap distance on the image between the first portion and second portion; and

approximating the depth of the first position based on the gap distance.

32. The method of claim 31, wherein capturing an image of a light pattern that forms on the object includes detecting reflected light using light sensitive pixels.

33. The method of claim 31, wherein approximating the depth of the first position includes mapping the gap distance to one value from a set of values, wherein each value in the set of values corresponds to a range of the gap distance.

34. The method of claim 31, wherein directing structured light across the monitored region includes generating the first beam to be proximate to a surface of the monitored region and to be thicker than the second beam.

35. The method of claim 31, wherein the method further comprises:

determining that the object intersects the structured light and makes contact with a surface of the monitored region at the first position using the tight pattern formed on the object intersecting the structured light.

36. The method of claim 31, wherein the method further comprises:

determining that the object intersects the structured light and makes contact with a surface of the monitored region at the first position using the gap distance identified from the image.

37. A method for determining depth of an object'"'"'s placement onto a monitored region, the method comprising:

directing structured light across the monitored region, the structured light including a first beam, wherein the structured light is capable of creating a geometric characteristic when forming a light pattern on the object in the monitored region;

capturing an image of a light pattern that forms on the object as a result of the object intersecting the structured light when the object is placed at a first position in the monitored region;

identifying a dimension of the image of the light pattern; and

approximating the depth of the first position based on the dimension.

38. The method of claim 37, wherein capturing an image of a light pattern that forms on the object includes detecting reflected light using light-sensitive pixels.

39. The method of claim 37, wherein:

directing structured light across the monitored region includes generating a single beam to extend over the monitored region;

capturing an image of a light pattern that forms on the object includes capturing a light pattern formed by the object intersecting the single beam; and

identifying a dimension of the image of the light pattern includes measuring a dimension of the light pattern formed by the single beam.

40. The method of claim 39, measuring a dimension of the image of the light pattern includes measuring a height of the image of the light, wherein the height of the image of the light pattern is in a direction that is substantially orthonormal to a direction of the depth of the object'"'"'s placement.

41. The method of claim 40, wherein approximating the distance based on the dimension includes mapping the height of the image of the light pattern to one value from a set of values, wherein each value in the set of values corresponds to a range of the measured height.

42. The method of claim 37, wherein the method further comprises:

determining that the object intersects the structured light and makes contact with a surface of the monitored region at the first position using the light pattern formed on the object intersecting the structured light.

43. The method of claim 37, wherein the method further comprises:

determining that the object intersects the structured light and makes contact with a surface of the monitored region at the first position using the dimension of the image of the light pattern formed on the object intersecting the structured light.