Interaction interface for controlling an application

• US 8,068,641 B1
• Filed: 06/19/2008
• Issued: 11/29/2011
• Est. Priority Date: 06/19/2008
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Hildreth, Evan
• Primary Examiner(s): Johns; Andrew W
• Assistant Examiner(s): GORADIA, SHEFALI DINESH
• Application Number:
  US12/142,455
• Time in Patent Office: 1,258 Days
• Field of Search: 382/103, 348/162, 356/614, 250/559.11, 250/203.1, 250/559.29, 250/559.38
• US Class Current: 382/103
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: G01S 17/42 Simultaneous measurement of distance and other co-ordinates indirect measurement G01S17/46
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 7/4815 using multiple transmitters
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen

Abstract

Image sequences are accessed, the sequences each taken, from a different perspective, of a portion of a radiation region defined by projected electromagnetic radiation. The image sequences each include a projection image taken during an emitting period in which the projected electromagnetic radiation is emitted, the projection image being of ambient electromagnetic radiation and of an object within the portion being illuminated with the projected electromagnetic radiation, and an ambient image taken during an extinguishing period in which the projected electromagnetic radiation is extinguished, the ambient image being of the ambient electromagnetic radiation. A position of the object using the projection images and the ambient images is determined, an application is controlled based on the determined position.

Claims

1. A computer-implemented method comprising:

accessing image sequences, the sequences each taken, from a different perspective, of a portion of a radiation region defined by projected electromagnetic radiation, and each image sequence including;

a projection image taken during an emitting period in which the projected electromagnetic radiation is emitted, the projection image being of ambient electromagnetic radiation and of an object within the portion being illuminated with the projected electromagnetic radiation, and

an ambient image taken during an extinguishing period in which the projected electromagnetic radiation is extinguished, the ambient image being of the ambient electromagnetic radiation;

determining a position of the object using the projection images and the ambient images; and

controlling an application based on the determined position.

2. The method of claim 1 further comprising alternating between, in the emitting and extinguishing periods, emitting and extinguishing the projected electromagnetic radiation.

3. The method of claim 1 further comprising generating modified images based upon modifying each projection image with a corresponding ambient image, wherein the position is determined using the modified images.

4. The method of claim 3, wherein generating the modified images comprises subtracting the corresponding ambient image from each projection image.

5. The method of claim 4, further comprising:

determining whether the modified images include pixel values less than zero, and

if the modified images include pixels values less than zero, setting any pixel value less than zero to zero.

6. The method of claim 1, wherein the projection image is of the ambient electromagnetic radiation, of the object within the portion being illuminated with the projected electromagnetic radiation, and of a reflection of a background illuminated by the projected electromagnetic radiation.

7. The method of claim 6, further comprising:

modeling the reflection of the background; and

modifying the projected image based on the modeled reflection.

8. The method of claim 1 further comprising compensating the ambient images to account for illumination of the object by the ambient electromagnetic radiation.

9. The method of claim 8, wherein the ambient light comprises unobscured ambient light not obscured by the object.

10. The method of claim 1, wherein each image sequences comprises at least a pair of images, the pair of images comprising the projection image and the ambient image.

11. The method of claim 10, wherein the ambient image is taken during an extinguishing period that immediately proceeds or immediately follows the emitting period during which the projected image is taken.

12. The method of claim 1, wherein determining the position comprises determining two-dimensional or three-dimensional coordinates representing the position relative to an edge of the region.

13. The method of claim 1, further comprising outputting the determined position to the application.

14. The method of claim 1, further comprising determining a motion of the object based on the determined position, where the application is controlled based on the determined motion.

15. The method of claim 1, wherein determining the position of the object using the projection images and the ambient images comprises:

determining a first position of the object using the projection images and the ambient images from a first sequence taken from the first perspective,

determining a second position of the object using the projection images and the ambient images from a second sequence taken from the second perspective, and

combining the first position of the object and the second position of the object.

16. The method of claim 15, wherein the first and second positions are combined using triangulation.

17. The method of claim 1, wherein the projected electromagnetic radiation comprises infrared light.

18. The method of claim 1, further comprising cropping each projection image and ambient image, wherein the position of the object is determined using the cropped projection images and ambient images.

19. The method of claim 1, further comprising a defining a region of interest within the radiation region.

20. The method of claim 19, wherein the position is expressed as a percentage of a height of the region of interest and a percentage of a width of the region of interest.

21. A device comprising:

a processor configured to;

access image sequences, the sequences each taken, from a different perspective, of a portion of a radiation region defined by projected electromagnetic radiation, and each image sequence including;

a projection image taken during an emitting period in which the projected electromagnetic radiation is emitted, the projection image being of ambient electromagnetic radiation and of an object within the portion being illuminated with the projected electromagnetic radiation, and

an ambient image taken during an extinguishing period in which the projected electromagnetic radiation is extinguished, the ambient image being of the ambient electromagnetic radiation,

determine a position of the object using the projection images and the ambient images, and

control an application based on the determined position; and

a memory configured to;

store the image sequences, and

transfer the image sequences to the processor.

22. A system comprising:

an interaction interface configured to control an application based on a position of an object, the interaction interface comprising;

a processor configured to;

access image sequences, the sequences each taken, from a different perspective, of a portion of a radiation region defined by projected electromagnetic radiation, and each image sequence including;

a projection image taken during an emitting period in which the projected electromagnetic radiation is emitted, the projection image being of ambient electromagnetic radiation and of an object within the portion being illuminated with the projected electromagnetic radiation, and

an ambient image taken during an extinguishing period in which the projected electromagnetic radiation is extinguished, the ambient image being of the ambient electromagnetic radiation, and

determine a position of the object using the projection images and the ambient images;

an application interface configured to output the position; and

a memory configured to;

store the image sequences, and

transfer the image sequences to the processor; and

a device executing the application, the device receiving the position and transferring the position to the application such that the application reacts based on the position.

23. A non-transitory computer-readable medium encoded with a computer program product comprising instructions that, when executed, operate to cause a computer to perform a method comprising:

accessing image sequences, the sequences each taken, from a different perspective, of a portion of a radiation region defined by projected electromagnetic radiation, and each image sequence including;

a projection image taken during an emitting period in which the projected electromagnetic radiation is emitted, the projection image being of ambient electromagnetic radiation and of an object within the portion being illuminated with the projected electromagnetic radiation, and

an ambient image taken during an extinguishing period in which the projected electromagnetic radiation is extinguished, the ambient image being of the ambient electromagnetic radiation;

determining a position of the object using the projection images and the ambient images; and

controlling an application based on the determined position.