SURFACE UI FOR GESTURE-BASED INTERACTION

• US 20100027843A1
• Filed: 06/24/2009
• Published: 02/04/2010
• Est. Priority Date: 08/10/2004
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Wilson, Andrew D.
• Application Number:
  US12/490,327
• Publication Number:
  US 20100027843A1
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Disclosed is a unique system and method that facilitates gesture-based interaction with a user interface. The system involves an object sensing configured to include a sensing plane vertically or horizontally located between at least two imaging components on one side and a user on the other. The imaging components can acquire input images taken of a view of and through the sensing plane. The images can include objects which are on the sensing plane and/or in the background scene as well as the user as he interacts with the sensing plane. By processing the input images, one output image can be returned which shows the user objects that are in contact with the plane. Thus, objects located at a particular depth can be readily determined. Any other objects located beyond can be “removed” and not seen in the output image.

Claims

1. A method of determining a command, comprising:

capturing an image of an object with a camera;

determining a gesture based at least partly upon the image;

detecting an audio input; and

determining, at one or more processors, the command based at least partly upon the gesture or the audio input.

2. The method of claim 1, further comprising:

determining a depth of the object; and

determining the command based at least partly upon the depth of the object.

3. The method of claim 2, wherein determining the depth of the object includes capturing a second image of the object with a second camera.

4. The method of claim 1, wherein the camera is a video camera.

5. The method of claim 1, wherein the camera detects infrared light.

6. The method of claim 1, wherein determining the gesture includes capturing a second image of the object with the camera and comparing the image with the second image.

7. A computer-readable medium having instruction that cause a processor to execute steps, the steps comprising:

capturing an image of an object with a camera;

determining a gesture based at least partly upon the image;

detecting an audio input; and

determining, at one or more processors, a command based at least partly upon the gesture or the audio input.

8. The computer-readable medium of claim 7, the steps further comprising:

determining a depth of the object; and

determining the command based at least partly upon the depth of the object.

9. The computer-readable medium of claim 8, wherein determining the depth of the object includes capturing a second image of the object with a second camera.

10. The computer-readable medium of claim 7, wherein the camera is a video camera.

11. The computer-readable medium of claim 7, wherein the camera detects infrared light.

12. The computer-readable medium of claim 7, wherein determining the gesture includes capturing a second image of the object with the camera and comparing the image with the second image.

13. A command determining system, comprising:

a camera configured to capture an image of an object;

a first determiner configured to determine a gesture based at least partly upon the image;

an audio detection unit configured to detect an audio input; and

a second determiner configured to determine the command based at least partly upon the gesture or the audio input.

14. The command determining system of claim 13, further comprising:

a third determiner configured to determine a depth of the object, wherein the second determiner is further configured to determine the command based at least partly upon the depth of the object.

15. The command determining system of claim 14, wherein determining the depth of the object includes capturing a second image of the object with a second camera.

16. The command determining system of claim 13, wherein the camera is a video camera.

17. The command determining system of claim 13, wherein the camera detects infrared light.

18. The command determining system of claim 13, wherein determining the gesture includes capturing a second image of the object with the camera and comparing the image with the second image.