Virtual Touch Interface

• US 20110298708A1
• Filed: 06/07/2010
• Published: 12/08/2011
• Est. Priority Date: 06/07/2010
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Hsu, Feng-Hsiung, Zhang, Chunhui
• Application Number:
  US12/795,024
• Publication Number:
  US 20110298708A1
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 1/169 the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes G06F1/1643 takes precedence; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 1/1692 the I/O peripheral being a secondary touch screen used as control interface, e.g. virtual buttons or sliders
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0213 Arrangements providing an integrated pointing device in a keyboard, e.g. trackball, mini-joystick for pointing devices integrated in a laptop computer G06F1/169; joysticks G05G9/047; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A user may issue commands to a computing device by moving a pointer within a light field. Sensors may capture light reflected from the moving pointer. A virtual touch engine may analyze the reflected light captured as light portions in a sequence of images by the sensors to issue a command to a computing device in response to the movements. Analyzing the sequence of images may include finding the light portions in the sequence of images, determining a size of the light portions, and determining a location of the light portions.

Claims

1. A computer-implemented method comprising:

emitting light from a source to generate a light field parallel to a work surface;

capturing light at one or more sensors positioned outside the light field as a sequence of images, the light reflected from a pointer positioned within the light field;

analyzing the reflected light captured in the sequence of images to track a movement of the pointer; and

analyzing the tracked movement to issue a command to a computing device.

2. The computer-implemented method as recited in claim 1, wherein the generating a virtual interface comprises generating an infrared field via one or more infrared light-emitting diode (LED) bars, an aspect ratio of the infrared field substantially equivalent to an aspect ratio of a display device of the computing device.

3. The computer-implemented method as recited in claim 1, wherein the generating a virtual interface comprises generating an infrared field via one or more infrared laser diodes.

4. The computer-implemented method as recited in claim 1, wherein the command is one of a zoom command, a navigate command, and a rotate command to manipulate a user interface via the computing device.

5. The computer-implemented method as recited in claim 1, wherein the analyzing the reflected light analyzes a color intensity gradient of each image of the sequence of images to locate an edge of one or more light portions within each image of the sequence of images.

6. The computer-implemented method as recited in claim 1, wherein the one or more sensors include one or more infrared cameras.

7. The computer-implemented method as recited in claim 1, wherein the analyzing the reflected light in the sequence of images includes:

determining a vertical pixel location of the pointer for each image of the sequence of images;

determining a lateral pixel location of the pointer for each image of the sequence of images; and

triangulating an approach location based on the vertical pixel location and the lateral pixel location for each image of the sequence of images.

8. One or more computer-readable media storing computer-executable instructions that, when executed on one or more processors, causes the one or more processors to perform acts comprising:

receiving a sequence of images that capture a movement of a pointer in a light field as one or more light portions, the light portions indicative of a command issued to a computing device;

finding the one or more light portions in each image of the sequence of images;

determining a size of each of the one or more light portions in each image of the sequence of images;

determining a location of each of the one or more light portions in each image of the sequence of images to track the movement of the pointer;

translating the tracked movement to the command issued to the computing device; and

issuing the command to the computing device.

9. The one or more computer-readable media as recited in claim 8, wherein the determining the location of each of the one or more light portions in each image of the sequence of images further includes:

calculating a vertical pixel distance of each of the one or more light portions;

calculating a lateral pixel distance of each of the one or more light portions; and

triangulating an approach distance based on the vertical pixel distance and the lateral pixel distance.

10. The one or more computer-readable media as recited in claim 8, wherein the finding the one or more light portions utilizes an edge-based detection technique to find the one or more light portions.

11. The one or more computer-readable media as recited in claim 10, wherein the edge-based detection technique analyzes a color intensity gradient within each image of the sequence of images to find the one or more light portions.

12. The one or more computer-readable media as recited in claim 8, further comprising providing feedback based on the issued command, the feedback being one or more of changing an appearance of an object, displaying a temporary window describing the issued command, and outputting a voice command describing the issued command.

13. The one or more computer-readable media as recited in claim 8, further comprising omitting one or more light portions having the size outside a predetermined range of light portion sizes.

14. The one or more computer-readable media as recited in claim 8, wherein the issuing the command to the computing device includes issuing a multi-touch command to the computing device.

15. The one or more computer-readable media as recited in claim 14, wherein the issuing the multi-touch command to the computing device includes issuing one of a zooming and rotating command to the computing device.

16. A virtual touch interface system comprising:

one or more processors; and

memory to store modules executable by the one or more processors, the modules comprising;

an interface module to generate an infrared light field;

a tracking module to analyze a moving pointer within a light field captured in a sequence of images by a sensor positioned outside the light field, the tracking module analyzing the moving pointer to;

(1) locate one or more light portions in each image of the sequence of images, (2) track a movement of the moving pointer based on the located one or more light portions, and (3) translate the tracked movement to a command; and

a command module to issue the command to the computing device.

17. The virtual touch interface system as recited in claim 16, wherein the tracking module further determines a size of each of the one or more light portions.

18. The method as recited in claim 16, wherein the locating one or more light portions locates the one or more light portions as a vertical pixel distance, a lateral pixel distance, and an approach distance.

19. The method as recited in claim 16, wherein the command module issues a multi-touch command to the computing device.

20. The method as recited in claim 16, wherein the command module further provides feedback based on the issued command, the feedback being one or more of changing an appearance of an object, displaying a temporary window describing the issued command, and outputting a voice command describing the issued command.