Computer vision-based multi-touch sensing using infrared lasers

• US 8,952,894 B2
• Filed: 05/12/2008
• Issued: 02/10/2015
• Est. Priority Date: 05/12/2008
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Wilson, Andrew David
• Primary Examiner(s): Mengistu, Amare
• Assistant Examiner(s): ZUBAJLO, JENNIFER L
• Application Number:
  US12/118,955
• Publication Number:
  US 20090278799A1
• Time in Patent Office: 2,465 Days
• Field of Search: 345156-175, 400/472, 400/489
• US Class Current: 345/158
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/0428 by sensing at the edges of the touch surface the interruption of optical paths, e.g. an illumination plane, parallel to the touch surface which may be virtual sensing beam interruptions in a planar beam grid of an optical touch-screen G06F3/0421

Abstract

The claimed subject matter provides a system and/or a method that facilitates detecting a plurality of inputs simultaneously. A laser component can be coupled to a line generating (LG) optic that can create a laser line from an infrared (IR) laser spot, wherein the laser component and line generating (LG) optic emit a plane of IR light. A camera device can capture a portion of imagery within an area covered by the plane of light. The camera device can be coupled to an IR-pass filter that can block visible light and pass IR light in order to detect a break in the emitted plane of IR light. An image processing component can ascertain a location of the break within the area covered by the emitted plane of IR light.

Claims

1. A system, comprising:

a laser component;

a line generating (LG) optic coupled to the laser component to emit a plane of light substantially parallel to a first side of a substantially flat surface;

a camera device mounted on the first side of the substantially flat surface to capture a portion of imagery within an area covered by the plane of light and to detect a break in the plane of light caused by a presence of an object located on the first side of the substantially flat surface; and

an image processing component to ascertain a location of the break within the area covered by the plane of light, the image processing component further configured to calculate at least one of a size of the break or a shape of the break and to display a corresponding shape or size of the break at a corresponding location on a display for interaction with data presented on the display at the corresponding location where the corresponding shape or size of the break is displayed, the corresponding location on the display comprising a mapped coordinate representative of the location of the break.

2. The system of claim 1, wherein the interaction with the data presented on the display is based at least in part on the shape of the break, and wherein the shape of the break corresponds to a pattern associated with the interaction, and wherein the interaction with the data presented on the display comprises at least one of highlighting the data presented on the display or displaying an image or video on the display at the corresponding location.

3. The system of claim 1, wherein the substantially flat surface comprises the display.

4. The system of claim 1, further comprising a plurality of laser components with at least one LG optic coupled thereto, the plurality of laser components are oriented to emit the plane of light.

5. The system of claim 1, wherein the plane of light comprises infrared (IR) light, the system further comprising an IR pass filter coupled to the camera device to block visible light and to pass IR light permitting the image processing component to detect the break in the IR light.

6. The system of claim 5, further comprising an IR modulation component to compensate for a portion of IR light generated by a source other than the laser component and the LG optic.

7. The system of claim 6, wherein the IR modulation component is further configured to control the laser component with a sequence of an “

OFF”

command and an “

ON”

command while the camera device captures the following;

a first collection of IR light including the portion of IR light generated by the source other than the laser component and the LG optic during the “

OFF”

command; and

a second collection of IR light including the plane of IR light and the portion of IR light generated by the source other than the laser component and the LG optic during the “

ON”

command.

8. The system of claim 7, wherein the IR modulation component is further configured to identify the portion of IR light generated by the source other than the laser component and the LG optic by subtracting the second collection of IR light and the first collection of IR light, the portion of IR light generated by the source other than the laser component and the LG optic is filtered out from the camera device.

9. The system of claim 1, wherein the image processing component is further configured to detect touch events based at least in part on the break.

10. The system of claim 1, further comprising an application framework that leverages the location of the break for at least one of a data input or data interaction.

11. The system of claim 10, wherein the application framework is further configured to utilize surface computing techniques with a touch-sensitive screen in which touch detection is provided by the break.

12. The system of claim 1, further comprising an application framework to cause a video to be played in response to recognizing the size of the break as a defined size or recognizing the shape of the break as a particular shape.

13. A method, comprising:

generating, using a laser component and a line generating (LG) optic, a plane of infrared (IR) light substantially parallel to a first side of a substantially flat surface via a sequence of an “

OFF”

command and an “

ON”

command to the laser component while a camera device mounted on the first side of the substantially flat surface captures the following;

a first collection of IR light including a portion of IR light generated by a source other than the laser component and the LG optic during the “

OFF”

command; and

a second collection of IR light including the plane of IR light and the portion of IR light generated by the source other than the laser component and the LG optic during the “

ON”

command;

identifying the portion of IR light generated by the source other than the laser component and the LG optic by calculating the difference between the first collection of IR light and the second collection of IR light;

detecting a break in the plane of IR light caused by a presence of an object located on the first side of the substantially flat surface;

calculating a location of the break within an area covered by the plane of IR light;

ascertaining at least one of a size of the break or a shape of the break; and

mapping the location and the size or the shape of the break to a display such that a corresponding size or shape of the break is displayed at a corresponding location on the display resulting from the mapping for interaction with data presented on the display at the corresponding location on the display where the corresponding size or shape is displayed.

14. The method of claim 13, further comprising:

detecting a plurality of sequential breaks in the plane of IR light;

calculating a plurality of locations corresponding to the plurality of sequential breaks; and

tracking the plurality of locations.

15. The method of claim 13, further comprising:

receiving the location, size, and shape of the break with an application framework; and

utilizing the application framework to interact with data based at least in part upon the break, the application framework employs at least one surface computing technique.

16. A method, comprising:

emitting, using a laser component and a line generating (LG) optic coupled to the laser component, a plane of light substantially parallel to a first side of a substantially flat surface;

capturing, using a camera device mounted on the first side of the substantially flat surface, a portion of imagery within an area covered by the plane of light;

detecting a break in the plane of light caused by a presence of an object located on the first side of the substantially flat surface;

determining, using an image processing component, a location of the break within the area covered by the plane of light;

calculating at least one of a size of the break or a shape of the break; and

displaying a corresponding shape or size of the break at a corresponding location on a display for interaction with data presented on the display at the corresponding location where the corresponding shape or size is displayed, the corresponding location on the display comprising a mapped coordinate representative of the location of the break.

17. The method of claim 16, further comprising:

determining a motion of the object by tracking a plurality of sequential breaks in the plane of light; and

determining a touch event based on the motion.

18. The method of claim 16, wherein the plane of light comprises infrared (IR) light, the method further comprising blocking, using an IR pass filter coupled to the camera device, visible light and passing IR light permitting the detecting the break in the IR light.

19. The method of claim 18, further comprising compensating, using an IR modulation component, for a portion of IR light that is generated by a source other than the laser component and the LG optic.

20. The method of claim 16, further comprising inferring a touch event using value of information (VOI) computing techniques, the VOI computing techniques comprising inferring a plurality of touch events as a result of a plurality of breaks detected by the camera and assigning a priority to each of the plurality of touch events relative to one another based at least in part on an association of each of the plurality of touch events with respective data interactions.