Camera-based multi-touch mouse

• US 8,446,367 B2
• Filed: 04/17/2009
• Issued: 05/21/2013
• Est. Priority Date: 04/17/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Benko, Hrvoje, Rosenfeld, Daniel Allen, Ofek, Eyal, Chen, Billy, Izadi, Shahram, Villar, Nicolas, Helmes, John
• Primary Examiner(s): Bolotin, Dmitriy
• Application Number:
  US12/425,408
• Publication Number:
  US 20100265178A1
• Time in Patent Office: 1,495 Days
• Field of Search: 345163-166, 345/156, 345173-178, 178/18.01, 178/18.09, 178/18.11
• US Class Current: 345/166
• CPC Class Codes: G06F 2203/04104 Multi-touch detection in digitiser, i.e. details about the simultaneous detection of a plurality of touching locations, e.g. multiple fingers or pen and finger
• CPC Class Codes: G06F 3/03543 Mice or pucks G06F3/03541 takes precedence
• CPC Class Codes: G06F 3/042 by opto-electronic means

Abstract

Technologies for a camera-based multi-touch input device operable to provide conventional mouse movement data as well as three-dimensional multi-touch data. Such a device is based on an internal camera focused on a mirror or set of mirrors enabling the camera to image the inside of a working surface of the device. The working surface allows light to pass through. An internal light source illuminates the inside of the working surface and reflects off of any objects proximate to the outside of the device. This reflected light is received by the mirror and then directed to the camera. Imaging from the camera can be processed to extract touch points corresponding to the position of one or more objects outside the working surface as well as to detect gestures performed by the objects. Thus the device can provide conventional mouse functionality as well as three-dimensional multi-touch functionality.

Claims

1. A multi-touch input device comprising:

a base;

a working surface with an inside and an outside, the working surface disposed over the base forming a cavity between the base and the inside of the working surface;

a light source integral to the multi-touch input device;

a curved mirror disposed in the cavity;

a camera focused on the curved mirror;

a mechanism operable to provide movement data corresponding to movement of the multi-touch input device; and

an interface configured to send the movement data and multi-touch data to a computer system via the interface,

wherein;

the curved mirror is operable to receive light reflected or scattered by multiple objects outside of the working surface onto the curved mirror and to further reflect the light into the camera,

the camera is operable to receive the light reflected from the curved mirror, and

the multi-touch input device is operable to process the light reflected or scattered by the multiple objects and reflected by the curved mirror to the camera to provide the multi-touch data that is sent to the computer system via the interface.

2. The multi-touch input device of claim 1 wherein the camera is operable to image all of the inside of the working surface that is unobscured to the curved mirror.

3. The multi-touch input device of claim 1 wherein the light source provides infrared light.

4. The multi-touch input device of claim 3 further comprising an infrared pass filter disposed between the camera and incoming light to the camera.

5. The multi-touch input device of claim 1 further comprising a light barrier operable to restrict the camera to imaging light reflected from the curved mirror.

6. The multi-touch input device of claim 1 wherein the working surface is a diffuse surface.

7. The multi-touch input device of claim 1, wherein the multi-touch input device is operable to sense a plurality of gestures simultaneously.

8. The multi-touch input device of claim 1, wherein the curved mirror is at least partly spherical.

9. The multi-touch input device of claim 8, wherein the curved mirror is a curved ¼

sphere.

10. The multi-touch input device according to claim 1, further comprising a projector configured to project a known pattern onto the inside of the working surface, wherein the multi-touch input device is operable to measure distortions in the known pattern to reveal areas of distortion on the working surface.

11. The multi-touch input device according to claim 1, further comprising a projector configured to project a user interface viewable from the outside of the working surface.

12. The multi-touch input device of claim 11, wherein the projector is configured to project the user interface in a visible spectrum while the light source is configured to project a non-visible wavelength of light for multi-touch sensing via the camera.

13. The multi-touch input device of claim 1, wherein the working surface includes an embedded pattern formed of material that is configured to reflect the light.

14. A device comprising:

a base;

a working surface with an inside and an outside, the working surface disposed over the base forming a cavity between the base and the inside of the working surface, wherein the working surface is a diffuse surface;

a curved mirror disposed within the cavity and above the base;

a camera focused on the curved mirror;

a light source of infrared light integral to the device, wherein the infrared light from the light source;

reflects off of multiple objects positioned outside the working surface,

is received by the curved mirror, and

is then reflected from the curved mirror to the camera;

an infrared pass filter operable to filter non-infrared light reflected by the curved mirror to the camera;

a mechanism operable to provide movement data corresponding to movement of the device; and

an interface configured to send the movement data and multi-touch data to a computer system via the interface,

wherein;

the camera is operable to receive the infrared light reflected by the curved mirror, and

the device is operable to process the infrared light reflected by the multiple objects and the curved mirror to the camera to provide the multi-touch data that is sent to the computer system via the interface.

15. The device of claim 14 wherein the device is operable to process an image of the inside of the working surface to extract a touch point corresponding to a position of an individual object positioned outside the working surface.

16. The device of claim 14 wherein the device is operable to process an image of the inside of the working surface to detect a gesture performed by one or more of the multiple objects positioned outside the working surface.

17. A multi-touch input device comprising:

a base;

a working surface with an inside and an outside, the working surface disposed over the base forming a cavity between the base and the inside of the working surface, wherein the working surface is a diffuse surface;

a toroidal mirror;

a conical mirror;

a camera disposed in the cavity and focused on the conical mirror;

a light source of light integral to the device, wherein light from the light source is scattered by an object in optical contact with the working surface;

a mouse mechanism operable to provide conventional mouse movement data corresponding to movement of the multi-touch input device; and

an interface configured to provide the conventional mouse movement data and multi-touch data to a computer system via the interface, the multi-touch data corresponding to the object;

wherein;

the toroidal mirror is operable to receive the light and reflect the light to the conical mirror;

the conical mirror is operable to reflect the light to the camera, and

the multi-touch input device is operable to process the light reflected by the object, the toroidal mirror, and the conical mirror to the camera to provide the multi-touch data that is sent to the computer system via the interface.

18. The multi-touch input device of claim 17, wherein the device is operable to process an image of the inside of the working surface to extract a touch point corresponding to a position of the object.

19. The multi-touch input device of claim 17, wherein the multi-touch input device is operable to process an image of the inside of the working surface to detect a gesture performed by the object.

20. The multi-touch input device of claim 17, wherein the multi-touch input device is operable to perform multi-touch functionality responsive to a plurality of objects disposed at multiple points on the outside of the working surface simultaneously.