User Input Using Proximity Sensing

• US 20090139778A1
• Filed: 11/30/2007
• Published: 06/04/2009
• Est. Priority Date: 11/30/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Izadi, Shahram, Butler, David Alexander, Hodges, Stephen E., Hall, Malcolm
• Granted Patent: US 8,432,372 B2
• Time in Patent Office: Days
• US Class Current: 178/18.30
• CPC Class Codes: G06F 1/1626 with a single-body enclosure integrating a flat display, e.g. Personal Digital Assistants [PDAs]
• CPC Class Codes: G06F 1/1684 Constructional details or arrangements related to integrated I/O peripherals not covered by groups G06F1/1635 - G06F1/1675
• CPC Class Codes: G06F 1/169 the I/O peripheral being an integrated pointing device, e.g. trackball in the palm rest area, mini-joystick integrated between keyboard keys, touch pads or touch stripes G06F1/1643 takes precedence; constructional details of pointing devices G06F3/033
• CPC Class Codes: G06F 2203/0339 Touch strips, e.g. orthogonal touch strips to control cursor movement or scrolling; single touch strip to adjust parameter or to implement a row of soft keys
• CPC Class Codes: G06F 2203/04106 Multi-sensing digitiser, i.e. digitiser using at least two different sensing technologies simultaneously or alternatively, e.g. for detecting pen and finger, for saving power or for improving position detection
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/03547 Touch pads, in which fingers can move on a surface
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/042 by opto-electronic means
• CPC Class Codes: G06F 3/0446 using a grid-like structure of electrodes in at least two directions, e.g. using row and column electrodes
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

A device is described which enables users to interact with software running on the device through gestures made in an area adjacent to the device. In an embodiment, a portable computing device has proximity sensors arranged on an area of its surface which is not a display, such as on the sides of the device. These proximity sensors define an area of interaction adjacent to the device. User gestures in this area of interaction are detected by creating sensing images from data received from each of the sensors and then analyzing sequences of these images to detect gestures. The detected gestures may be mapped to particular inputs to a software program running on the device and therefore a user can control the operation of the program through gestures.

Claims

1. A device comprising:

a plurality of proximity sensing points arranged on a non-display portion of the device, the plurality of proximity sensing points defining an interaction area adjacent to the device;

a processor; and

a memory arranged to store executable instructions which when executed by the processor, cause the processor to;

create a sequence of proximity sensor images from data received from each of the plurality of proximity sensing points that is operational; and

analyze the sequence of proximity sensor images to identify a user gesture in the interaction area.

2. A device according to claim 1, wherein the memory is further arranged to store executable instructions which when executed by the processor, cause the processor to:

control a software program running on the device based on the identified gesture.

3. A device according to claim 1, wherein the memory is further arranged to store executable instructions which when executed by the processor, cause the processor to:

map the identified gesture to a control input to a software program running on the device.

4. A device according to claim 1, wherein each proximity sensing point comprises a proximity sensor.

5. A device according to claim 4, wherein each proximity sensor comprises an emitter and a detector.

6. A device according to claim 5, wherein each emitter is modulated and data received at each detector is filtered.

7. A device according to claim 1, further comprising:

at least one emitter;

at least one detector; and

a plurality of waveguides,

and wherein each proximity sensing point is optically connected to a detector by one of the plurality of waveguides and each proximity sensing point is optically connected to an emitter by one of the plurality of waveguides.

8. A device according to claim 1, wherein each proximity sensor image comprises a map of intensities detected at each of the plurality of proximity sensing points.

9. A device according to claim 1, wherein the memory is further arranged to store executable instructions which when executed by the processor, cause the processor to:

control which of the plurality of sensing points are operational in order to control one of more of the resolution and sensitivity of gesture detection in the interaction area.

10. A device according to claim 1, wherein analyzing the sequence of proximity sensing images comprises:

subtracting a background image from each image to create a processed version of each image; and

analyzing a sequence of processed versions of images to detect a gesture.

11. A device according to claim 1, wherein analyzing the sequence of proximity sensing images comprises:

analyzing one of the sequence of proximity sensing images; and

determining a position of a sensed object from the one of the sequence of proximity sensing images.

12. A device according to claim 1, wherein the non-display portion of the device comprises one or more sides of the device.

13. A device according to claim 1, wherein the device comprises a portable computing device.

14. A device according to claim 1, wherein the device comprises a cover for a portable computing device.

15. One or more tangible device-readable media with device-executable instructions for performing steps comprising:

collecting data from a plurality of proximity sensing points arranged on a surface of a device;

generating a sequence of proximity sensing images from the data; and

analyzing the sequence of proximity sensing images to detect a gesture.

16. One or more tangible device-readable media according to claim 15 further comprising device-executable instructions for performing steps comprising:

mapping the gesture to an input to software running on the device.

17. One or more tangible device-readable media according to claim 16, further comprising device-executable instructions for performing steps comprising:

controlling the software according to the input.

18. One or more tangible device-readable media according to claim 15, wherein analyzing the sequence of proximity sensing images comprises:

subtracting a background image from each image to create a processed version of each image; and

analyzing a sequence of processed versions of images to detect a gesture.

19. One or more tangible device-readable media according to claim 15, wherein each proximity sensing point comprises an emitter and a detector and wherein the emitter is modulated, and wherein collecting data from a plurality of proximity sensing points comprises:

filtering a signal received at each detector to produce filtered data; and

collecting filtered data from each detector.

20. A portable computing device comprising:

a plurality of proximity sensors arranged in a pattern along a side of the device;

an interaction zone adjacent to the side of the device, wherein the interaction zone is defined by the pattern of sensors; and

processing apparatus arranged to;

collect data each sensor which is operational;

generate a sequence of proximity sensing images from the data; and

analyze the sequence of proximity sensing images to detect a gesture.