SYSTEM AND METHOD FOR GESTURE DETECTION AND FEEDBACK

• US 20110234543A1
• Filed: 03/25/2011
• Published: 09/29/2011
• Est. Priority Date: 03/25/2010
• Status: Active Grant

• Current Assignee: Malikie Innovations Limited (Key Patent Innovations Limited)
• Original Assignee: User Interface In Sweden Ab
• Inventors: Johansson, Karl-Anders, Haliburton, James, Gärdenfors, Dan
• Granted Patent: US 9,218,119 B2
• Time in Patent Office: Days
• US Class Current: 345/175
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/041 Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence

Abstract

A computer device with a sensor subsystem for detecting off-surface objects, that carries out continued processing of the position and shape of objects detected in the vicinity of the device, associates these positions and shapes with predetermined gesture states, determines if the object is transitioning between gesture states and provides feedback based on the determined transition between the gesture states.

Claims

1. A computing device, comprising:

a display;

a sensor subsystem for detecting off-surface objects;

memory having instructions stored on it which when run on a processor, causes the processor to perform the steps of;

detecting an off-surface object using the sensor subsystem;

determining a position and a shape of said detected off-surface object;

matching the determined position with a set of reference zones and the determined shape with a set of reference poses to thereby determine a gesture state;

detecting changes in the determined position and shape using the sensor subsystem;

matching said changes in the determined position with transitions between said reference zones and matching said changes in determined shapes with transitions between said reference poses to thereby determine a transition between gesture states; and

,

instructing a user interface to provide feedback based on the determined transition between gesture states.

2. The computing device of claim 1, further comprising a means for detecting user input.

3. The computing device of claim 2, wherein the processor performs the further step of:

setting the sensor subsystem to a powered-on state when the means for detecting a user input has detected a predetermined user input.

4. The computing device of claim 2, wherein the means for detecting a user input is a touch screen.

5. The computer device of claim 4, wherein the processor performs the further steps of:

detecting an on-surface gesture via the touch screen; and

,

determining a compound gesture using a combination of the detected on-surface gesture and an off-surface gesture determined using the sensor subsystem.

6. The computing device of claim 2, wherein the means for detecting user input is a hardware button.

7. The computing device of claim 1, wherein the feedback is at least one of graphical, auditory and haptic in nature.

8. The computing device of any preceding claim, wherein the feedback is a continuous feedback.

9. The computing device of claim 8, wherein the processor performs the further step of:

determining a degree of transition between gesture states and providing the continuous feedback based on said degree of transition.

10. The computing device of claim 9, wherein the degree of transition includes at least one of:

where in between the reference zones the determined positions are present, where in between the reference poses the determined shapes are, which reference zone the determined position is closest and which reference pose the determined shape is closest.

11. The computing device of claim 1, wherein the reference zones are represented as at least one of volumes in a three-dimensional space, points in a three-dimensional space, areas in a two-dimensional space and points in a two-dimensional space.

12. The computing device of claim 1, wherein the sensor subsystem includes at least one of a single camera and two cameras in stereoscopic configuration.

13. The computing device of claim 1, wherein the set of reference zones and the set of reference poses are changeable.

14. A computer-implemented method, for operating a computing device comprising a display, sensor sub system, processor and memory, comprising the steps of:

detecting an off-surface object using the sensor subsystem;

determining a position and a shape of said detected off-surface object;

matching the determined position with a set of reference zones and the determined shape with a set of reference poses to thereby determine a gesture state;

detecting changes in the determined position and shape using the sensor subsystem;

matching said changes in the determined position with transitions between said reference zones and matching said changes in determined shapes with transitions between said reference poses to thereby determine a transition between gesture states; and

,

instructing a user interface to provide feedback based on the determined transition between gesture states.

15. A computer program product having instructions which when run on a processor causes the processor to carry out the method of claim 14.

16. A computing device, comprising:

a display;

a sensor subsystem for detecting off-surface objects;

a means for detecting user input;

memory having instructions stored on it which when run on a processor, causes the processor to perform the steps of;

detecting an off-surface object using the sensor subsystem;

determining a position and a shape of said detected off-surface object;

matching the determined position with a set of reference zones and the determined shape with a set of reference poses to thereby determine a gesture state;

setting the sensor subsystem to a powered-on state when the means for detecting a user input has detected a predetermined user input.

17. A computer-implemented method for use in a computing device comprising a display, a sensor subsystem, a means for detecting user input, a processor and memory, comprising the steps of:

detecting an off-surface object using the sensor subsystem;

determining a position and a shape of said detected off-surface object;

matching the determined position with a set of reference zones and the determined shape with a set of reference poses to thereby determine a gesture state;

setting the sensor subsystem to a powered-on state when the means for detecting a user input has detected a predetermined user input.

18. A computer program product having instructions which when run on a processor causes the processor to carry out the method of claim 17.