Interaction Using Touch and Non-Touch Gestures

• US 20100104134A1
• Filed: 10/29/2008
• Published: 04/29/2010
• Est. Priority Date: 10/29/2008
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Corporation
• Inventors: Wang, Hao, Yu, Kun, Chai, Xiujuan
• Granted Patent: US 8,433,138 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0425 using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected tracking a projected light spot to determine a position on a display surface G06F3/0386
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A computer interface may use touch- and non-touch-based gesture detection systems to detect touch and non-touch gestures on a computing device. The systems may each capture an image, and interpret the image as corresponding to a predetermined gesture. The systems may also generate similarity values to indicate the strength of a match between a captured image and corresponding gesture, and the system may combine gesture identifications from both touch- and non-touch-based gesture identification systems to ultimately determine the gesture. A threshold comparison algorithm may be used to apply different thresholds for different gesture detection systems and gesture types.

Claims

1. A method comprising:

receiving image data corresponding to a user'"'"'s gesture;

processing the image data to generate a touch gesture similarity value and a non-touch gesture similarity value;

processing the similarity values to identify the user'"'"'s gesture; and

generating an output in response to the identified gesture.

2. The method of claim 1, wherein processing the similarity values comprises comparing the touch gesture similarity value with a first threshold value;

and

comparing the non-touch similarity value with a second threshold value.

3. The method of claim 2, further comprising setting the first and second threshold values to different values based on gesture type.

4. The method of claim 1, wherein a first portion of the image data is received from a touch-based gesture detection system, and a second portion of the image data is received from a non-touch based gesture detection system.

5. The method of claim 4, wherein processing the image data includes using image data from the touch-based gesture detection system to detect a non-touch gesture.

6. The method of claim 1, further comprising determining a differential similarity value, and comparing the differential similarity value with a difference between the touch gesture and non-touch gesture similarity values.

7. The method of claim 1, further comprising:

using contextual information to select between a touch and non-touch gesture when processing the similarity values.

8. The method of claim 7, wherein the contextual information includes information identifying a previously-identified gesture.

9. The method of claim 7, wherein the contextual information includes information identifying a command set of an application.

10. One or more computer-readable media, storing computer-executable instructions that, when executed, cause a computer to perform the following:

receive image data corresponding to a user'"'"'s gesture;

process the image data to generate a touch gesture similarity value and a non-touch gesture similarity value;

process the similarity values to identify the user'"'"'s gesture; and

output an indication in response to the user'"'"'s gesture.

11. The one or more computer-readable media of claim 10, wherein the instructions for processing the similarity values cause a computer to perform the following when executed:

compare the touch gesture similarity value with a first threshold value; and

compare the non-touch gesture similarity value with a second threshold value.

12. The one or more computer-readable media of claim 11, wherein the first and second threshold values differ based on different gesture types.

13. The one or more computer-readable media of claim 10, wherein a first portion of the image data is received from a touch-based gesture detection system, and a second portion of the image data is received from a non-touch based gesture detection system.

14. The one or more computer-readable media of claim 13, further storing computer-executable instructions that, when executed, cause a computer to perform the following:

using image data from the touch-based gesture detection system to detect a non-touch gesture.

15. The one or more computer-readable media of claim 10, wherein instructions further comprise instructions that, when executed, cause the computer to compare a differential similarity value with a difference between the touch gesture and non-touch gesture similarity values.

16. The one or more computer-readable media of claim 10, further storing computer-executable instructions that, when executed, cause a computer to perform the following:

using contextual information to select between a touch gesture and a non-touch gesture.

17. The one or more computer-readable media of claim 16, wherein the contextual information includes information identifying a previously-identified gesture.

18. The one or more computer-readable media of claim 16, wherein the contextual information includes information identifying a command set of an application.

19. An apparatus, comprising:

a processor configured to;

receive image data corresponding to a user'"'"'s gesture;

process the image data to generate touch gesture and non-touch gesture similarity values;

process the similarity values to identify the user'"'"'s gesture; and

generate an output in response to the detected user gesture.

20. The apparatus of claim 19, wherein processing the similarity values comprises comparing the touch gesture similarity value with a first threshold value;

and

comparing the non-touch similarity value with a second threshold value.

21. The apparatus of claim 20, wherein the first and second threshold values differ based on different gesture types.

22. The apparatus of claim 19, wherein a first portion of the image data is received from a touch-based gesture detection system, and a second portion of the image data is received from a non-touch based gesture detection system.

23. The apparatus of claim 19, wherein the processor is further configured to use image data from a touch-based gesture detection system to detect a non-touch gesture.

24. The apparatus of claim 19, wherein the processor is further configured to compare a differential similarity value with a difference between the touch gesture and non-touch gesture similarity values.

25. The apparatus of claim 19, wherein the processor is further configured to:

use contextual information to select between a touch gesture and a non-touch gesture.

26. The apparatus of claim 25, wherein the contextual information includes information identifying a previously-identified gesture.

27. The apparatus of claim 25, wherein the contextual information includes information identifying a command set of an application.

28. The apparatus of claim 19, further comprising an infrared camera and a plurality of infrared light sources positioned below a screen, wherein the infrared camera is configured to supply at least a portion of the image data to the processor.

29. The apparatus of claim 19, further comprising a visible light camera positioned above a screen, wherein the visible light camera is configured to supply at least a portion of the image data to the processor.