FREE HAND GESTURE CONTROL OF AUTOMOTIVE USER INTERFACE

• US 20130271370A1
• Filed: 04/13/2012
• Published: 10/17/2013
• Est. Priority Date: 04/13/2012
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Corporation
• Inventors: Wang, Kongqiao, Krkkinen, Leo
• Granted Patent: US 9,239,624 B2
• Time in Patent Office: Days
• US Class Current: 345/158
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/113 Recognition of static hand signs

Abstract

A free hand gesture control user interface is described. Components of the interface system may include a stereo vision camera, or multiple cameras, a control unit, a projector and a projecting surface. In an automotive application, the projecting surface may be the windscreen in the vehicle or a mounted screen. The camera together with software modules in the control unit define a gesture acceptance border and a gesture acceptance space within which hand gestures may be imaged, defined for various control actions, and executed. The hand images are analyzed in slices of the gesture acceptance space to determine hand orientation (open or closed) and/or motion.

Claims

1. A method comprising:

defining a distance from an imaging device as a gesture acceptance border;

defining a gesture acceptance space beyond the gesture acceptance border;

training a gesture controlled system to recognize a user'"'"'s hand position and movement within the gesture acceptance space; and

associating, with a processor, each of the trained hand positions and movements with a control command or function.

2. The method of claim 1 further comprising:

executing a control command or function associated with the user'"'"'s hand position or movement.

3. The method of claim 1 further comprising:

providing a controller in a gesture controlled system, and

providing said controller with computer program instructions for performing gesture recognition.

4. The method of claim 1 further comprising:

providing a display device for interactive control in the gesture controlled system.

5. The method of claim 1 further comprising:

segmenting the gesture acceptance space into slices for analysis of an image.

6. The method of claim 5 further comprising:

computing a distance between each of a plurality of sampling points on a hand image contour and a center point on the image,

computing a mean distance from the plurality of distances, and

computing a mean variance of the plurality of distances.

7. The method of claim 6 further comprising:

comparing the calculated mean variance of one image to the calculated mean variance of a second image to distinguish an image of an open hand from an image of a closed fist, and assigning control functions to respective open and closed hand gestures.

8. The method of claim 6 further comprising:

causing the computed mean variance of an image of an open hand close to the gesture acceptance border and the computed mean variance of an image of an open hand extended into the gesture acceptance space to be stored, and

determining movement of said hand respectively toward and away from the imaging device by comparing the relative mean variances of the open hand segmented slices as the hand moves through the gesture acceptance space.

9. The method of claim 8 further comprising:

assigning a display function to movement of the hand toward the imaging device, and assigning a different display function to movement of the hand away from the imaging device.

10. An apparatus comprising:

at least one processor and at least one memory having computer program instructions stored therein, wherein the at least one memory and the computer instructions, with the processor, cause the apparatus to at least;

define a distance from an imaging device as a gesture acceptance border;

define a gesture acceptance space beyond the gesture acceptance border;

train a gesture controlled system to recognize a user'"'"'s hand position or movement within the gesture acceptance space; and

associate each of the trained hand positions and movements with a control command or function.

11. The apparatus of claim 10 wherein the at least one memory and the computer program instructions, with the processor, cause the apparatus to:

execute a control command or function associated with the user'"'"'s hand position or movement.

12. The apparatus of claim 10 wherein the at least one memory and the computer program instructions, with the processor, cause the apparatus to:

segment the gesture acceptance space into slices for analysis of an image.

13. The apparatus of claim 10 wherein the at least one memory and the computer program instructions, with the processor, cause the apparatus to:

compute a distance between each of a plurality of sampling points on a hand image contour and a center point on the image,

compute a mean distance from the plurality of distances, and

compute a mean variance of the plurality of distances.

14. The apparatus of claim 13 wherein the at least one memory and the computer program instructions, with the processor, cause the apparatus to:

compare the calculated mean variance of one image to the calculated mean variance of a second image to distinguish an image of an open hand from an image of a closed fist, and

assign control functions to respective open and closed hand gestures.

15. The apparatus of claim 13 wherein the at least one memory and the computer program instructions, with the processor, cause the apparatus to:

cause the computed mean variance of an image of an open hand close to the gesture acceptance border and the computed mean variance of an image of an open hand extended far into the gesture acceptance space to be stored, and

determine movement of said hand respectively toward and away from the imaging device by comparing the relative mean variances of the open hand segmented slices as the hand moves through the gesture acceptance space.

16. The apparatus of claim 13 wherein the at least one memory and the computer program instructions, with the processor, cause the apparatus to:

assign a display function to movement of the hand toward the imaging device, and assign a different display function to movement of the hand away from the imaging device.

17. A computer program product comprising a non-transitory computer readable medium having computer instructions stored therein, said instructions when executed causing an apparatus to:

define a distance from the imaging device as a gesture acceptance border;

define a gesture acceptance space beyond the gesture acceptance border;

train a gesture controlled system to recognize a user'"'"'s hand position or movement within the gesture acceptance space; and

associate each of the trained hand positions or movements with a control command or function to be executed.

18. The computer program product of claim 17, the instructions when executed causing the apparatus to:

execute a control command or function associated with the user'"'"'s hand position or movement.

19. The computer program product of claim 17, the instructions when executed causing the apparatus to:

segment the gesture acceptance space into slices for analysis of an image.

20. The computer program product of claim 17, the instructions when executed causing the apparatus to:

compute the distance between each of a plurality of sampling points on a hand image contour and a center point on the image,

compute the mean distance from the plurality of distances, and

compute the mean variance of the plurality of distances.

21. The computer program product of claim 20, the instructions when executed causing the apparatus to:

compare the calculated mean variance of one image to the calculated mean variance of a second image to distinguish an image of an open hand from an image of a closed fist, and assigning control functions to respective open and closed hand gestures.

22. The computer program product of claim 20, the instructions when executed causing the apparatus to:

cause storage of the computed mean variance of an image of an open hand close to the gesture acceptance border and the computed mean variance of an image of an open hand extended far into the gesture acceptance space, and

determine movement of said hand respectively toward and away from the imaging device by comparing the relative mean variances of the open hand segmented slices as the hand moves through the gesture acceptance space.

23. The computer program product of claim 20, the instructions when executed causing the apparatus to:

assign a display function to movement of the hand toward the imaging device, and

assign a different display function to movement of the hand away from the imaging device.

24. An apparatus comprising:

means for defining a distance from an imaging device as a gesture acceptance border;

means for defining a gesture acceptance space beyond the gesture acceptance border;

means for training a gesture controlled system to recognize a user'"'"'s hand position and movement within the gesture acceptance space; and

means for associating each of the trained hand positions and movements with a control command or function.