Free hand gesture control of automotive user interface

• US 9,239,624 B2
• Filed: 04/13/2012
• Issued: 01/19/2016
• Est. Priority Date: 04/13/2012
• Status: Active Grant

• Current Assignee: Nokia Technologies Oy (Nokia Corporation)
• Original Assignee: Nokia Technologies Oy (Nokia Corporation)
• Inventors: Wang, Kongqiao, Krkkinen, Leo
• Primary Examiner(s): Sternbane, Larry
• Assistant Examiner(s): MICHAUD, ROBERT J
• Application Number:
  US13/446,793
• Publication Number:
  US 20130271370A1
• Time in Patent Office: 1,376 Days
• Field of Search: 345/173, 345/175, 345/104, 345/158, 345/156, 382/224
• US Class Current: 1/1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 40/113 Recognition of static hand signs

Abstract

A free hand gesture control user interface is described. Components of the interface system may include a stereo vision camera, or multiple cameras, a control unit, a projector and a projecting surface. In an automotive application, the projecting surface may be the windscreen in the vehicle or a mounted screen. The camera together with software modules in the control unit define a gesture acceptance border and a gesture acceptance space within which hand gestures may be imaged, defined for various control actions, and executed. The hand images are analyzed in slices of the gesture acceptance space to determine hand orientation (open or closed) and/or motion.

Claims

1. A method comprising:

defining a distance from an imaging device as a gesture acceptance border, wherein the distance of the gesture acceptance border from the imaging device is user adjustable independent of a position of the user, and wherein a user is between the imaging device and the gesture acceptance border;

defining a gesture acceptance space beyond the gesture acceptance border;

providing an alert in response to detecting a user'"'"'s hand moving through the gesture acceptance border;

training a gesture controlled system to recognize a user'"'"'s hand position and movement within the gesture acceptance space;

associating, with a processor, each of the trained hand positions and movements with a control command or function;

segmenting the gesture acceptance space into slices for analysis of an image; and

recognizing a user'"'"'s hand position and movement by;

computing a distance between each of a plurality of sampling points on a hand image contour and a center point on the image,

computing a mean distance from the plurality of distances, and

computing a mean variance of the plurality of distances.

2. The method of claim 1 further comprising:

executing a control command or function associated with the user'"'"'s hand position or movement.

3. The method of claim 1 further comprising:

providing a controller in a gesture controlled system, and

providing said controller with computer program instructions for performing gesture recognition.

4. The method of claim 1 further comprising:

providing a display device for interactive control in the gesture controlled system.

5. The method of claim 1 further comprising:

comparing the calculated mean variance of one image to the calculated mean variance of a second image to distinguish an image of an open hand from an image of a closed fist, and assigning control functions to respective open and closed hand gestures.

6. The method of claim 1 further comprising:

causing the computed mean variance of an image of an open hand close to the gesture acceptance border and the computed mean variance of an image of an open hand extended into the gesture acceptance space to be stored, and

determining movement of said hand respectively toward and away from the imaging device by comparing the relative mean variances of the open hand segmented slices as the hand moves through the gesture acceptance space.

7. The method of claim 6 further comprising:

assigning a display function to movement of the hand toward the imaging device, and assigning a different display function to movement of the hand away from the imaging device.

8. The method of claim 1, further comprising:

defining a gesture rejection space between the imaging device and the gesture acceptance border.

9. The method of claim 1, wherein the gesture acceptance border comprises a curved surface on which each point has the same distance to the imaging device.

10. An apparatus comprising:

at least one processor and at least one memory having computer program instructions stored therein, wherein the at least one memory and the computer instructions, with the processor, cause the apparatus to at least;

define a distance from an imaging device as a gesture acceptance border, wherein the distance of the gesture acceptance border from the imaging device is user adjustable independent of a position of the user, and wherein a user is between the imaging device and the gesture acceptance border;

define a gesture acceptance space beyond the gesture acceptance border;

provide an alert in response to detecting a user'"'"'s hand moving through the gesture acceptance border;

train a gesture controlled system to recognize a user'"'"'s hand position or movement within the gesture acceptance space;

associate each of the trained hand positions and movements with a control command or function;

segment the gesture acceptance space into slices for analysis of an image; and

recognize a user'"'"'s hand position and movement by causing the apparatus to;

compute a distance between each of a plurality of sampling points on a hand image contour and a center point on the image,

compute a mean distance from the plurality of distances, and

compute a mean variance of the plurality of distances.

11. The apparatus of claim 10 wherein the at least one memory and the computer program instructions, with the processor, cause the apparatus to:

execute a control command or function associated with the user'"'"'s hand position or movement.

12. The apparatus of claim 10 wherein the at least one memory and the computer program instructions, with the processor, cause the apparatus to:

compare the calculated mean variance of one image to the calculated mean variance of a second image to distinguish an image of an open hand from an image of a closed fist, and

assign control functions to respective open and closed hand gestures.

13. The apparatus of claim 10 wherein the at least one memory and the computer program instructions, with the processor, cause the apparatus to:

cause the computed mean variance of an image of an open hand close to the gesture acceptance border and the computed mean variance of an image of an open hand extended far into the gesture acceptance space to be stored, and

determine movement of said hand respectively toward and away from the imaging device by comparing the relative mean variances of the open hand segmented slices as the hand moves through the gesture acceptance space.

14. The apparatus of claim 10 wherein the at least one memory and the computer program instructions, with the processor, cause the apparatus to:

assign a display function to movement of the hand toward the imaging device, and assign a different display function to movement of the hand away from the imaging device.

15. A computer program product comprising a non-transitory computer readable medium having computer instructions stored therein, said instructions when executed causing an apparatus to:

define a distance from the imaging device as a gesture acceptance border, wherein the distance of the gesture acceptance border from the imaging device is user adjustable independent of a position of the user, and wherein a user is between the imaging device and the gesture acceptance border;

define a gesture acceptance space beyond the gesture acceptance border;

provide an alert in response to detecting a user'"'"'s hand moving through the gesture acceptance border;

train a gesture controlled system to recognize a user'"'"'s hand position or movement within the gesture acceptance space;

associate each of the trained hand positions or movements with a control command or function to be executed;

segment the gesture acceptance space into slices for analysis of an image; and

recognize a user'"'"'s hand position and movement including computer instructions, when executed, causing the apparatus to;

compute a distance between each of a plurality of sampling points on a hand image contour and a center point on the image,

compute a mean distance from the plurality of distances, and

compute a mean variance of the plurality of distances.

16. The computer program product of claim 15, the instructions when executed causing the apparatus to:

execute a control command or function associated with the user'"'"'s hand position or movement.

17. The computer program product of claim 15, the instructions when executed causing the apparatus to:

compare the calculated mean variance of one image to the calculated mean variance of a second image to distinguish an image of an open hand from an image of a closed fist, and assigning control functions to respective open and closed hand gestures.

18. The computer program product of claim 15, the instructions when executed causing the apparatus to:

cause storage of the computed mean variance of an image of an open hand close to the gesture acceptance border and the computed mean variance of an image of an open hand extended far into the gesture acceptance space, and

determine movement of said hand respectively toward and away from the imaging device by comparing the relative mean variances of the open hand segmented slices as the hand moves through the gesture acceptance space.

19. The computer program product of claim 15, the instructions when executed causing the apparatus to:

assign a display function to movement of the hand toward the imaging device, and

assign a different display function to movement of the hand away from the imaging device.

20. An apparatus comprising:

means for defining a distance from an imaging device as a gesture acceptance border, wherein the distance of the gesture acceptance border from the imaging device is user adjustable independent of a position of the user, and wherein a user is between the imaging device and the gesture acceptance border;

means for defining a gesture acceptance space beyond the gesture acceptance border;

means for segmenting the gesture acceptance space into slices for analysis of an image;

means for training a gesture controlled system to recognize a user'"'"'s hand position and movement within the gesture acceptance space;

means for associating each of the trained hand positions and movements with a control command or function; and

means for recognizing a user'"'"'s hand position and movement including;

means for computing a distance between each of a plurality of sampling points on a hand image contour and a center point on the image,

means for computing a mean distance from the plurality of distances, and

means for computing a mean variance of the plurality of distances.