MOTOR VEHICLE CONTROL INTERFACE WITH GESTURE RECOGNITION

• US 20160368382A1
• Filed: 06/25/2014
• Published: 12/22/2016
• Est. Priority Date: 06/29/2013
• Status: Active Grant

• Current Assignee: Audi AG (Volkswagen AG)
• Original Assignee: Audi AG (Volkswagen AG)
• Inventors: SCHLITTENBAUER, Michael, ROEHDER, Martin, KUEHNE, Marcus, BOHRER, Lorenz
• Granted Patent: US 9,738,158 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B60K 2360/146 Instrument input by gesture
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A method for operating a motor vehicle control panel having a camera system and a gesture recognition device. includes filming a person gesticulating using at least one arm in the inside of the vehicle. Description data relating to a position and/or a sequence of movements of the at least one gesticulating arm is determined by the gesture recognition device and is associated with a control gesture. At least one situation parameter, which describes a gesticulation context of the person, is determined by a plausibility device which decides whether the person has performed a possible control gesture or only a gesture which is to be ignored. If it is determined that it is a possible control gesture, a control command is generated for the control gesture.

Claims

1-15. -15. (canceled)

16. A method for operating a user interface in a motor vehicle, wherein the user interface comprises a camera, the method comprising:

filming a user who is gesticulating with a gesticulating arm, within an interior of the motor vehicle by the user being filmed by the camera to thereby produce image data;

generating descriptive data of a position or movement of the gesticulating arm by a gesture recognition device, based on the image data produced by the camera;

recognizing an operating gesture by associating the descriptive data with one of a plurality of predefined operating gestures based on a first classification;

determining, using a plausibility verification device of the user interface, at least one situation parameter that describes a gesticulation context of the user;

determining, using the at least one situation parameter and a second classification, whether the user is performing a possible operating gesture or only a gesture that is not defined for operation and that is to be ignored;

outputting a control command to a receiver for a motor vehicle system depending on the operating gesture recognized based on the first classification if it is determined that the user is performing a possible operating gesture; and

inhibiting the output of the control command if it is determined that the gesture is to be ignored.

17. The method as claimed in claim 16, wherein

the at least one situation parameter comprises a speed of at least one movable part or joint of the gesticulating arm, and

it is determined that the user is performing a possible operating gesture only if each speed lies within a respective predefined range of speed values.

18. The method as claimed in claim 16, wherein

the at least one situation parameter comprises an acceleration of at least one movable part or joint of the at least one arm, and

it is determined that the user is performing a possible operating gesture only if each acceleration lies within a respective predefined range of acceleration values.

19. The method as claimed in claim 16, wherein

the at least one situation parameter indicates whether at least one movable part or joint of the gesticulating arm has been moved with a jerky movement, by which a change of position within a predefined time is greater than a predetermined maximum change of position, and

if the at least one situation parameter indicates the jerky movement, it is determined that the user is performing a gesture to be ignored.

20. The method as claimed in claim 16, wherein

the at least one situation parameter comprises an angle of at least one movable part or joint of the gesticulating arm relative to at least one other movable part or joint, and

it is determined that the user is performing a possible operating gesture only if each angle lies within a respective predefined range of angle values.

21. The method as claimed in claim 16, wherein

the at least one situation parameter comprises a location of each movable part or joint of the gesticulating arm, and

it is determined that the user is performing a possible operating gesture if each location lies within a respective predefined spatial region.

22. The method as claimed in claim 16, wherein the control command is output without the user first activating a dedicated gesture activation button in the motor vehicle after the motor vehicle has been started.

23. The method as claimed in claim 16, wherein

the method further comprises determining from a displacement history using the image data, the displacement history relating to a prior movement process of the gesticulating arm, and

the determination as to whether the user is performing a possible operating gesture is based on how a current movement process of the gesticulating arm relates to the displacement history.

24. The method as claimed in claim 23, wherein

the displacement history relates to a prior movement process of the gesticulating arm, and

the displacement history is assembled so as to specify whether the prior movement process of the gesticulating arm was classified based on the first classification and caused an uninhibited control command to be output.

25. The method as claimed in claim 16, wherein the at least one situation parameter is determined from the image data.

26. The method as claimed in claim 16, wherein to suppress a defined control command, an associated operating gesture that will trigger the defined control command is excluded from the first classification.

27. The method as claimed in claim 16, wherein the camera is at least one of:

a stereo camera and a PMD camera.

28. The method as claimed in claim 16, wherein a body model of at least one part of the user is merged into image data of an optical sensor of the camera and the at least one situation parameter is determined using the body model.

31. The method as claimed in claim 28, wherein determining the gesticulation context associates a probability value relating to an operating intention of the user with a statistical model with a respective current parameter value of the parameter and an overall probability value for the operating intention is determined based on the probability values of all checked parameters, and the control command is inhibited if the overall probability value is smaller than a predetermined minimum value.

32. The method as claimed in claim 16, wherein

a plurality of situation parameters are determined, each relating to a different aspect of user behavior, and

a statistical model is used to consider all of the situation parameters, to determine an overall behavior model for the user and to only output the control command if the behavior model corresponds with the recognized operating gestures.

33. The method as claimed in claim 16, wherein

the at least one situation parameter determines whether the user is currently involved in a conversation, and

if the user is currently involved in a conversation, it is determined that the user is performing a gesture to be ignored.

29. The method as claimed in 16, wherein

in addition to the gesticulating arm, image data is captured relating to a body model and a current body position of the user,

the current position of the user is used to identify control commands that are currently appropriate, and

all control commands other than the control commands that are currently appropriate are inhibited.

30. The method as claimed in 16, wherein

in addition to the gesticulating arm, image data is captured relating to a body model and a body position of the user,

the at least one situation parameter relates to at least one of a position of the gesticulating arm, a speed of movement of the gesticulating arm, an acceleration of the gesticulating arm and an angle present in the gesticulating arm, and

the body model is used to define a range for each operating parameter such that which positions or movements of the gesticulating arm will produce the control command is varied based on the current body position of the user.

34. A user interface operating at least one device in a motor vehicle, comprising:

a camera to film a user who is gesticulating with a gesticulating arm in the interior of the motor vehicle, and to produce image data;

a gesture recognition device coupled to the camera, to determine a position or a series of motions of the at least one gesticulating arm based on the image data of the camera, to recognize an operating gesture by associating the image data with one of a plurality of predefined operating gestures, and to output a control command to the at least one device depending on the operating gesture recognized; and

a plausibility verification device to determine at least one situation parameter that describes a gesticulation context of the user, to determine, using the at least one situation parameter and a second classification, whether the user is performing a possible operating gesture or only a gesture that is not defined for operation and that is to be ignored, and to inhibit the output of the control command if it is determined that the gesture is to be ignored.

35. A motor vehicle with a user interface as claimed in claim 34.