System and method for executing gesture based control of a vehicle system

• US 9,809,231 B2
• Filed: 10/28/2015
• Issued: 11/07/2017
• Est. Priority Date: 10/28/2015
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Ling, Calvin, Nehmad, Michael
• Primary Examiner(s): Shaawat, Mussa A
• Assistant Examiner(s): JHA, ABDHESH K
• Application Number:
  US14/924,765
• Publication Number:
  US 20170120930A1
• Time in Patent Office: 741 Days
• Field of Search: None
• CPC Class Codes: B60K 2360/146 Instrument input by gesture
• CPC Class Codes: B60K 2360/1464 3D-gesture
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 2370/146 Input by gesture
• CPC Class Codes: B60K 2370/1464 3D-gesture
• CPC Class Codes: B60K 2370/21 using cameras
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60R 16/023 for transmission of signals between vehicle parts or subsystems
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: F16H 59/02 Selector apparatus
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

A method and system for executing gesture based control of a vehicle system that include receiving at least one signal that pertains to at least one muscle movement from sensors disposed within a gear shift knob. The method and system also include determining at least one gesture based on the at least one signal. Additionally, the method and system include determining an interaction command with the vehicle system based on the at least one gesture. The method and system further include outputting the interaction command to the vehicle system.

Claims

1. A computer-implemented method for executing gesture based control of a vehicle system, comprising:

receiving a plurality of signals that pertain to a plurality of muscle movements from a plurality of electromyography (EEG) sensors disposed within a gear shift knob, wherein receiving the plurality of signals includes analyzing each of the plurality of signals and determining a predetermined code value that corresponds to each of the plurality of signals;

determining at least one gesture based on the plurality of signals, wherein determining the at least one gesture includes acquiring a plurality of gestural attribute values that correspond to the predetermined code value, wherein determining the at least one gesture further includes combining the plurality of gestural attribute values that correspond with the predetermined code value into a gesture code;

determining an interaction command with at least one vehicle system that is currently being utilized based on the at least one gesture, wherein the at least one vehicle system is identified as currently being utilized based on a vehicle system user interface associated with the at least one vehicle system that is being presented based on a user input, wherein a first haptic feedback is provided through the gear shift knob when the interaction command is not determined based on the at least one gesture; and

outputting the interaction command to the at least one vehicle system, wherein the interaction command is outputted to control the at least one vehicle system to execute the interaction command, wherein a second haptic feedback is provided to indicate at least one of;

the at least one vehicle system is executing the interaction command based on the gesture, and the at least one vehicle system has executed the interaction command based on the gesture.

2. The computer-implemented method of claim 1, wherein receiving the plurality of signals includes disposing the plurality of EEG sensors on the gear shift knob to sense the plurality of muscle movements from at least one of:

a hand, a wrist, and a forearm that is positioned on a hand rest portion of the gear shift knob.

3. The computer-implemented method of claim 1, predetermined code value is associated to at least one of:

a specific muscle that is moved, a degree of movement of the specific muscle, a type of movement of the specific muscle, a direction of movement of the specific muscle, and a duration of the movement of the specific muscle.

4. The computer-implemented method of claim 3, wherein acquiring the plurality of gestural attribute values includes providing a query on a first data table to determine a record that includes the predetermined code value and acquiring the plurality of gestural attribute values, wherein each of the plurality of gestural attribute values is associated to at least one of:

a specific type of hand movement, a specific type of finger movement, and a specific type of thumb movement that represents a portion of a full gesture that is provided.

5. The computer-implemented method of claim 4, wherein the gesture code corresponds to at least one of:

a type of gesture, a direction of the gesture, a duration of the gesture, and an orientation of the gesture.

6. The computer-implemented method of claim 5, wherein determining the interaction command includes providing a query on a second data table to acquire a plurality of records containing a designation of the at least one vehicle system.

7. The computer-implemented method of claim 6, wherein determining the interaction command includes providing a query on the plurality of records of the second data table containing the designation of the at least one vehicle system to determine a record that contains the gesture code and acquiring the interaction command that is associated to the gesture code.

8. The computer-implemented method of claim 1, wherein the second haptic feedback is provided when a confirmation signal is received from the vehicle system, wherein the second haptic feedback includes at least one of:

vibrational feedback, or force feedback.

9. A system for executing gesture based control of a vehicle system comprising:

a non-transitory memory storing instructions when executed by a processor causes the processor to;

receive a plurality of signals that pertain to a plurality of muscle movements from a plurality of electromyography (EEG) sensors disposed within a gear shift knob, wherein receiving the plurality of signals includes analyzing each of the plurality of signals and determining a predetermined code value that corresponds to each of the plurality of signals;

determine at least one gesture based on the plurality of signals, wherein determining the at least one gesture includes acquiring a plurality of gestural attribute values that correspond to the predetermined code value, wherein determining the at least one gesture further includes combining the plurality of gestural attribute values that correspond with the predetermined code value into a gesture code;

determine an interaction command with at least one vehicle system that is currently being utilized based on the at least one gesture, wherein the at least one vehicle system is identified as currently being utilized based on a vehicle system user interface associated with the at least one vehicle system that is being presented based on a user input, wherein a first haptic feedback is provided through the gear shift knob when the interaction command is not determined based on the at least one gesture; and

output the interaction command to the at least one vehicle system, wherein the interaction command is outputted to control the at least one vehicle system to execute the interaction command, wherein a second haptic feedback is provided to indicate at least one of;

the at least one vehicle system is executing the interaction command based on the gesture, and the at least one vehicle system has executed the interaction command based on the gesture.

10. The system of claim 9, wherein the plurality of EEG sensors are disposed on the gear shift knob to sense the plurality of muscle movements from at least one of:

a hand, a wrist, and a forearm that is positioned on a hand rest portion of the gear shift knob.

11. The system of claim 10, wherein the predetermined code value is associated to at least one of:

a specific muscle that is moved, a degree of movement of the specific muscle, a type of movement of the specific muscle, a direction of movement of the specific muscle, and a duration of the movement of the specific muscle.

12. The system of claim 11, wherein a query is provided on a first data table to determine a record that includes the predetermined code value and the plurality of gestural attribute values is acquired, wherein each of the plurality of gestural attribute values is associated to at least one of:

a specific type of hand movement, a specific type of finger movement, and a specific type of thumb movement that represents a portion of a full gesture that is provided.

13. The system of claim 12, wherein the gesture code corresponds to at least one of:

a type of gesture, a direction of the gesture, a duration of the gesture, and an orientation of the gesture.

14. The system of claim 13, wherein wherein a query is provided on a second data table to acquire a plurality of records containing a designation of the at least one vehicle system.

15. The system of claim 14, wherein determining the interaction command includes providing a query on the plurality of records of the second data table containing the designation of the at least one vehicle system to determine a record that contains the gesture code, wherein the interaction command that is associated to the gesture code is acquired.

16. The system of claim 9, further including wherein the second haptic feedback is provided when a confirmatory signal is received, wherein the second haptic feedback includes at least one of:

vibrational feedback, or force feedback.

17. A non-transitory computer-readable storage medium storing instructions that when executed by a processor of a computer, causes the processor of the computer to implement a method, comprising:

receiving a plurality of signals that pertain to a plurality of muscle movements from a plurality of electromyography (EEG) sensors disposed within a gear shift knob, wherein receiving the plurality of signals includes analyzing each of the plurality of signals and determining a predetermined code value that corresponds to each of the plurality of signals;

determining at least one gesture based on the plurality of signals, wherein determining the at least one gesture includes acquiring a plurality of gestural attribute values that correspond to the predetermined code value, wherein determining the at least one gesture further includes combining the plurality of gestural attribute values that correspond with the predetermined code value into a gesture code;

determining an interaction command with at least one vehicle system that is currently being utilized based on the at least one gesture, wherein the at least one vehicle system is identified as currently being utilized based on a vehicle system user interface associated with the at least one vehicle system that is being presented based on a user input, wherein a first haptic feedback is provided through the gear shift knob when the interaction command is not determined based on the at least one gesture; and

outputting the interaction command to the at least one vehicle system, wherein the interaction command is outputted to control the at least one vehicle system to execute the interaction command, wherein a second haptic feedback is provided to indicate at least one of;

the at least one vehicle system is executing the interaction command based on the gesture, and the at least one vehicle system has executed the interaction command based on the gesture.

18. The non-transitory computer-readable storage medium of claim 17, wherein acquiring the plurality of gestural attribute values includes providing a query on a first data table to determine a record that includes the predetermined code value and acquiring the plurality of gestural attribute values, wherein each of the plurality of gestural attribute values is associated to at least one of:

a specific type of hand movement, a specific type of finger movement, and a specific type of thumb movement that represents a portion of a full gesture that is provided.

19. The non-transitory computer-readable storage medium of claim 18, wherein the gesture code corresponds to at least one of:

a type of gesture, a direction of the gesture, a duration of the gesture, and an orientation of the gesture.

20. The non-transitory computer-readable storage medium of claim 19, wherein determining the interaction command includes providing a query on the plurality of records of a second data table containing a designation of the at least one vehicle system that is currently being utilized to determine a record that contains the gesture code and acquiring the interaction command that is associated to the gesture code, wherein the interaction command is provided to the at least one vehicle system.