System and Method for Controlling a Vehicle User Interface Based on Gesture Angle

• US 20140277936A1
• Filed: 03/15/2013
• Published: 09/18/2014
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Edge 3 Technologies, LLC, Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Edge 3 Technologies, LLC
• Inventors: El Dokor, Tarek A., Cluster, Jordan, Holmes, James E., Vaghefinazari, Pedram, Yamamoto, Stuart M.
• Granted Patent: US 8,886,399 B2
• Time in Patent Office: Days
• US Class Current: 701/36
• CPC Class Codes: B60R 1/072 for adjusting the mirror relative to its housing
• CPC Class Codes: B60W 50/10 Interpretation of driver requests or demands
• CPC Class Codes: G06F 2203/0381 Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113

Abstract

An in-vehicle computing system allows a user to control components of the vehicle by performing gestures. The user provides a selecting input to indicate that he wishes to control one of the components. After the component is identified, the user performs a gesture to control the component. The gesture and the component that was previously selected are analyzed to generate a command for the component. Since the command is based on both the gesture and the identified component, the user can perform the same gesture in the same position within the vehicle to control different components.

Claims

1. A vehicle-based computer-implemented method for controlling a component of the vehicle, the method comprising:

identifying a first component of the vehicle based on a first selecting input performed by the user within the vehicle;

receiving a first data signal representing a gesture performed by the user, the gesture performed in a capture region inside the vehicle;

performing gesture recognition on the first data signal to determine a first command for controlling the first identified component;

identifying a second component of the vehicle based on a second selecting input performed by the user within the vehicle, the second identified component different from the first identified component;

receiving a second data signal representing the same gesture performed by the user in the same capture region within the vehicle; and

performing gesture recognition on the second data signal to determine a second command for controlling the second identified component, the second command different from the first command.

2. The method of claim 1, wherein the gesture is captured by a camera system, the camera system comprising a plurality of cameras positioned to collectively capture gestures performed in the capture region inside the vehicle.

3. The method of claim 1, wherein performing gesture recognition comprises measuring one or more gesture angles, each gesture angle representing an angular displacement between a current position of the gesture and a reference position of the gesture.

4. The method of claim 3, wherein the gesture angles comprise a horizontal angle, a vertical angle, and a rotational angle.

5. The method of claim 3, wherein the first command is a command for navigating a menu structure based on the gesture angle, and wherein the second command is a command for adjusting a mirror of the vehicle based on the gesture angle.

6. The method of claim 1, wherein the first selecting input comprises a pointing gesture directed at the first component, the pointing gesture performed in the capture region, and wherein identifying the first component comprises performing gesture recognition on an input signal representing the first selecting input.

7. The method of claim 1, wherein the first selecting input comprises a voice command containing a name of the first component, and wherein identifying the first component comprises performing voice recognition on an input signal representing the first selecting input.

8. The method of claim 1, wherein the first selecting input is at least one of a pointing gesture or a voice command.

9. The method of claim 1, further comprising:

responsive to identifying the first component, sending an output signal to an output device for playback to the user, the output signal indicating that the first component has been selected.

10. A non-transitory computer-readable storage medium for storing computer program instructions for controlling a component of a vehicle, the program instructions when executed by a processor cause the processor to perform steps including:

identifying a first component of the vehicle based on a first selecting input performed by the user within the vehicle;

receiving a first data signal representing a gesture performed by the user, the gesture performed in a capture region inside the vehicle;

performing gesture recognition on the first data signal to determine a first command for controlling the first identified component;

identifying a second component of the vehicle based on a second selecting input performed by the user within the vehicle, the second identified component different from the first identified component;

receiving a second data signal representing the same gesture performed by the user in the same capture region within the vehicle; and

performing gesture recognition on the second data signal to determine a second command for controlling the second identified component, the second command different from the first command.

11. The storage medium of claim 10, wherein the gesture is captured by a camera system, the camera system comprising a plurality of cameras positioned to collectively capture gestures performed in the capture region inside the vehicle.

12. The storage medium of claim 10, wherein performing gesture recognition comprises measuring one or more gesture angles, each gesture angle representing an angular displacement between a current position of the gesture and a reference position of the gesture.

13. The storage medium of claim 12, wherein the gesture angles comprise a horizontal angle, a vertical angle, and a rotational angle.

14. The storage medium of claim 12, wherein the first command is a command for navigating a menu structure based on the gesture angle, and wherein the second command is a command for adjusting a mirror of the vehicle based on the gesture angle.

15. The storage medium of claim 10, wherein the first selecting input comprises a pointing gesture directed at the first component, the pointing gesture performed in the capture region, and wherein identifying the first component comprises performing gesture recognition on an input signal representing the first selecting input.

16. The storage medium of claim 10, wherein the first selecting input comprises a voice command containing a name of the first component, and wherein identifying the first component comprises performing voice recognition on an input signal representing the first selecting input.

17. The storage medium of claim 10, wherein the first selecting gesture is at least one of a pointing gesture or a voice command.

18. The storage medium of claim 10, further comprising instructions for:

responsive to identifying the first component, sending an output signal to an output device for playback to the user, the output signal indicating that the first component has been selected.

19. A vehicle-based computing system for controlling a component of the vehicle, the system comprising:

one or more processors; and

a non-transitory computer-readable storage medium for storing computer program instructions, the program instructions when executed by the processors cause the processors to perform steps including;

identifying a first component of the vehicle based on a first selecting input performed by the user within the vehicle;

receiving a first data signal representing a gesture performed by the user, the gesture performed in a capture region inside the vehicle;

performing gesture recognition on the first data signal to determine a first command for controlling the first identified component;

identifying a second component of the vehicle based on a second selecting input performed by the user within the vehicle, the second identified component different from the first identified component;

receiving a second data signal representing the same gesture performed by the user in the same capture region within the vehicle;

performing gesture recognition on the second data signal to determine a second command for controlling the second identified component, the second command different from the first command.

20. The vehicle-based computing system of claim 19, further comprising a camera system for capturing the gesture, the camera system comprising a plurality of cameras positioned to collectively capture gestures performed in the capture region inside the vehicle.

21. The vehicle-based computing system of claim 19, wherein performing gesture recognition comprises measuring one or more gesture angles, each gesture angle representing an angular displacement between a current position of the gesture and a reference position of the gesture.

22. The vehicle-based computing system of claim 21, wherein the gesture angles comprise a horizontal angle, a vertical angle, and a rotational angle.

23. The vehicle-based computing system of claim 21, wherein the first command is a command for navigating a menu structure based on the gesture angle, and wherein the second command is a command for adjusting a mirror of the vehicle based on the gesture angle.

24. The vehicle-based computing system of claim 19, wherein the first selecting input comprises a pointing gesture directed at the first component, the pointing gesture performed in the capture region, and wherein identifying the first component comprises performing gesture recognition on an input signal representing the first selecting input.

25. The vehicle-based computing system of claim 19, wherein the first selecting input comprises a voice command containing a name of the first component, and wherein identifying the first component comprises performing voice recognition on an input signal representing the first selecting input.

26. The vehicle-based computing system of claim 19, wherein the first selecting input is at least one of a pointing gesture or a voice command.

27. The vehicle-based computing system of claim 19, the computer-readable storage medium further comprising instructions for:

responsive to identifying the first component, sending an output signal to an output device for playback to the user, the output signal indicating that the first component has been selected.