Smart tutorial for gesture control system

• US 10,409,382 B2
• Filed: 10/02/2015
• Issued: 09/10/2019
• Est. Priority Date: 04/03/2014
• Status: Active Grant

• Current Assignee: Edge 3 Technologies, LLC, Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Yamamoto, Stuart, El Dokor, Tarek A.
• Primary Examiner(s): Leggett, Andrea C
• Application Number:
  US14/874,175
• Publication Number:
  US 20160187992A1
• Time in Patent Office: 1,439 Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 40/113 Recognition of static hand signs

Abstract

A method includes monitoring a plurality of system inputs, and detecting a behavioral pattern performed by a user and associated with the plurality of system inputs, When the behavioral pattern is detected, the method includes associating, in a memory, a gesture with at least one action, the at least one action being determined by the plurality of system inputs, and, upon detecting the gesture, executing the action associated with the gesture.

Claims

1. A computer-implemented method, comprising:

monitoring a plurality of system inputs received by a plurality of sensors;

detecting a behavioral pattern performed by a user and associated with the plurality of system inputs, wherein the behavioral pattern is a repetitive behavior which occurs over time;

when the behavioral pattern is detected, associating, in a memory, a gesture with at least one action, the at least one action being determined by the plurality of system inputs, the gesture including at least one of a hand movement and a spoken word;

instructing the user to perform the gesture associated with the at least one action through at least one of a visual display device and an audio system by teaching the user how to perform the gesture with a visual or audio representation of a predetermined gesture; and

upon detecting the gesture by at least one of a visual sensor and an audio sensor, executing the action associated with the gesture.

2. The method of claim 1, including presenting the user with a plurality of options to customize the gesture.

3. The method of claim 2, wherein the plurality of options to customize the gesture includes presenting the user with a plurality of pre-defined gestures, the user being able to select a desired gesture from the plurality of pre-defined gestures.

4. The method of claim 2, further including the steps of presenting the user with a plurality of actions to associate with the gesture.

5. The method of claim 4, wherein the plurality of actions to associate with the gesture includes allowing the user to at least one of add and remove actions associated with the gesture.

6. The method of claim 1, wherein the plurality of system inputs correspond to a plurality of sub-systems.

7. The method of claim 6, wherein the plurality of sub-systems includes at least one of a navigation system, a phone system and a music system.

8. The method of claim 1, further comprising the step of instructing the system to store the gesture and associated action in a system memory.

9. The method of claim 1, wherein the action associated with the gesture enables a plurality of system outputs.

10. A computer-implemented method for developing customized gestures in an implicit gesture control system based on user behavior, comprising:

monitoring, using a plurality of sensors, a plurality of actions performed by a user;

detecting behavioral patterns associated with the plurality of actions performed by the user, wherein the behavioral patterns are repetitive behaviors which occur over time;

correlating a series of system outputs to the detected behavioral patterns;

instructing the user to perform a suggested gesture through at least one of a visual display device and an audio system by teaching the user how to perform the gesture with a visual or audio representation of a predetermined gesture, the gesture including at least one of a hand movement and a spoken word, the suggested gesture associated with the series of system outputs in a memory;

executing the series of system outputs when the user performs the suggested gesture, and

instructing the user to accept or reject the executed series of system outputs associated with the suggested gesture.

11. The method of claim 10, wherein the user is instructed to perform the suggested gesture by an audio command.

12. The method of claim 10, wherein the user is instructed to perform the suggested gesture by a visual command.

13. The method of claim 10, wherein the user is instructed to perform the suggested gesture automatically after a behavioral pattern has been detected.

14. A learning tutorial system, comprising:

a plurality of sensors coupled to a vehicle;

a processor, the processor in communication with the plurality of sensors;

the processor being configured to perform the steps of;

monitoring a plurality of system inputs received by the plurality of sensors;

detecting behavioral patterns associated with the plurality of system inputs, wherein the behavioral patterns are repetitive behaviors which occur over time;

prompting a user to perform a gesture using at least one of a visual display device and an audio system by teaching the user how to perform the gesture with a visual or audio representation of a predetermined gesture, the gesture associated with the detected behavioral patterns and including at least one of a hand movement and a spoken word; and

executing an action associated with the gesture when the gesture is performed by the user.

15. The system of claim 14, wherein the processor is further configured present the user with a plurality of options to customize the gestures.

16. The system of claim 14, wherein the processor is in further communication with a vehicle infotainment device.

17. The system of claim 16, wherein the processor can accept inputs from the infotainment device indicating the plurality of options selected by the user.

18. The system of claim 16, wherein the system can prompt the user to perform the gesture using the infotainment device.

19. The system of claim 14, wherein the plurality of sensors sense the plurality of system inputs presented to a plurality of vehicle subsystems.

20. The system of claim 14, wherein the plurality of vehicle subsystems includes at least one of a navigation system, a phone system and a music system.