Systems, methods, and apparatus for controlling gesture initiation and termination

• US 9,965,169 B2
• Filed: 12/29/2011
• Issued: 05/08/2018
• Est. Priority Date: 12/29/2011
• Status: Active Grant

• Current Assignee: Tahoe Research Limited (f/k/a Learndale Limited) (Vector Capital Corporation)
• Original Assignee: Intel Corporation
• Inventors: Graumann, David L.
• Primary Examiner(s): Heinle, Courtney D
• Application Number:
  US13/977,591
• Publication Number:
  US 20140223384A1
• Time in Patent Office: 2,322 Days
• Field of Search: 701 36
• CPC Class Codes: B60K 2360/145 Instrument input by combination of touch screen and hardware input devices
• CPC Class Codes: B60K 2360/146 Instrument input by gesture
• CPC Class Codes: B60K 2360/1464 3D-gesture
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 2360/5899 Internet
• CPC Class Codes: B60K 2360/592 Data transfer involving external databases
• CPC Class Codes: B60K 28/02 responsive to conditions relating to the driver
• CPC Class Codes: B60K 28/066 actuating a signalling device B60K28/063 takes precedence
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/85 Arrangements for transferring vehicle- or driver-related data
• CPC Class Codes: B60W 2540/22 Psychological state; Stress level or workload
• CPC Class Codes: B60W 50/08 Interaction between the driver and the control system
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/023 Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text

Abstract

Certain embodiments of the invention may include systems, methods, and apparatus for controlling gesture initiation and termination of a user and controlling devices based on a user'"'"'s gestures. According to one embodiment, a vehicle can include at least one actuator; at least one gesture detection device; and one or more processors. The one or more processors receive an initiation indication from the at least one actuator; receive gesture information from the at least one gesture detection device; receive a termination indication from the at least one actuator; determine, from the received gesture information, a gesture from at least one occupant of the vehicle, wherein the gesture is determined based at least in part on the initiation indication; select a command from a plurality of commands, based at least in part on the determined gesture; and output a control signal or a command control signal associated with the command.

Claims

1. A vehicle comprising:

at least one button;

at least one gesture detection device configured to detect gestures when an object enters a field of view; and

one or more processors configured to;

detect a hand gesture within the field of view of the at least one gesture detection device;

receive a button push-from the at least one button, wherein the at least one button is configured to be manipulated by a hand of an occupant of the vehicle, and wherein the hand gesture precedes the button push;

determine, based at least in part on the hand gesture in addition to the button push, that at least the hand gesture is to be translated into a command;

receive gesture information associated with at least the hand gesture from the at least one gesture detection device;

determine, from the gesture information, the hand gesture;

select the command based at least in part on the hand gesture; and

output at least one control signal or at least one command control signal associated with the command.

2. The vehicle of claim 1, wherein the at least one gesture detection device comprises one or more of an image capture device, a touchpad, a switch, a proximity sensor, a light path interrupter, or a capacitive sensor.

3. The vehicle of claim 1, wherein the one or more processors are further configured to receive a termination indication from the at least one button, and wherein to output the at least one control signal or the at least one command control signal comprises to output based at least in part on the termination indication.

4. The vehicle of claim 3, wherein the button push and the termination indication comprise one or more of a change in voltage, a change in current, or a change in light level from the at least one button, wherein the change is controlled at least in part by the occupant of the vehicle.

5. The vehicle of claim 1, wherein the at least one button comprises one or more of a touch sensitive switch, a toggle switch, a momentary switch, a normally open switch, a normally closed switch, a light path interrupter, a capacitive sensor, or a proximity sensor.

6. The vehicle of claim 1, wherein to determine the second hand gesture comprises to determine a placement or to track a movement path associated with the hand.

7. The vehicle of claim 1, wherein the one or more processors are further configured to learn a new gesture, wherein to learn the new gesture comprises to:

receive a learn indication from the at least one button;

determine a placement or track a movement path associated with the hand, based at least in part on the learn indication;

associate a second command with the placement or the movement path; and

store at least a representation of the placement or the movement path and the second command.

8. The vehicle of claim 1, wherein the command is selected from among one or more of vehicle control commands, climate control commands, audio system commands, window control commands, seat control commands, display control commands, web commands, share a gesture commands, or communications control commands.

9. The vehicle of claim 1, wherein the one or more processors are further configured to detect, after the button push, a second hand gesture within the field of view of the at least one gesture detection device, wherein the gesture information is associated with at least the hand gesture and the second hand gesture.

10. An apparatus comprising:

one or more processors configured to;

detect a hand gesture within a field of view of at least one gesture detection device;

receive a button push from at least one button, wherein the at least one button is configured to be manipulated by a hand of an occupant of a vehicle, and wherein the hand gesture precedes the button push;

determine, based at least in part on the hand gesture in addition to the button push, that at least the hand gesture is to be translated into a command;

receive gesture information associated with at least the hand gesture from the at least one gesture detection device;

determine, from the gesture information, the hand gesture;

select the command based at least in part on the hand gesture; and

output at least one control signal or at least one command control signal associated with the command.

11. The apparatus of claim 10, wherein the at least one gesture detection device comprises one or more of an image capture device, a touchpad, a switch, a proximity sensor, a light path interrupter, or a capacitive sensor.

12. The apparatus of claim 10, wherein the one or more processors are further configured to receive a termination indication from the at least one button, and wherein to output the at least one control signal or the at least one command control signal comprises to output based at least in part on the termination indication.

13. The apparatus of claim 12, wherein one or more of the button push or the termination indication comprises one or more of a change in voltage, a change in current, or a change in light level from the at least one button attached to a vehicle, wherein the change is controlled at least in part by the occupant of the vehicle.

14. The apparatus of claim 10, wherein the at least one button comprises one or more of a touch sensitive switch, a toggle switch, a momentary switch, a normally open switch, a normally closed switch, a light path interrupter, a capacitive sensor, or a proximity sensor.

15. The apparatus of claim 10, wherein to determine the hand gesture comprises to determine a placement or to track a movement path associated with the hand.

16. The apparatus of claim 10, wherein the one or more processors are further configured to learn a new gesture, wherein to learn the new gesture comprises to:

receive a learn indication from the at least one button;

determine a placement or track a movement path associated with the hand based at least in part on the learn indication;

associate a second command with the placement or the movement path; and

store at least a representation of the placement or the movement path and the second command.

17. The apparatus of claim 10, wherein the command is selected from among one or more of vehicle control commands, climate control commands, audio system commands, window control commands, seal control commands, display control commands, web commands, share gesture commands, or communications control commands.

18. One or more non-transitory computer-readable media comprising computer-executable instructions that, when executed by one or more processors, configure the one or more processors to:

detect a hand gesture within a field of view of at least one gesture detection device;

receive a button push from at least one button, wherein the at least one button is configured to be manipulated by a hand of an occupant of a vehicle, and wherein the hand gesture precedes the button push;

determine, based at least in part on the hand gesture in addition to the button push, that at least the hand gesture is to be translated into a command;

receive gesture information associated with at least the hand gesture from the at least one gesture detection device;

determine, from the gesture information, the hand gesture;

select the command based at least in part on the hand gesture; and

output at least one control signal or at least one command control signal associated with the command.

19. The non-transitory computer-readable media of claim 18, wherein the instructions further configure the one or more processors to receive a termination indication from the at least one button, and wherein to output the at least one control signal or the at least one command control signal comprises to output based at least in part on the termination indication.

20. The non-transitory computer-readable media of claim 18, wherein the at least one button comprises one or more of a touch sensitive switch, a toggle switch, a momentary switch, a normally open switch, a normally closed switch, a light path interrupter, a capacitive sensor, or a proximity sensor.

21. The non-transitory computer-readable media of claim 18, wherein the instructions further configure the one or more processors to learn a new gesture, wherein to learn the new gesture comprises to:

receive a learn indication from the at least one button;

determine a placement or track a movement path associated with the hand of the occupant of the vehicle, based at least in part on the learn indication;

associate a second command with the placement or the movement path; and

store at least a representation of the placement or the movement path and the second command.

22. The non-transitory computer-readable media of claim 18, wherein the command is selected from among one or more of vehicle control commands, climate control commands, audio system commands, window control commands, seat control commands, display control commands, web commands, share gesture commands, or communications control commands.

23. A method comprising executing computer-executable instructions by one or more processors, the method further comprising:

detecting a hand gesture within a field of view of at least one gesture detection device;

identifying a button push from at least one button, wherein the at least one button is configured to be manipulated by a hand of an occupant of a vehicle, and wherein the hand gesture precedes the button push;

determining, based at least in part on the hand gesture in addition to the button push, that at least the hand gesture is to be translated into a command;

identifying gesture information associated with at least the hand gesture, based at least in part on detecting the hand gesture;

determining, from the gesture information, the hand gesture;

selecting the command from a plurality of commands, based at least in part on the hand gesture; and

outputting at least one control signal or at least one command control signal associated with the command.

24. The method of claim 23, further comprising receiving and processing one or more images obtained from an image capture device.

25. The method of claim 23, further comprising identifying a gesture termination indication by receiving an indication from the at least one button, wherein outputting the at least one control signal or the at least one command control signal is based at least in part on the identifying of the gesture termination indication.

26. The method of claim 25, wherein identifying one or more of the button push or the gesture termination indication comprises detecting one or more of a change in voltage, a change in current, or a change in light level from the at least one button attached to the vehicle, wherein the change is controlled at least in part by the occupant of the vehicle.

27. The method of claim 25, wherein identifying one or more of the button push or the gesture termination indication comprises receiving the indication from a push button comprising one or more of a switch, a momentary switch, a normally open switch, a normally closed switch, a light path interrupter, a capacitive sensor, or a proximity sensor.

28. The method of claim 23, wherein detecting the hand gesture comprises determining a placement or tracking a movement path associated with the hand.

29. The method of claim 23, further comprising learning a new gesture, wherein learning the new gesture comprises:

identifying a learn indication;

determining a placement or tracking a movement path associated with the hand based at least in part on the learn indication;

associating a second command with the placement or the movement path; and

storing at least a representation of the placement or the movement path and the second command.

30. The method of claim 23, wherein the command is selected from among one or more of vehicle control commands, climate control commands, audio system commands, window control commands, seat control commands, display control commands, web commands, share a gesture commands, or communications control commands.