Orientation-independent air gesture detection service for in-vehicle environments

• US 10,082,877 B2
• Filed: 03/15/2016
• Issued: 09/25/2018
• Est. Priority Date: 03/15/2016
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Tokish, Stephen Ronald, Buttolo, Pietro, Rankin, II, James Stewart, Salter, Stuart C.
• Primary Examiner(s): Yeung, Matthew
• Application Number:
  US15/070,129
• Publication Number:
  US 20170269695A1
• Time in Patent Office: 924 Days
• Field of Search: None
• CPC Class Codes: B60W 40/10 related to vehicle motion
• CPC Class Codes: G06F 3/016 Input arrangements with force or tactile feedback as computer generated output to the user
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry

Abstract

A vehicle sensor may indicate heading angle. A plurality of seating zones may each include an orientation sensor indicating seating angle. A component within one of the seating zones may provide interface information descriptive of commands and associated air gestures oriented to the one seating zone, and perform one of the commands responsive to a personal device detecting the corresponding air gesture, accounting for the heading angle and seating angle. Gesture input from a personal device may be adjusted to create acceleration data in a reference frame of an in-vehicle component in a seating zone using orientation of the device, vehicle heading relative to magnetic North, and orientation angle for the seating zone including the device. When the acceleration data matches an air gesture in a library of air gestures, a command may be sent to the component corresponding to the matching air gesture.

Claims

1. A system, comprising:

a vehicle sensor configured to measure heading angle of a vehicle;

a plurality of seating zones, each including an orientation sensor configured to measure seating angle for the respective seating zone relative to the vehicle; and

a component within one of the seating zones, programmed to;

provide interface information descriptive of commands and associated air gestures oriented to the one seating zone; and

perform one of the commands responsive to a personal device detecting the corresponding air gesture, accounting for the heading angle and seating angle.

2. The system of claim 1, wherein the orientation sensors are configured to indicate the seating angle as an offset from the heading angle.

3. The system of claim 1, wherein the vehicle sensor is configured to indicate the heading angle as an offset from magnetic North.

4. The system of claim 1, wherein the component is a display screen, and one of:

the air gesture is a movement from right to left and the command is a move next command, and the air gesture is a movement from left to right and the command is a move previous command.

5. The system of claim 1, wherein the component is a speaker, and one of:

the air gesture is a movement from down to up and the command is an increase volume command, and the air gesture is a movement from up to down and the command is a decrease volume command.

6. The system of claim 1, wherein the component is further programmed to receive a user selection to invoke acceptance of the air gestures for the commands.

7. The system of claim 6, wherein the user selection includes one or more of:

selecting a control of the component, or determining increasing signal strength from the personal device indicative of approach of the personal device to the component.

8. A method comprising:

adjusting gesture input from a personal device to create acceleration data in a reference frame of a component in a seating zone using orientation of the device, vehicle heading relative to magnetic North, and sensor-measured angle offset of the zone from vehicle heading; and

when the acceleration data matches an air gesture in a library of air gestures, sending a command to the component corresponding to the matching air gesture.

9. The method of claim 8, further comprising determining the orientation angle for the seating zone of the vehicle according to an orientation module embedded in a seat of the seating zone providing data with respect to the orientation angle of the seating zone offset from vehicle heading.

10. The method of claim 8, further comprising receiving, from the component, interface information descriptive of commands and corresponding air gestures for the commands.

11. The method of claim 8, further comprising:

preprocessing the air gestures of the library into clusters, each having a defined centroid;

determining a point for the acceleration data utilizing a Hidden Markov Model (HMM); and

matching the air gesture as being the air gesture of the library with a smallest Cartesian distance between the point and the defined centroid.

12. The method of claim 11, further comprising ensuring that the point is within a radius of the defined centroid to confirm identification of the air gesture.

13. The method of claim 8, further comprising:

retrieving magnetic North from a compass of the vehicle; and

retrieving the orientation angle from an orientation sensor in the seating zone of the vehicle.

14. The method of claim 8, further comprising normalizing the acceleration data to remove gravity effects.

15. The method of claim 8, further comprising, to communicate that air gestures are ready to be recognized, initiating one or more of playing a sound or actuating a haptic feedback device.

16. A device comprising:

a processor;

an acceleration sensor;

an orientation sensor;

a wireless transceiver; and

a memory storing a gesture detection service and an application,

wherein the gesture detection service includes instructions that, when executed by the processor, cause the processor to create acceleration data in a frame of reference defined according to orientation of the device, vehicle heading angle, and orientation angle of a seating zone of the vehicle including the device measured using the orientation sensor, thereby accounting for heading angle, orientation angle, and device orientation, and to detect air gestures performed by occupants of a vehicle according to data from the acceleration sensor and the orientation sensor, and the application includes instructions that, when executed by the processor, cause the processor to receive interface information descriptive of commands and corresponding air gestures for the commands from a vehicle component using the wireless transceiver, utilize the gesture detection service to determine when the gestures are performed, and send one of the commands to the vehicle component using the wireless transceiver responsive to the gesture detection service determining that the corresponding air gesture for the command is performed.

17. The device of claim 16, wherein the gesture detection service is further programmed to create acceleration data in a frame of reference defined according to orientation of the device, vehicle heading angle, and orientation angle of a seating zone of the vehicle including the device, thereby accounting for heading angle, orientation angle, and device orientation.

18. The device of claim 17, wherein the gesture detection service includes instructions that, when executed by the processor, cause the processor to:

preprocess the air gestures of a library of air gestures into clusters, each having a defined centroid;

determine a point for the acceleration data utilizing a Hidden Markov Model (HMM); and

match the air gesture to the air gesture of the library with a smallest Cartesian distance between the point and the defined centroid.

19. The device of claim 18, wherein the gesture detection service includes instructions that, when executed by the processor, cause the processor to ensure that the point is within a radius of the defined centroid to confirm identification of the air gesture.