Method and device for operating functions displayed on a display unit of a vehicle using gestures which are carried out in a three-dimensional space, and corresponding computer program product

• US 9,440,537 B2
• Filed: 12/08/2012
• Issued: 09/13/2016
• Est. Priority Date: 01/09/2012
• Status: Active Grant

• Current Assignee: Daimler AG (Mercedes-Benz Group AG)
• Original Assignee: Daimler AG (Mercedes-Benz Group AG)
• Inventors: Zhang-Xu, Tingting, Entenmann, Volker, Mattes, Stefan, Reisinger, Joerg
• Primary Examiner(s): TISSOT, ADAM D
• Application Number:
  US14/371,022
• Publication Number:
  US 20140358368A1
• Time in Patent Office: 1,375 Days
• Field of Search: 345/156, 345/157, 345/158, 382/104, 382/181, 701/36, 701/48, 701/49, 715/863
• US Class Current: 1/1
• CPC Class Codes: B60K 2360/146 Instrument input by gesture
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 2370/146 Input by gesture
• CPC Class Codes: B60K 2370/21 using cameras
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883

Abstract

A method to control functions displayed on a display unit of a vehicle uses gestures carried out in three-dimensional space. When a first gesture carried out in three-dimensional space is detected it is determined whether the detected first gesture is allocated to an activation of an operation of a function. If it is determined that the detected first gesture is the gesture allocated to the activation of the operation of the function, then the function is activated. Next a second gesture carried out in three-dimensional space is detected and it is determined whether the detected second gesture is a gesture allocated to the operation of the function. If it is determined that the detected second gesture is the gesture allocated to the operation of the function then the operation of the function is performed.

Claims

1. A method to control functions displayed on a display unit of a vehicle using gestures carried out in three-dimensional space, the method comprising:

a) determining whether a first gesture carried out in three-dimensional space is detected by a control device;

b) determining, if it is determined that the first gesture has been detected, whether or not the detected first gesture is a gesture allocated to activation of an operation of a function displayed on the display unit of the vehicle by the control device;

c) activating operation of the function if it is determined that the detected first gesture is the gesture allocated to the activation of the operation of the function by the control device;

d) determining whether a second gesture carried out in three-dimensional space is detected by the control device;

e) determining, if it is determined that the second gesture has been detected, whether or not the detected second gesture is a gesture allocated to the operation of the function displayed on the display unit of the vehicle by the control device;

f) operating the function displayed on the display unit of the vehicle if it is determined that the detected first gesture is the gesture allocated to the activation of the operation of the function and the detected second gesture is the gesture allocated to the operation of the function by the control device; and

g) setting a control direction for the function to be operated in agreement with a direction of a first detection of a second gesture allocated to the operation of the function, wherein the setting of the control direction is sustained if repeated implementation of steps d) to f) directly after one another after steps a) to c) have been carried out.

2. The method of claim 1, wherein after performing steps a) to c), steps d) to f) are repeatedly performed directly one after the other.

3. The method of claim 1, wherein it is determined in step b) that the detected first gesture is the gesture allocated to the activation of the operation of the function if the detected first gesture is static for a first predetermined amount of time in a first predetermined region in three-dimensional space.

4. The method of claim 1, wherein it is determined in step e) that the detected second gesture is the gesture allocated to the operation of the function if the detected second gesture is dynamic in a second predetermined region in three-dimensional space.

5. The method of claim 1, further comprising:

h) determining whether a third gesture carried out in three-dimensional space is detected by the control device;

i) determining, if it is determined that the third gesture has been detected, whether or not the detected third gesture is a gesture allocated to the resetting of the control direction by the control device;

j) resetting the control direction if it is determined that the detected third gesture is the gesture allocated to the resetting of the control direction by the control device.

6. The method of claim 5, wherein it is determined in step i) that the detected third gesture is the gesture allocated to the resetting of the control direction if the detected third gesture is static for a third predetermined amount of time in a third predetermined region in three-dimensional space.

7. The method of claim 1, wherein in step c), a display element depicting the activation of the function is displayed on the display unit.

8. The method of claim 7, wherein the display element depicting the activation of the function is no longer displayed on the display unit after a fourth predetermined amount of time in which no gesture is detected.

13. The method of claim 1, further comprising, after the setting of the control direction, a subsequent detection of the second gesture allocated to the operation of the function in the set control direction operates the function.

9. A device to control functions displayed on a display unit of a vehicle using gestures carried out in three-dimensional space, wherein the device is configured to:

a) determine whether a first gesture carried out in three-dimensional space is detected;

b) determine, if it is determined that the first gesture has been detected, whether or not the detected first gesture is a gesture allocated to activation of an operation of a function;

c) activate the operation of the function if it is determined that the detected first gesture is the gesture allocated to the activation of the operation of the function;

d) determine whether a second gesture carried out in three-dimensional space is detected;

e) determine, if it is determined that the second gesture has been detected, whether or not the detected second gesture is a gesture allocated to the operation of the function;

f) operate the function if it is determined that the detected first gesture is the gesture allocated to the activation of the operation of the function and the detected second gesture is the gesture allocated to the operation of the function; and

g) set a control direction for the function to be operated in agreement with a direction of a first detection of a second gesture allocated to the operation of the function and sustain the set control direction if repeated implementation of steps d) to f) directly after one another after steps a) to c) have been carried out.

10. The device of claim 9, wherein an infra-red detection device is configured to detect the gestures carried out in three-dimensional space.

11. The device of claim 10, wherein the infra-red detection device has several infra-red sensors arranged in one or more rows.

14. The device of claim 9, wherein the device is further configured to, after the control direction is set, operate the function based on a subsequent detection of the second gesture allocated to the operation of the function in the set control direction.

12. A non-transitory computer-readable medium to control functions displayed on a display unit of a vehicle using gestures carried out in three-dimensional space, wherein the computer-readable medium contains instructions, which when executed by a device, cause the device to:

a) determine whether a first gesture carried out in three-dimensional space is detected;

b) determine, if it is determined that the first gesture has been detected, whether or not the detected first gesture is a gesture allocated to activation of an operation of a function;

c) activate the operation of the function if it is determined that the detected first gesture is the gesture allocated to the activation of the operation of the function;

d) determine whether a second gesture carried out in three-dimensional space is detected;

e) determine, if it is determined that the second gesture has been detected, whether or not the detected second gesture is a gesture allocated to the operation of the function;

f) operate the function if it is determined that the detected first gesture is the gesture allocated to the activation of the operation of the function and the detected second gesture is the gesture allocated to the operation of the function; and

g) set a control direction for the function to be operated in agreement with a direction of a first detection of a second gesture allocated to the operation of the function and sustain the set control direction if repeated implementation of steps d) to f) directly after one another after steps a) to c) have been carried out.

15. The non-transitory computer-readable medium of claim 12, wherein the computer-readable medium contains instructions, which when executed by the device, further cause the device to, after the control direction is set, operate the function based on a subsequent detection of the second gesture allocated to the operation of the function in the set control direction.