Method for inputting a control command for a component of a motor vehicle

• US 9,612,655 B2
• Filed: 10/18/2013
• Issued: 04/04/2017
• Est. Priority Date: 10/31/2012
• Status: Active Grant

• Current Assignee: Audi AG (Volkswagen AG)
• Original Assignee: Audi AG (Volkswagen AG)
• Inventors: Kuehner, Manuel, Mueller, Ulrich, Schneider, Johann, Kuehne, Marcus
• Primary Examiner(s): Sitta, Grant
• Application Number:
  US14/438,986
• Publication Number:
  US 20150301591A1
• Time in Patent Office: 1,264 Days
• Field of Search: 345156
• CPC Class Codes: B60K 2360/146 Instrument input by gesture
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 20/653 by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: H04N 23/20 for generating image signals from infrared radiation only
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A method inputs a control command for a component of a motor vehicle. The method involves generating an image sequence of an input object guided by a user in a specified detection region using an imaging device, detecting a change in position of the input object on the basis of the image sequence, and generating a control command for the component of the motor vehicle on the basis of the detected change in position. The imaging device employs at least one infrared-sensitive camera, and the detection region is illuminated using at least one infrared source.

Claims

1. A method for inputting a control command for a component of a motor vehicle, comprising:

acquiring a calibration image for an input object, the calibration image being determined when the input object is at a predetermined position within a predetermined acquisition region, in order to calibrate a skeleton model to the input object;

generating an image sequence for the input object, the image sequence being generated by imaging the input object within the acquisition region as the user guides the input object through the acquisition region, the input object being imaged by an imaging device comprising an infrared camera;

illuminating the input object within the acquisition region using an infrared source so that infrared light from the infrared source is scattered by the input object back to the camera;

identifying a position change of the input object based on the image sequence by;

adapting a first image of the image sequence to the skeleton model using first parameters;

adapting a second image of the image sequence to the skeleton model using second parameters; and

comparing the first and second parameters;

determining position information from a ratio between a size of the input object in the calibration image and a size of the input object in at least one image of the image sequence, and from a position of the input object along an axis parallel to a viewing direction of the camera determined based on a luminous intensity of the infrared light scattered back to the camera; and

generating a control command for, and inputting the control command to, the component of the motor vehicle based on the position change and the position information,

wherein

a duration for which the camera is allowed to be exposed to the infrared light scattered back to the camera is varied cyclically by exposing the camera to the infrared light scattered back to the camera for a first period of time and exposing the camera to the infrared light scattered back to the camera for a second period of time which is greater than the first period of time.

2. The method as claimed in claim 1, wherein

coordinates of a point on the input object are acquired on a detector matrix of the camera, and

a position of the point on the input object in a plane perpendicular to a viewing direction of the camera is determined using the coordinates of the point on the detector matrix of the camera.

3. The method as claimed in claim 2, wherein

first and second infrared cameras are used to obtain the image sequence, and

the first and second infrared cameras have pixels and detector matrices that are positionally offset from one another.

4. The method as claimed in claim 1, wherein

there are two points on the input object that are separated by a known object distance, and

in images of the input object, the two points on the input object are separated by an image distance, and

the position of the input object along an axis parallel to a viewing direction of the camera is determined from a ratio of the image distance to the known object distance.

5. The method as claimed in claim 4, wherein

the position information determines an absolute distance of the input object from the imaging device along the axis parallel to the viewing direction and

the ratio of the image distance to the known object distance determines a distance change of the input object along the axis parallel to the viewing direction.

6. The method as claimed in claim 1, wherein

an illumination power of the infrared source is varied cyclically between at least two predetermined power values.

7. The method as claimed in claim 1, wherein

the duration for which the camera is allowed to be exposed to the infrared light scattered back to the camera is varied cyclically by exposing the camera to the infrared light scattered back to the camera for a third period of time which is greater than the first period of time and the second period of time.

8. The method as claimed in claim 1, wherein

the motor vehicle has a vehicle-fixed object with a known position,

an image of the vehicle-fixed object is acquired by the infrared camera, and

a position of the input object relative to the vehicle-fixed object is determined based on the image of the vehicle-fixed object.

9. The method as claimed in claim 1, wherein the position change is identified and the position information is determined without stereoscopic image recognition.

10. The method as claimed in claim 1, wherein the skeleton model relates to a skeleton of a human hand.

11. The method as claimed in claim 10, wherein the first and second parameters relate to flexion angles of finger joints of the human hand.

12. The method as claimed in claim 1, wherein the calibration image is obtained in an initialization phase to determine a size of a hand of the user.

13. The method as claimed in claim 1, wherein the position change identifies movements in a hand of the user.

14. The method as claimed in claim 1, wherein the position information determines a distance of the input object from the imaging device.

15. The method as claimed in claim 1, wherein a single infrared camera is used as the imaging device.

16. The method as claimed in claim 1, wherein the position change and the position information identify how the user is interacting with the component of the motor vehicle.

19. The method as claimed in claim 1, wherein an illumination power of the infrared source is varied cyclically by using at least a first predetermined power setting greater than zero for a first period of time and a second predetermined power setting for a second period of time which is greater than the first predetermined power setting.

20. The method as claimed in claim 1, further comprising discarding images among the image sequence that are determined to be overly exposed and underexposed.

17. A motor vehicle comprising:

an electronic apparatus; and

a gesture control device for acquiring an operating gesture of an input object in a passenger compartment of the motor vehicle, the gesture control device comprising;

an acquisition unit to acquire a calibration image for the input object, the calibration image being determined when the input object is at a predetermined position within the passenger compartment, in order to calibrate a skeleton model to the input object;

an imaging device comprising an infrared camera to generate an image sequence for the input object, by imaging the input object within the passenger compartment as the user guides the input object through the passenger compartment;

an infrared source to illuminate the input object within the passenger compartment so that infrared light from the infrared source is scattered by the input object back to the camera; and

a position change unit to identify a position change of the input object based on the image sequence by;

adapting a first image of the image sequence to the skeleton model using first parameters;

adapting a second image of the image sequence to the skeleton model using second parameters; and

comparing the first and second parameters;

a position information unit to determine position information from a ratio between a size of the input object in the calibration image and a size of the input object in at least one image of the image sequence, and from a position of the input object along an axis parallel to a viewing direction of the camera, the position being determined based on a luminous intensity of the infrared light scattered back to the camera; and

a command unit to generate a control command for, and input the control command to, the electronic device, based on the position change and the position information,

wherein

a duration for which the camera is allowed to be exposed to the infrared light scattered back to the camera is varied cyclically by exposing the camera to the infrared light scattered back to the camera for a first period of time and exposing the camera to the infrared light scattered back to the camera for a second period of time which is greater than the first period of time.

18. A method for inputting a control command for a component of a motor vehicle, comprising:

acquiring a calibration image for an input object, the calibration image being determined when the input object is at a predetermined position within a predetermined acquisition region, in order to calibrate a skeleton model to the input object;

generating an image sequence for the input object, the image sequence being generated by imaging the input object within the acquisition region as the user guides the input object through the acquisition region, the input object being imaged by an imaging device comprising an infrared camera;

illuminating the input object within the acquisition region using an infrared source so that infrared light from the infrared source is scattered by the input object back to the camera,

identifying a position change of the input object based on the image sequence by;

adapting a first image of the image sequence to the skeleton model using first parameters,

adapting a second image of the image sequence to the skeleton model using second parameters, and

comparing the first and second parameters;

determining position information from a ratio between a size of the input object in the calibration image and a size of the input object in at least one image of the image sequence, and from a position of the input object along an axis parallel to a viewing direction of the camera determined based on a luminous intensity of the infrared light scattered back to the camera; and

generating a control command for, and inputting the control command to, the component of the motor vehicle based on the position change and the position information,

wherein

a duration for which the camera is allowed to be exposed to the infrared light scattered back to the camera is varied cyclically by exposing the camera to the infrared light scattered back to the camera for a first period of time and exposing the camera to the infrared light scattered back to the camera for a second period of time which is greater than the first period of time,

a first infrared camera and a second infrared camera are used to obtain the image sequence,

the first infrared camera has first pixels and a first detector matrix from which first two-dimensional coordinates of a first point on the input object in a plane perpendicular to a viewing direction of the first infrared camera are acquired,

the second infrared camera has second pixels and a second detector matrix from which second two-dimensional coordinates of a second point on the input object in a plane perpendicular to a viewing direction of the second infrared camera are acquired,

the second pixels and the second detector matrix of the second infrared camera partially overlap with and are positionally offset from the first pixels and the first detector matrix of the first infrared camera, and

a two-dimensional coordinate position of the input object is determined using the first two-dimensional coordinates obtained by the first infrared camera and the second two-dimensional coordinates obtained by the second infrared camera.