Method for calibrating a camera for a gaze direction detection in a vehicle, device for a motor vehicle having a camera and at least one further element, and computer program product

• US 10,068,143 B2
• Filed: 02/28/2016
• Issued: 09/04/2018
• Est. Priority Date: 03/10/2015
• Status: Active Grant

• Current Assignee: Robert Bosch GmbH
• Original Assignee: Robert Bosch GmbH
• Inventors: Niem, Wolfgang
• Primary Examiner(s): Elahi, Shan E
• Application Number:
  US15/055,608
• Publication Number:
  US 20160267336A1
• Time in Patent Office: 919 Days
• Field of Search: 348 78
• CPC Class Codes: B60W 2040/0872 Driver physiology
• CPC Class Codes: B60W 40/08 related to drivers or passengers
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06T 2207/30201 Face
• CPC Class Codes: G06T 2207/30268 Vehicle interior
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris

Abstract

A method for calibrating a camera for a gaze direction detection in a vehicle, to a device for a motor vehicle having a camera and at least one further element, and to a computer program product. The at least one further element is situated in a fixed geometric relationship with the camera the calibration takes place by detecting, in images of the camera, at least one relative change in the gaze direction in which a user changes his gaze in the direction of the further element. The method is characterized in that it is detected by at least one action of the user that the gaze change of the user has taken place or will take place. For the gaze direction then offers an advantage as a result of a dual benefit: it is used both for the calibration and for the action of the user.

Claims

1. A method for calibrating a camera for detecting an absolute gaze direction of a user in a vehicle, the method comprising:

situating at least one operating element and/or display element in a fixed spatial-geometric relationship with the camera;

detecting, in images of the camera, at least one relative change in a gaze direction in which the user changes a gaze so as to gaze in a direction of the at least one operating and/or display element;

calibrating the camera by detecting, based on at least one action of the user, the at least one relative change in the gaze direction of the user; and

synchronizing a video time stamp of the camera with at least one of an operating event and a display event of the at least one operating element and/or display element that triggers the action or is triggered by the action.

2. The method as recited in claim 1, further comprising:

detecting the absolute gaze direction while using the relative gaze direction change and the fixed spatial-geometric relationship.

3. The method as recited in claim 2, wherein camera parameters of the camera from a band-end calibration are utilized for ascertaining the absolute gaze direction.

4. The method as recited in claim 2, wherein multiple absolute gaze directions are ascertained, which are utilized for calibrating the gaze direction detection.

5. The method as recited in claim 2, wherein the calibrating includes ascertaining a head position and an eye position in an image recorded by the camera, and ascertaining the relative change in the gaze direction based on the ascertained head position and the ascertained eye position.

6. The method as recited in claim 1, wherein the at least one operating and/or display element includes at least one of a radio, an automotive navigation system, a climate control system, and the action includes at least one of an operation of a pushbutton, operation of a touch pad, and operation of a touch screen.

7. The method as recited in claim 1, wherein the at least one operating and/or display element includes at least one of a rear view mirror, and a screen, and the action includes a driving maneuver.

8. A device for calibrating a camera for detecting an absolute gaze direction of a user in a motor vehicle, comprising:

a camera;

at least one operating element and/or display element situated in a fixed spatial-geometric relationship with the camera; and

a calibrating arrangement configured to detect, in images of the camera, at least one relative change in a gaze direction in which the user changes a gaze in a direction of the at least one element, and to provide calibrating of the camera by detecting, based on at least one action of the user, the at least one relative change in the gaze direction of the user, and to synchronize a video time stamp of the camera with at least one of an operating event and a display event of the at least one operating element and/or display element that triggers the action or is triggered by the action.

9. A non-transitory computer-readable storage device having a computer program, which is executable by a processor, comprising:

a program code arrangement having program code for calibrating a camera for detecting an absolute gaze direction of a user in a motor vehicle, by performing the following;

detecting, in images of the camera, at least one relative change in a gaze direction in which a user changes a gaze so as to gaze in a direction of the at least one operating element and/or display element situated in a fixed spatial-geometric relationship with the camera; and

calibrating the camera by detecting, based on at least one action of the user, the at least one relative change in the gaze direction of the user; and

synchronizing a video time stamp of the camera with at least one of an operating event and a display event of the at least one operating element and/or display element that triggers the action or is triggered by the action.