Operating device with eye tracker unit and method for calibrating an eye tracker unit of an operating device

• US 10,635,170 B2
• Filed: 06/06/2017
• Issued: 04/28/2020
• Est. Priority Date: 06/10/2016
• Status: Active Grant

• Current Assignee: Volkswagen AG
• Original Assignee: Volkswagen AG
• Inventors: Yosha, Omer, Perkuhn, Janine, Schulz, Stefan, Tendyck, Christian, Werner, Thomas, Schwarz, Thomas
• Primary Examiner(s): Findley, Christopher G
• Application Number:
  US16/307,918
• Publication Number:
  US 20190265788A1
• Time in Patent Office: 1,057 Days
• Field of Search: None
• CPC Class Codes: B60K 2360/122 Instrument input devices with reconfigurable control functions, e.g. reconfigurable menus
• CPC Class Codes: B60K 2360/141 Activation of instrument input devices by approaching fingers or pens
• CPC Class Codes: B60K 2360/149 Instrument input by detecting viewing direction not otherwise provided for
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/10 Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
• CPC Class Codes: B60K 35/654 the user being the driver
• CPC Class Codes: B60K 35/658 the instruments being ergonomically adjustable to the user
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/0093 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/18 Eye characteristics, e.g. of the iris

Abstract

The invention relates to a method for calibrating an eye tracker unit of an operating device, wherein at least one apparatus is controlled by means of the operating device depending on at least one user input and wherein, in the case of the method, at least one graphical element is displayed respectively by a control unit of the operating device in at least one predetermined usage step of the operating device and at least one piece of viewing direction information is determined by means of the eye tracker unit for the user viewing a screen. The invention provides that, from the at least one displayed element, an optically salient element, on which the user must focus with their eyes in order to successfully perform the usage step, is established, and calibration data for the calibration of the eye tracker unit is generated depending on a display position of the optically salient element on the screen and depending on the at least one piece of viewing direction information.

Claims

1. A method for calibrating an eye tracker unit of an operating device, wherein by the operating device at least one apparatus can be controlled depending on at least one user input and wherein in the method by a control unit of the operating device in at least one predetermined usage step of the operating device respectively:

on a screen, at least one graphical operating element for receiving the at least one user input and/or at least one output element for describing an apparatus state of the at least one controlled apparatus and/or at least one media element for describing a situation is displayed, so that at least one graphical element is displayed during the usage step; and

at least one viewing direction information is determined for the user viewing the screen by the eye tracker unit, wherein

from the at least one displayed element, an optically salient element, which meets a predetermined salience criterion and on which the user must focus with their eyes in order to successfully perform the usage step, is determined, and calibration data for the calibration of the eye tracker unit is generated depending on a display position of the optically salient element on the screen and depending on the at least one viewing direction information, and wherein

a plurality of graphical elements are displayed, wherein

a first of the graphical elements is optically marked clear or unchanged bright or colored or by a marking element or a combination thereof during the usage step and is hereby shown in an optically salient manner to steer a user'"'"'s focus to the first of the graphical elements; and

wherein

all remaining graphical elements are displayed intermittently unclear or intermittently shaded or intermittently only in shades of gray or a combination thereof.

2. The method according to claim 1, wherein a plurality of graphical elements are displayed and these are displayed appearing in succession chronologically and/or one is currently displayed in a respective animation of its size and/or shape and/or position and/or location in a changing manner.

3. The method according to claim 1, wherein a plurality of graphical elements are displayed and it is determined which of the displayed elements have a highest salience value determined according to the salience criterion, and the element with the highest salience value is established as the optically salient element.

4. The method according to claim 1, wherein in the specified flow of a usage step, the respective graphical element to be operated and/or viewed next is established in advance as the optically salient element.

5. The method according to claim 1, wherein only one single graphical element, in particular an operating element for receiving the at least one user input, is displayed and established as the optically salient element.

6. The method according to claim 1, wherein based on the at least one viewing direction information a lingering of a gaze of the user within a focus area with a predetermined area size for more than a predetermined minimum duration is recognized and, upon recognition of the lingering, the calibration data is generated from viewing direction information which was determined during the lingering.

7. The method according to claim 1, wherein at least one preliminarily calibrated viewing direction information is calculated based on predetermined default calibration data and only at least one such viewing direction information is used for generating the calibration data, which correspond with coordinates, which fulfill a predetermined plausibility criterion.

8. The method according to claim 7, wherein multiple sets of viewing direction information are received, which comprise the plausibility criterion, wherein the coordinates of all selected viewing direction information correspond with the relative position change in the case of a change and/or a movement of the optically salient element.

9. The method according to claim 1, wherein the screen is designed as a touchscreen and the at least one user input is recognized as a touch event by the screen and in the case of a recognized touch event the viewing direction information generated respectively before and/or during and/or after the touch event and/or coordinates of the touched graphical element on the screen and/or touch coordinates of the touch event are taken as a basis to generate the calibration data.

10. The method according to claim 9, wherein a control panel for entering a button actuation and/or a keyboard for receiving a character string is displayed.

11. The method according to claim 9, wherein a slider element for entering a swipe gesture along a movement line is displayed as a graphical operating element and multiple sets of viewing direction information describing a movement progression along the movement line are received and used to generate the calibration data.

12. The method according to claim 1, wherein a video with a user guide for the operating device and/or the at least one apparatus is displayed as the media element and, at least one point in time, a graphical element momentarily displayed in the video and announced via an audio response and/or optically highlighted in the video is established as the respective optically salient element.

13. The method according to claim 1, wherein a plurality of usage steps are performed in succession and calibration data is respectively generated and the eye tracker unit is iteratively calibrated with the calibration data from each usage step.

14. The method according to claim 10, wherein a slider element for entering a swipe gesture along a movement line is displayed as a graphical operating element and multiple sets of viewing direction information describing a movement progression along the movement line are received and used to generate the calibration data.

15. An operating device for a motor vehicle, having an eye tracker unit, a screen and a control unit, configured to perform the method according to claim 1.