Adjusting an electronic display based on gaze tracking

• US 10,514,756 B2
• Filed: 05/05/2017
• Issued: 12/24/2019
• Est. Priority Date: 05/06/2016
• Status: Active Grant

• Current Assignee: Visteon Global Technologies Incorporated (Visteon Corporation)
• Original Assignee: Visteon Global Technologies Incorporated (Visteon Corporation)
• Inventors: Mere, Shadi, Weindorf, Paul Fredrick Luther, Morris, Paul O.
• Primary Examiner(s): Ketema, Benyam
• Application Number:
  US15/587,650
• Publication Number:
  US 20170322625A1
• Time in Patent Office: 963 Days
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G09G 2320/0261 in the context of movement of objects on the screen or movement of the observer relative to the screen
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 2360/144 the light being ambient light
• CPC Class Codes: G09G 5/10 Intensity circuits

Abstract

Disclosed herein are systems, methods, and devices for employing gaze tracking systems with automatic luminance adjustment devices, in particular those commonly used with the Silverstein-method. By employing the aspects disclosed herein, it can been seen that with the disclosed concepts a pupil-size detector may be employed with both a forward looking light sensor, and ambient light sensor, and an adjustment system to utilize the data to determine luminance adjustment.

Claims

1. A system for adjusting an electronic display, comprising:

a gaze tracking device operable to capture an image of a pupil associated with a viewer of the electronic display, wherein the gaze tracking device is operable to determine a diameter of the pupil from the image of the pupil, and communicate a signal indicating the diameter of the pupil to a microcontroller;

a logarithmic ambient light sensor operable to detect ambient light around the electronic display and communicate a logarithmic signal to the microcontroller indicating the detected ambient light around the electronic display;

a logarithmic forward looking light sensor operable to detect light viewed by the viewer of the electronic display and communicate a logarithmic signal to the microcontroller indicating the detected light viewed by the viewer; and

wherein the microcontroller is operable to;

adjust a luminance of the electronic display based on a combination of the diameter of the pupil of the viewer, a detected change in a pupil size, the logarithmic signal from the logarithmic ambient light sensor indicating the detected ambient light around the electronic display, and the logarithmic signal from the logarithmic forward looking light sensor indicating the detected light viewed by the viewer;

initialize a slope to zero;

determine if the viewer is oriented at the gaze tracking device, and in response to this;

introduce a pupil latency delay;

measure the diameter of the pupil and record a time stamp of the measurement;

determine if the pupil is still gazing at the gaze tracking device, and in response to this determination being yes, and in response to a predetermined time elapsing, re-measure the diameter of the pupil;

based on an original diameter of the pupil, the re-measured diameter of the pupil, and a change in time, calculating the slope, an acceleration, a time constant and a steady state pupil diameter value;

re-calculate a relationship with the re-measured diameter of the pupil;

determine whether the time constant calculated is correct;

ascertain a luminance step based on the steady state pupil diameter value; and

determine a final luminance based on the luminance step.

2. The system according to claim 1, wherein the microcontroller is operable to transform the logarithmic signals from both the logarithmic forward looking light sensor and the logarithmic ambient light sensor using an analog-to-digital process.

3. The system according to claim 1, wherein the microcontroller is operable to transform the signal indicating the diameter of the pupil from the gaze tracking device using an analog-to-digital process.

4. The system according to claim 1, wherein the microcontroller is operable to transform the signals from the gaze tracking device, the logarithmic ambient light sensor, and the logarithmic forward looking light sensor using a Silverstein model.

6. The system according to claim 1, wherein the change in the pupil size is corrected with the following relationship:

D=(D_I−

D_F)e<sup>−

T/t</sup>+D_F

wherein D is the corrected pupil diameter, D_I is the initial pupil diameter, D_F is the steady state final pupil diameter, T is the response time constant, and t is the time change from the initial measurement of pupil diameter to the final measurement of pupil diameter.

5. A system for adjusting an electronic display comprising:

a gaze tracking device to capture an image of a pupil associated with a viewer of the electronic display, a diameter of the pupil being ascertained via the image of the pupil;

an ambient light sensor receiver to logarithmically receive light information from an ambient light sensor;

a forward looking light sensor receiver to logarithmically receive light information from a forward looking light sensor;

a microcontroller to adjust a luminance of the electronic display based on a combination of the diameter and the light information received from both the ambient light sensor and the forward looking light sensor;

wherein the light information from both the forward looking light sensor and the ambient light sensor is transformed via an analog-to-digital (A/D) process;

wherein the microcontroller adjusts the luminance based on a detected change in a pupil size; and

wherein the change in the pupil size is corrected with the following relationship;

D=(D_I−

D_F)e<sup>−

T/t</sup>+D_F

where;

D=Pupil Diameter

DI=Initial Pupil Diameter

DF=Final Pupil Diameter (steady state)

T=Response Time Constant

t=time change from initial measurement to final measurement.

7. A system for adjusting an electronic display, the system comprising:

a gaze tracking device operable to capture an image of a pupil associated with a viewer of the electronic display;

a logarithmic ambient light sensor operable to detect ambient light around the electronic display;

a logarithmic forward looking light sensor operable to detect light viewed by the viewer of the electronic display;

a microcontroller including a processor and a memory having an algorithm stored thereon, wherein the processor is operable to execute the algorithm to;

receive the image of the pupil from the gaze tracking device;

calculate an original diameter of the pupil;

receive a logarithmic signal from the logarithmic ambient light sensor, wherein the logarithmic signal from the logarithmic ambient light sensor indicates an amount of ambient light around the electronic display;

receive a logarithmic signal from the logarithmic forward looking light sensor, wherein the logarithmic signal from the logarithmic forward looking light sensor indicates an amount of light viewed by the viewer of the electronic display;

detect a change in the diameter of the pupil of the viewer;

correct the detected change in the diameter of the pupil of the viewer with the following relationship;

D=(D_I−

D_F)e<sup>−

T/t</sup>+D_F

wherein D is the corrected pupil diameter, D_I is the initial pupil diameter, D_F is the steady state final pupil diameter, T is the response time constant, and t is the time change from the initial measurement of pupil diameter to the final measurement of pupil diameter;

determine a final luminance for the electronic display based on the original diameter of the pupil, the logarithmic signal from the logarithmic ambient light sensor indicating the amount of ambient light around the electronic display, and the logarithmic signal from the logarithmic forward looking light sensor indicating the amount of light viewed by the viewer of the electronic display; and

adjust the luminance of the electronic display to achieve the final luminance.

8. The system set forth in claim 7, wherein the processor is operable to execute the algorithm to:

re-calculate the final luminance based on the change in the diameter of the pupil of the viewer; and

re-adjust the luminance of the electronic display to achieve the re-calculated final luminance.