Stereoscopic Viewing Comfort Through Gaze Estimation

• US 20110228051A1
• Filed: 03/16/2011
• Published: 09/22/2011
• Est. Priority Date: 03/17/2010
• Status: Abandoned Application

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Dedeoglu, Goksel, Batur, Aziz Umit
• Application Number:
  US13/049,714
• Publication Number:
  US 20110228051A1
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes

Abstract

A method of improving stereo video viewing comfort is provided that includes capturing a video sequence of eyes of an observer viewing a stereo video sequence on a stereoscopic display, estimating gaze direction of the eyes from the video sequence, and manipulating stereo images in the stereo video sequence based on the estimated gaze direction, whereby viewing comfort of the observer is improved.

Claims

1. A method of improving stereo video viewing comfort, the method comprising:

capturing a video sequence of eyes of an observer viewing a stereo video sequence on a stereoscopic display;

estimating gaze direction of the eyes from the video sequence; and

manipulating stereo images in the stereo video sequence based on the estimated gaze direction, whereby viewing comfort of the observer is improved.

2. The method of claim 1, wherein manipulating stereo images comprises:

computing disparity between a left stereo image and a right stereo image in the stereo video sequence;

computing a representative disparity value; and

adjusting horizontal shift for the stereoscopic display based on the representative disparity value.

3. The method of claim 2, wherein adjusting horizontal shift comprises setting a horizontal shift parameter to −

d, wherein d is the representative disparity value.

4. The method of claim 2, wherein

computing disparity generates a disparity image;

and

computing a representative disparity value comprises;

determining a region of interest in the disparity image based on the estimated gaze direction; and

computing the representative disparity value in the region of interest.

5. The method of claim 2, wherein

computing disparity comprises:

determining a region of interest in the left stereo image and the right stereo image based on the estimated gaze direction; and

computing disparity over the region of interest to generate a disparity region of interest; and

computing a representative disparity value comprises computing the representative disparity value in the disparity region of interest.

6. The method of claim 2, further comprising:

computing on-screen parallax based on the estimated gaze direction; and

adjusting horizontal shift comprises adjusting the horizontal shift based on the representative disparity value and the on-screen parallax.

7. The method of claim 6, wherein adjusting horizontal shift comprises:

changing the horizontal shift based on a difference between the on-screen parallax and the representative disparity value; and

adjusting the horizontal shift incrementally to achieve zero disparity.

8. The method of claim 7, wherein zero disparity is achieved when the horizontal shift has a value equal to −

d, wherein d is the representative disparity value.

9. The method of claim 1, wherein manipulating stereo images comprises:

adjusting orientations of stereo video cameras capturing the stereo video sequence based on the estimated gaze direction.

10. A method of improving stereo video viewing comfort, the method comprising:

capturing continuously a video sequence of eyes of an observer viewing at least a portion of a first stereo video sequence on a stereoscopic display;

estimating gaze directions of the eyes from the video sequence;

computing convergence points of the eyes based on the estimated gaze directions;

analyzing the computed convergence points to determine a minimum convergence depth and a maximum convergence depth; and

using the minimum and maximum convergence depth to adjust horizontal shift of the stereoscopic display as the observer views a second stereo video sequence, whereby viewing comfort of the observer is improved.

11. A stereoscopic display system comprising:

a stereo video source configured to provide a stereo video sequence;

a stereoscopic display configured to display the stereo video sequence;

an eye video capture component configured to capture a video sequence of eyes of an observer viewing the stereo video sequence on the stereoscopic display; and

an eye tracking component configured to estimate gaze direction of the eyes from the video sequence,

wherein stereo images in the stereo video sequence are manipulated based on the estimated gaze direction, whereby viewing comfort of the observer is improved.

12. The stereoscopic display system of claim 11, wherein the stereo video source is configured to adjust orientations of stereo video cameras capturing the stereo video sequence based on the estimated gaze direction.

13. The stereoscopic display system of claim 11, further comprising:

a disparity estimation component configured to compute disparity between a left stereo image and a right stereo image in the stereo video sequence, and

wherein the eye tracking component is further configured to

compute a representative disparity value from the estimated disparity; and

adjust horizontal shift for the stereoscopic display based on the representative disparity value.

14. The stereoscopic display system of claim 13, wherein the eye tracking component is configured to adjust horizontal shift by setting a horizontal shift parameter to −

d, wherein d is the representative disparity value.

15. The stereoscopic display system of claim 13,

wherein the disparity estimation component is configured to compute disparity by generating a disparity image, and

wherein the eye tracking component is configured to compute the representative disparity value by

determining a region of interest in the disparity image based on the estimated gaze direction;

and

computing the representative disparity value in the region of interest.

16. The stereoscopic display system of claim 13,

wherein the disparity estimation component is configured to compute disparity by

determining a region of interest in the left stereo image and the right stereo image based on the estimated gaze direction;

and

computing disparity over the region of interest to generate a disparity region of interest; and

wherein the eye tracking component is configured to compute the representative disparity value in the disparity region of interest.

17. The stereoscopic display system of claim 13, wherein the eye tracking component is further configured to

compute on-screen parallax based on the estimated gaze direction;

and

adjust the horizontal shift based on the representative disparity value and the on-screen parallax.

18. The stereoscopic display system of claim 17, wherein the eye tracking component is further configured to adjust the horizontal shift by

changing the horizontal shift based on a difference between the on-screen parallax and the representative disparity value;

and

adjusting the horizontal shift incrementally to achieve zero disparity.