Process and system for encoding and playback of stereoscopic video sequences

• US 10,341,643 B2
• Filed: 10/24/2016
• Issued: 07/02/2019
• Est. Priority Date: 04/09/2002
• Status: Active Grant

• Current Assignee: 3DN, LLC
• Original Assignee: 3DN, LLC
• Inventors: Routhier, Nicholas, Thibeault, Claude, Belzile, Jean, Malouin, Daniel, Carpentier, Pierre-Paul, Dallaire, Martin
• Primary Examiner(s): Philippe, Gims S
• Application Number:
  US15/332,772
• Publication Number:
  US 20170041595A1
• Time in Patent Office: 981 Days
• Field of Search: 37524019, 345 6, 345 7, 345102, 348 56
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 13/139 Format conversion, e.g. of frame-rate or size
• CPC Class Codes: H04N 13/144 for flicker reduction
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/167 Synchronising or controlling image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/339 using spatial multiplexing H04N13/337 takes precedence
• CPC Class Codes: H04N 13/341 using temporal multiplexing
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 2013/0096 Synchronisation or controlling aspects
• CPC Class Codes: H04N 2213/002 Eyestrain reduction by processing stereoscopic signals or controlling stereoscopic devices
• CPC Class Codes: H04N 2213/007 Aspects relating to detection of stereoscopic image format, e.g. for adaptation to the display format

Abstract

A method for decoding a compressed image stream, the image stream having a plurality of frames, each frame consisting of a merged image including pixels from a left image and pixels from a right image. The method involves the steps of receiving each merged image; changing a clock domain from the original input signal to an internal domain; for each merged image, placing at least two adjacent pixels into an input buffer and interpolating an intermediate pixel, for forming a reconstructed left frame and a reconstructed right frame according to provenance of the adjacent pixels; and reconstructing a stereoscopic image stream from the left and right image frames. The invention also teaches a system for decoding a compressed image stream.

Claims

1. A method for displaying images, comprising:

in a conventional two-dimensional (2D) viewing mode;

driving a 2D image of a plurality of 2D images to a head mountable display; and

in a stereoscopic three-dimensional (3D) viewing mode;

driving to the head mountable display a left image of a plurality of left images for left eye viewing, wherein at least a portion of the plurality of left images are generated using time interpolation to increase a frame rate of the display, and a right image of a plurality of right images for right eye viewing, the left image and the right image being time-synchronized with parallax for perception of depth in the stereoscopic 3D viewing mode, wherein the driving comprises simultaneous dual presentation of the left image and the right image.

2. The method of claim 1, further comprising:

in the stereoscopic 3D viewing mode, time interpolating the left image and another left image of the plurality of left images using movement anticipation.

3. The method of claim 1, further comprising:

time-interpolating at least a portion of the plurality of left images to improve fluidity of playback.

4. The method of claim 1, wherein variations in clock signals are accommodated by a rate controller to avoid de-synchronization.

5. The method of claim 1, further comprising, in the stereoscopic 3D viewing mode:

monitoring variations in clock signals; and

correcting an output frequency to maintain synchronicity.

6. The method of claim 1, further comprising:

proportionally decreasing colours of at least one pixel in the left image when at least one colour value of the at least one pixel is above a certain value to decrease flicking.

7. The method of claim 1, wherein the head mountable display is a first head mountable display, the method further comprising in the stereoscopic 3D viewing mode:

driving one of the left image or the right image to a second non head mountable display for display in a conventional 2D viewing mode.

8. The method of claim 1, further comprising:

receiving at least one image at a digital input over an internet connection.

9. The method of claim 1, wherein the driving includes an output frame frequency that is controlled by removing pixels from a left image or a right image.

10. A system comprising:

an image buffer for storing image data; and

an output controller for communicating with a head mountable display according to a plurality of viewing modes including a conventional two-dimensional (2D) viewing mode and a stereoscopic three-dimensional (3D) viewing mode, the output controller having access to the image buffer for reading data therein and for generating therefrom an output signal, wherein;

in the conventional 2D viewing mode the output controller drives a 2D image of a plurality of 2D images to the head mountable display, and

in the stereoscopic 3D viewing mode, the output controller drives a left image of a plurality of left images and a right image of a plurality of right images to the head mountable display, wherein at least a portion of the plurality of the left images are generated using time interpolation to increase a frame rate of the display, the left and right images being time-synchronized with parallax for perception of depth, wherein the driving comprises simultaneous dual presentation of the left image and the right image.

11. The system of claim 10, further comprising a temporal interpolator configured to, in the stereoscopic 3D viewing mode, time interpolate the left image and another left image of the plurality of left images using movement anticipation.

12. The system of claim 11, wherein the temporal interpolator is configured to time-interpolate at least a portion of the plurality of left images to improve fluidity of playback.

13. The system of claim 10, further comprising a rate controller configured to accommodate variations in clock signals to avoid de-synchronization.

14. The system of claim 10, further comprising a temporal interpolator and a rate controller, the rate controller being configured to, in the stereoscopic 3D viewing mode, monitor variations in clock signals and correct output frequency to maintain synchronicity.

15. The system of claim 10, further comprising an anti-flicker filter configured to decrease colours of at least one pixel in the left image proportionally when at least one colour of the at least one pixel is above a certain value to decrease flicking.

16. The system of claim 10, wherein the head mountable display is a first display, and wherein in the stereoscopic 3D viewing mode, the output controller drives one of the left image or the right image to a second display for display in a conventional 2D display mode.

17. The system of claim 10, further comprising a digital input connectable to the internet, the digital input configured to receive at least one image over an internet connection.

18. The system of claim 10, wherein the output controller controls a frequency of the output signal by removing pixels from a left image or a right image.

19. A method comprising:

receiving stereoscopic three-dimensional (3D) images, the images including a left image of a plurality of left images and a right image of a plurality of right images, the left image and the right image being time-synchronized with parallax for perception of depth,

topologically decimating at least a portion of the left image or the right image using a first pixel mosaic,

reconstructing the topologically decimated left image or the topologically decimated right image using interpolation,

driving a head mountable display wherein the driving comprises simultaneous dual presentation of the reconstructed left image and the reconstructed right image.

20. The method of claim 19, wherein the topologically decimating at least a portion of the left image or the right image using the first pixel mosaic and a second pixel mosaic.