APPARATUS FOR PROCESSING A STEREOSCOPIC IMAGE STREAM

• US 20100171814A1
• Filed: 01/22/2010
• Published: 07/08/2010
• Est. Priority Date: 04/09/2002
• Status: Active Grant

• Current Assignee: 3DN, LLC
• Original Assignee: SENSIO Technologies, Inc.
• Inventors: FORTIN, Etienne, ROUTHIER, Pierre-Hugues, MALOUIN, Daniel, ROUTHIER, Nicholas
• Granted Patent: US 8,384,766 B2
• Time in Patent Office: Days
• US Class Current: 348/46
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 13/139 Format conversion, e.g. of frame-rate or size
• CPC Class Codes: H04N 13/144 for flicker reduction
• CPC Class Codes: H04N 13/161 Encoding, multiplexing or demultiplexing different image signal components for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/167 Synchronising or controlling image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/339 using spatial multiplexing H04N13/337 takes precedence
• CPC Class Codes: H04N 13/341 using temporal multiplexing
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/398 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 2013/0096 Synchronisation or controlling aspects
• CPC Class Codes: H04N 2213/002 Eyestrain reduction by processing stereoscopic signals or controlling stereoscopic devices
• CPC Class Codes: H04N 2213/007 Aspects relating to detection of stereoscopic image format, e.g. for adaptation to the display format

Abstract

A system is provided for processing a compressed image stream of a stereoscopic image stream, the compressed image stream having a plurality of frames in a first format, each frame consisting of a merged image comprising pixels sampled from a left image and pixels sampled from a right image. A receiver receives the compressed image stream and a decompressing module in communication with the receiver decompresses the compressed image stream. The left and right images of the decompressed image stream are stored in a frame buffer. A serializing unit reads pixels of the frames stored in the frame buffer and outputs a pixel stream comprising pixels of a left frame and pixels of a right frame. A stereoscopic image processor receives the pixel stream, buffers the pixels, performs interpolation in order to reconstruct pixels of the left and right images and outputs a reconstructed left pixel stream and a reconstructed right pixel stream, the reconstructed streams having a format different from the first format. A display signal generator receives the stereoscopic pixel stream to provide an output display signal.

Claims

1. A system for processing a compressed image stream of a stereoscopic image stream, said compressed image stream having a plurality of frames in a first format, each frame consisting of a merged image comprising pixels sampled from a left image and pixels sampled from a right image, comprising:

a receiver for receiving said compressed image stream;

a decompressing module in communication with said receiver for decompressing said compressed image stream;

a frame buffer connected to said decompressing module for receiving and storing frames of said merged image of said decompressed image stream;

a serializing unit for reading pixels of said frames stored in said frame buffer and outputting a pixel stream comprising pixels of a left frame and pixels of a right frame;

a stereoscopic image processor receiving said pixel stream, buffering said pixels, performing interpolation in order to reconstruct pixels of said left and right images and outputting a reconstructed left pixel stream and a reconstructed right pixel stream, said reconstructed streams having a format different from said first format;

a display signal generator receiving said stereoscopic pixel stream to provide an output display signal.

2. A system as claimed in claim 1, wherein said stereoscopic image processor comprises:

an interpolation module for forming a reconstructed left frame and a reconstructed right frame from said pixel stream;

an output module for outputting said reconstructed stereoscopic image stream in a format different from said first format, said Output module comprising;

a display mode controller for selecting one of a plurality of stereoscopic display modes and outputting said reconstructed frames in said selected mode.

3. A system as claimed in claim 2, wherein said output module further comprises:

a display rate controller for controlling a display rate of said frames.

4. A system as claimed in claim 1, wherein said compressed image stream is stored as a DVD and further comprising a DVD reader providing said compressed image stream to said receiver.

5. A system as claimed in claim 1, wherein said receiver, said decompressing module, said frame buffer and said serializing unit are integrated into a single chip receiving said compressed image stream and outputting said pixel stream.

6. A system as claimed in claim 2, said plurality of stereoscopic display modes comprises at least two of a page flipping, line interleaved, a column interleaved, an anaglyphic, a dual stream display and a monoscopic display mode.

7. A system as claimed in claim 1, further comprising a synchronization module for synchronizing shutter eyeglasses.

8. A system as claimed in claim 3, wherein said display rate controller comprises a temporal interpolator and wherein said controlling said display rate of said frames comprises time interpolation of said frames.

9. A system as claimed in claim 1, wherein said serializing unit and said display signal generator are driven by a common pixel clock.

10. A system as claimed in claim 9, wherein said stereoscopic image processor uses said common pixel clock and outputs said stereoscopic pixel stream with a 5-line delay.

11. A system as claimed in claim 9, wherein said stereoscopic image processor uses said common pixel clock and outputs said stereoscopic pixel stream with one image frame delay.

12. A method for transmitting stereoscopic left and right image frames, a left image frame and a right image frame forming a pair, comprising:

for each of said left and right image frames, sampling said image frame such that said sampled image frame has half a number of original pixels forming a staggered quincunx pattern, wherein said original pixels surround missing pixels;

for each left and right image frame pair, creating a new frame by juxtaposing a sampled image frame of a left image and a sampled image frame of a right image;

transmitting said new frames in an image stream;

receiving said image stream of new frames;

reconstructing said stereoscopic left and right image frames by interpolating missing pixel values for each image frame, using only said original pixels, wherein said interpolating comprises, for each missing pixel;

calculating an edge-dependent weighting factor using said original pixel values including values of original pixels other than said original pixels immediately surrounding said missing pixel; and

calculating an interpolated value for said missing pixel using original pixels and said weighting factor.

13. A method as claimed in claim 12, wherein said calculating an edge-dependent weighting factor comprises:

calculating a horizontal change parameter by using percentages of said original pixel values in the columns immediately to the left and immediately to the right of said missing pixel;

calculating a vertical change parameter by using percentages of said original pixel values in the rows immediately above and immediately below said missing pixel;

computing said edge-dependent weighting factor from said horizontal factor and said vertical factor,

and wherein said calculating an interpolated value for said missing pixel, comprises;

calculating a vertical component by applying a filter to at least one neighboring pixel to the left and at least one neighboring pixel to right of said missing pixel, said neighboring pixels being located along a same horizontal line;

calculating a horizontal component by applying a filter to at least one neighboring pixel above and at least one neighboring pixel below said missing pixel, said neighboring pixels being located along a same vertical line;

computing said missing pixel value from said weighting factor, said horizontal component and said vertical component.

14. A method as claimed in claim 13, wherein said filter is the arithmetic mean of two pixels.

15. A method as claimed in claim 12, wherein said calculating an edge-dependent factor comprises:

providing a horizontal edge sensitivity parameter,

calculating horizontal weighting factors by using said horizontal edge sensitivity parameter and at least two neighboring pixels to the left and at least two neighboring pixels to the right of said missing pixel, said neighboring pixels being located along a same horizontal line;

providing a vertical edge sensitivity parameter;

calculating vertical weighting factors by using said vertical edge sensitivity parameter and at least two neighboring pixels above and at least two neighboring pixels below said missing pixel, said neighboring pixels being located along a same vertical line;

and wherein said calculating an interpolated value for said missing pixel comprises;

computing a first intermediate pixel value by using said weighting factors and intensity values of at least one neighboring pixel to the left and at least one neighboring pixel to right of said missing pixel, said neighboring pixels being located along a same horizontal line;

computing a second intermediate pixel value by using said weighting factors and intensity values of at least one neighboring pixel above and at least one neighboring pixel below said missing pixel, said neighboring pixels being located along a same vertical line;

computing said missing pixel by using said first intermediate pixel value and said second intermediate pixel value.

16. A method as claimed in claim 15, wherein said computing said missing pixel comprises taking the mean of said first intermediate pixel value and said second intermediate pixel value.