Method for compressing video information

• US 7,894,525 B2
• Filed: 01/23/2002
• Issued: 02/22/2011
• Est. Priority Date: 01/24/2001
• Status: Active Grant

• Current Assignee: Oy Gamecluster Ltd. (Broadmedia Corporation)
• Original Assignee: Oy Gamecluster Ltd. (Broadmedia Corporation)
• Inventors: Sallinen, Sami, Piehl, Erik
• Primary Examiner(s): Vo; Tung
• Assistant Examiner(s): HOLDER, ANNER N
• Application Number:
  US10/470,084
• Publication Number:
  US 20040095999A1
• Time in Patent Office: 3,317 Days
• Field of Search: 348/452, 375/240.16, 375135-137
• US Class Current: 375/240.16
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 19/51 Motion estimation or motion compensation
• CPC Class Codes: H04N 19/56 Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
• CPC Class Codes: H04N 19/597 specially adapted for multi-view video sequence encoding
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0085 Motion estimation from stereoscopic image signals

Abstract

A method (800, 820 ), device (900 ) and program product for compression video information is presented. The method comprises the following steps: projecting (803 ) points of a next image to a three-dimensional space using camera parameters and depth map and projecting (805 ) projected points from the three-dimensional space to reference image surface, thereby obtaining motion vectors for estimating changes between the next image and the reference image.

Claims

1. A device for motion compensated compression of a video sequence, comprising:

a first storage block which stores a reference image included in the video sequence,

a receiving block which receives a next image included in the video sequence, and

a determination block for determining a plurality of motion vectors, whereby each motion vector corresponds to a next point in a next image surface corresponding to the next image, for estimating changes between the next image and the reference image,

a compression block which compresses the video sequence using the motion vectors,

an establishment block for establishing information about a first depth map and first camera parameters corresponding to the next image, and

a second storage block for storing information about second camera parameters corresponding to the reference image, wherein

the determination block is further arranged to determine each motion vector corresponding to a next point in the next image surface by;

projecting the next point to a projected point in a three-dimensional space using the first depth map and first camera parameters,

projecting the projected point to a reference point included in a reference image surface corresponding to the reference image using the second camera parameters,

determining an estimated vector corresponding to the next point based on a location of the next point in the next image and a location of the reference point in the reference image, and

determining the motion vector corresponding to the next point using the estimated vector.

2. A device according to claim 1, further comprising

a first block storing which stores information about a second depth map corresponding to the reference image and

a second block which estimates the accuracy of the estimated vector by comparing a depth between the projected point and the reference point with a depth value obtained from the second depth map.

6. A computer, comprising the device according to claim 1.

7. A computer according to claim 6, wherein the computer further comprises a computer game.

8. A device according to claim 1, wherein

the first depth map is derived from a depth map residing in the memory of a 3D display graphics adapter controller of a computer;

and

the video sequence is generated by the 3D display graphics adapter controller.

3. A device for motion compensation compression of a video sequence comprising:

a first block which projects next points in a next image surface corresponding to a next image included in the video sequence to projected points in a three-dimensional space using camera parameters and a depth map corresponding to the next image,

a second block which projects the projected points to reference points included in a reference image surface corresponding to a reference image included in the video sequence, thereby obtaining estimated vectors corresponding to locations of the next points in the next image and locations of the reference points in the reference image, wherein the depth-map is derived from a depth map residing in the memory of a 3D graphics display adapter controller of a computer,

wherein the video sequence is generated by said 3D display graphics adapter, and

the device is configured to compress the video sequence using the estimated vectors.

4. A device for motion compensation compression of a video sequence, comprising:

means for storing a reference image included in the video sequence,

means for receiving a next image included in the video sequence,

means for determining a plurality of motion vectors, whereby each motion vector corresponds to a next point in a next image surface corresponding to the next image, for estimating changes between the next image and the reference image,

means for establishing information about a first depth map and first camera parameters corresponding to the next image,

means for storing information about second camera parameters corresponding to the reference image,

wherein the means for determining the plurality of motion vector is further configured to determine each motion vector corresponding to a next point in the next image surface by;

projecting the next point to a projected point in a three-dimensional space using the first depth map and first camera parameters,

projecting the projected point to a reference point included in a reference image surface corresponding to the reference image using the second camera parameters of the reference image,

determining an estimated vector corresponding to the next point based on a location of the next point in the next image and a location of the reference point in the reference image, and

determining the motion vector corresponding to the next point using the estimated vector.

5. A device for motion compensation compression of a video sequence, comprising:

means for projecting next points in a next image surface corresponding to a next image included in the video sequence to projected points in a three-dimensional space using camera parameters and a depth map corresponding to the next image;

means for projecting the projected points to reference points included in a reference image surface corresponding to a reference image included in the video sequence, thereby obtaining estimated vectors corresponding to locations of the next points in the next image and locations of the reference points in the reference image, wherein the depth-map is derived from a depth map residing in the memory of a 3D graphics display adapter controller of a computer; and

wherein the video sequence is generated by said 3D display graphics adapter; and

the device is configured to compress the video sequence using the estimated vectors.

9. A method for compressing a video sequence comprising:

receiving a next image included in the video sequence, first camera parameters corresponding to the next image, and a first depth map corresponding to the next image;

receiving a reference image included in the video sequence and second camera parameters corresponding to the reference image;

generating a plurality of motion vectors, whereby each motion vector corresponds to a next point in a next image surface corresponding to the next image, and is generated by steps comprising;

projecting the next point to a projected point in a three-dimensional space using the first camera parameters and the first depth map,

projecting the projected point to a reference point included in a reference image surface corresponding to the reference image using the second camera parameters,

determining an estimated vector corresponding to the next point based on a location of the next point in the next image and a location of the reference point in the reference image,

generating the motion vector corresponding to the next point using the estimated vector; and

compressing the video sequence using the plurality of motion vectors.

10. A method according to claim 9, wherein

the first depth map is derived from a depth map residing in the memory of a 3D display graphics adapter controller of a computer;

and

the video sequence is generated by the 3D display graphics adapter controller.

11. A method according to claim 9, further comprising:

receiving a second depth map corresponding to the reference image; and

wherein the steps for generating each motion vector further comprise;

estimating the accuracy of the estimated vector by comparing a depth between the projected point and the reference point with a depth value obtained from the second depth map.

12. A method according to claim 11, wherein the steps for generating each motion vector further comprise:

using the estimated vector as the motion vector or performing a motion vector search using the estimated vector as an initial value, depending on the estimated accuracy of the estimated vector.

13. A method according to claim 9, further comprising:

determining a motion compensated estimate for the next image by using the plurality of motion vectors.

14. A method according to claim 9, wherein each motion vector has a separate corresponding fixed-size image block.

15. A method according to claim 14, wherein the steps for generating each motion vector further comprise:

obtaining an average depth value from the first depth map for the image block corresponding to the motion vector; and

the next point corresponds to the center of the image block corresponding to the motion vector.

16. A tangible computer readable medium comprising a plurality of instructions which, when executed by a computer, cause the computer to perform the method of claim 9.