Method for the real-time composition of a video

• US 9,082,221 B2
• Filed: 06/29/2009
• Issued: 07/14/2015
• Est. Priority Date: 06/30/2008
• Status: Active Grant

• Current Assignee: Thomson Licensing (Vantiva SA)
• Original Assignee: Thomson Licensing (Vantiva SA)
• Inventors: Marvie, Jean-Eudes, Briand, Gerard
• Primary Examiner(s): Pe, Geepy
• Application Number:
  US12/737,253
• Publication Number:
  US 20110090307A1
• Time in Patent Office: 2,206 Days
• Field of Search: 348/42, 348/43, 348/46, 348/51, 348/60
• US Class Current: 1/1
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/106 Processing image signals for multi-view video sequence encoding H04N19/597
• CPC Class Codes: H04N 13/111 Transformation of image signals corresponding to virtual viewpoints, e.g. spatial image interpolation
• CPC Class Codes: H04N 13/156 Mixing image signals
• CPC Class Codes: H04N 13/172 image signals comprising non-image signal components, e.g. headers or format information
• CPC Class Codes: H04N 13/194 Transmission of image signals
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/271 wherein the generated image signals comprise depth maps or disparity maps

Abstract

A method for real time construction of a video sequence comprising a modelled 3D object is provided. The method comprises pre-calculating data representative of a first image of a three-dimensional environment and a first associated depth information. A live calculating of data representative of a second image representing the modelled object on which is mapped a current image of a live video stream, and a second depth information associated with the second image is performed. The sequence is composed by combining the first image and the second image according to the first and second depth information.

Claims

1. A method for live construction of a video sequence, said sequence comprising an object modelled in three dimensions, wherein the method comprises:

pre-calculating data representative of at least one first image of a three-dimensional environment and a first item of depth information associated with said image,

live calculation of;

data representative of at least one second image representing said modelled object on which is mapped a current image of a live video stream, said current image forming the texture of the modelled object, the mapping of the current image on said modeled object using only texture data of the current image, and

a second item of depth information associated with said at least one second image, and

live composition of said sequence by combining said at least one first image and at least one second image according to said first and second items of depth information.

2. The method according to claim 1, further comprising:

initializing a destination buffer of said at least one second image with the data representative of said at least one first image, and

initializing a destination buffer of said second item of depth information with said first item of depth information.

3. The method according to claim 1, wherein at least one occlusion test is carried out for each pixel of said sequence during said composition.

4. The method according to claim 1, wherein said environment is a video pre-calculated with a fixed camera.

5. The method according to claim 1, wherein said environment is a video pre-calculated with a mobile camera.

6. A device configured for live construction of a video sequence, said sequence comprising an object modelled in three dimensions, wherein the device comprises at least one processor configured for:

pre-calculating data representative of at least one first image of a three-dimensional environment and a first item of depth information associated with said image,

live calculation of;

data representative of at least one second image representing said modelled object on which is mapped a current image of a live video stream, said current image forming the texture of the modelled object, the mapping of the current image on said modelled object using only texture data of the current image, and

a second item of depth information associated with said at least one second image, and

live composition of said sequence by combining said at least one first image and at least one second image according to said first and second items of depth information.

7. The device according to claim 6, wherein the at least one processor is further configured for:

initializing a destination buffer of said at least one second image with the data representative of said at least one first image, and

initializing a destination buffer of said second item of depth information with said first item of depth information.

8. The device according to claim 6, wherein the at least one processor is further configured for carrying out at least one occlusion test for each pixel of said sequence during the live composition.

9. The device according to claim 6, wherein said environment is a video pre-calculated with a fixed camera.

10. The device according to claim 6, wherein said environment is a video pre-calculated with a mobile camera.