TEXTURE REPLACEMENT IN VIDEO SEQUENCES AND IMAGES

• US 20110292993A1
• Filed: 08/08/2011
• Published: 12/01/2011
• Est. Priority Date: 10/11/2001
• Status: Active Grant

• Current Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Original Assignee: AT&T Intellectual Property II LP (AT&T, Inc.)
• Inventors: Haskell, Barin Geoffry, Dumitras, Adriana
• Granted Patent: US 8,896,616 B2
• Time in Patent Office: Days
• US Class Current: 375/240.01
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 7/40 Analysis of texture depth or shape recovery from texture G06T7/529
• CPC Class Codes: G06T 7/41 based on statistical description of texture
• CPC Class Codes: G06T 9/00 Image coding bandwidth or redundancy reduction for static pictures H04N1/41; coding or decoding of static colour picture signals H04N1/64; methods or arrangements for coding, decoding, compressing or decompressing digital video signals H04N19/00
• CPC Class Codes: G06T 9/001 Model-based coding, e.g. wire frame
• CPC Class Codes: H04N 19/14 Coding unit complexity, e.g. amount of activity or edge presence estimation H04N19/146 takes precedence
• CPC Class Codes: H04N 19/167 Position within a video image, e.g. region of interest [ROI]
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/186 the unit being a colour or a chrominance component
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 19/27 involving both synthetic and natural picture components, e.g. synthetic natural hybrid coding [SNHC]

Abstract

Systems and methods for reducing bit rates by replacing original texture in a video sequence with synthesized texture. Reducing the bit rate of the video sequence begins by identifying and removing selected texture from frames in a video sequence. The removed texture is analyzed to generate texture parameters. New texture is synthesized using the texture parameters in combination with a set of constraints. Then, the newly synthesized texture is mapped back into the frames of the video sequence from which the original texture was removed. The resulting frames are then encoded. The bit rate of the video sequence with the synthesized texture is less than the bit rate of the video sequence with the original texture. Also, the ability of a decoder to decode the new video sequence is not compromised because no assumptions are made about the texture synthesis capabilities of the decoder.

Claims

1. A method comprising:

removing a region of interest in an initial frame within a set of frames in a video sequence;

determining, via a processor, texture parameters associated with the region of interest;

synthesizing new texture using the texture parameters according to one of a first set of constraints and a second set of constraints, wherein the first set of constraints is applied if the region of interest is structured and busy, and the second set of constraints is applied if the region is unstructured and smooth; and

inserting the new texture into the set of frames.

2. The method of claim 1, further comprising:

encoding the set of frames containing the new texture, to yield an encoded video sequence.

3. The method of claim 2, wherein a modified bit rate associated with the encoded video sequence is lower than an original texture bit rate associated with the video sequence.

4. The method of claim 1, wherein the new texture is similar to, but indistinguishable from, the region of interest.

5. The method of claim 1, further comprising:

selecting the region of interest based on the a percentage of area within an initial frame.

6. The method of claim 1, wherein the new texture has a color parameter equal to an average color of the region of interest.

7. The method of claim 1, wherein the texture parameters are determined using a parametric statistical model.

8. A system comprising:

a processor; and

a memory storing instructions for controlling the processor to perform steps comprising;

removing a region of interest in an initial frame within a set of frames in a video sequence;

determining texture parameters associated with the region of interest;

synthesizing new texture using the texture parameters according to one of a first set of constraints and a second set of constraints, wherein the first set of constraints is applied if the region of interest is structured and busy, and the second set of constraints is applied if the region is unstructured and smooth; and

inserting the new texture into the set of frames.

9. The system of claim 8, the memory storing instructions for controlling the processor to perform steps further comprising:

encoding the set of frames containing the new texture, to yield an encoded video sequence.

10. The system of claim 9, wherein a modified bit rate associated with the encoded video sequence is lower than an original texture bit rate associated with the video sequence.

11. The system of claim 8, wherein the new texture is similar to, but indistinguishable from, the region of interest.

12. The system of claim 8, the memory storing instructions for controlling the processor to perform steps further comprising:

selecting the region of interest based on the a percentage of area within an initial frame.

13. The system of claim 8, wherein the new texture has a color parameter equal to an average color of the region of interest.

14. The system of claim 8, wherein the texture parameters are determined using a parametric statistical model.

15. A non-transitory computer-readable storage medium storing instructions which, when executed by a computing device, cause the computing device to process texture, the instructions comprising:

removing a region of interest in an initial frame within a set of frames in a video sequence;

determining texture parameters associated with the region of interest;

synthesizing new texture using the texture parameters according to one of a first set of constraints and a second set of constraints, wherein the first set of constraints is applied if the region of interest is structured and busy, and the second set of constraints is applied if the region is unstructured and smooth; and

inserting the new texture into the set of frames.

16. The non-transitory computer-readable storage medium of claim 15, the instructions further comprising:

encoding the set of frames containing the new texture, to yield an encoded video sequence.

17. The non-transitory computer-readable storage medium of claim 16, wherein a modified bit rate associated with the encoded video sequence is lower than an original texture bit rate associated with the video sequence.

18. The non-transitory computer-readable storage medium of claim 15, wherein the new texture is similar to, but indistinguishable from, the region of interest.

19. The non-transitory computer-readable storage medium of claim 15, the instructions further comprising:

selecting the region of interest based on the a percentage of area within an initial frame.

20. The non-transitory computer-readable storage medium of claim 15, wherein the new texture has a color parameter equal to an average color of the region of interest.