Methods, apparatus, and systems for insertion of overlay content into a video signal with transrating capabilities

• US 7,688,889 B2
• Filed: 07/25/2007
• Issued: 03/30/2010
• Est. Priority Date: 09/18/2006
• Status: Active Grant

• Current Assignee: Imagine Communications Limited (Gores Broadcast Holdings LLC)
• Original Assignee: RGB Networks, Inc. (Gores Broadcast Holdings LLC)
• Inventors: Krause, Edward A.
• Primary Examiner(s): Harold; Marsha D Banks
• Assistant Examiner(s): Anyikire; Chikaodili E
• Application Number:
  US11/881,366
• Publication Number:
  US 20080068500A1
• Time in Patent Office: 979 Days
• Field of Search: 37524025-24027, 348584-589
• US Class Current: 375/240.01
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/235 Processing of additional data, e.g. scrambling of additional data or processing content descriptors
• CPC Class Codes: H04N 21/23655 Statistical multiplexing, e.g. by controlling the encoder to alter its bitrate to optimize the bandwidth utilization
• CPC Class Codes: H04N 21/42204 User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor constructive details of casings for the remote control device H01H9/0235; user interfaces for controlling a tuning device of a television receiver through a remote control H03J9/00; remote control of peripheral devices connected to a television receiver through the remote control device of the television receiver H04B1/205
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/434 Disassembling of a multiplex stream, e.g. demultiplexing audio and video streams, extraction of additional data from a video stream; Remultiplexing of multiplex streams; Extraction or processing of SI; Disassembling of packetised elementary stream demultiplexing of data packets for data networks, e.g. RTP/UDP H04L65/00
• CPC Class Codes: H04N 21/435 Processing of additional data, e.g. decrypting of additional data, reconstructing software from modules extracted from the transport stream
• CPC Class Codes: H04N 21/47 End-user applications
• CPC Class Codes: H04N 21/8153 comprising still images, e.g. texture, background image
• CPC Class Codes: H04N 5/45 Picture in picture , e.g. displaying simultaneously another television channel in a region of the screen

Abstract

Methods, apparatus, and systems for inserting overlay content into a video signal are provided. An overlay content signal containing overlay content is received. In addition, a compressed video signal is received from a video source and decoded. An insertion window portion of the decoded video signal is identified. The overlay content signal and the insertion window portion of the decoded video signal are processed to produce a modified insertion window portion of the video signal. The video signal and the modified insertion window portion are then processed to produce a modified video signal. The data rate of the video signal may be modified during the overlay content insertion process.

Claims

1. A method for inserting overlay content into a video signal, comprising:

receiving an overlay content signal containing overlay content;

receiving a compressed video signal;

decoding said compressed video signal;

identifying an insertion window portion of said decoded video signal;

processing said overlay content signal and said insertion window portion of said decoded video signal to produce a modified insertion window portion of said video signal; and

processing said video signal and said modified insertion window portion to produce a compressed modified video signal;

wherein;

an insert processing mode is provided for said processing of said overlay content signal and said insertion window portion of said video signal to produce said modified insertion window portion;

an obscured processing mode is provided for processing an obscured region of said video signal in which motion vectors of a current pixel block of the video signal reference a portion of said insertion window; and

a limited processing mode is provided for said processing when said obscured processing mode and said insert processing mode are not selected.

2. A method in accordance with claim 1, further comprising:

providing a closed-loop transrating path for modifying a data rate of the video signal;

providing an open-loop transrating path for modifying the data rate of the video signal; and

selecting said closed-loop transrating path for modifying the data rate of said insertion window portion of said video signal.

3. A method in accordance with claim 2, further comprising:

providing pixel quantization error control in said closed loop transrating path.

4. A method in accordance with claim 3, wherein said pixel quantization error control comprises:

obtaining errors from the quantization of transformation coefficients of incoming pixels of the video signal in the form of error transform coefficients;

performing an inverse transformation process on said error transform coefficients to provide pixel errors in the pixel domain;

storing said pixel errors;

retrieving said stored pixel errors during modification of said data rate of associated pixels;

adding said pixel errors to said incoming pixels obtained from an inverse transformation process to obtain corrected pixels in order to prevent accumulation of requantization artifacts during said quantization of transform coefficients corresponding to said incoming pixels; and

performing a transformation process on said corrected pixels to provide error corrected transform coefficients for said quantization.

5. A method in accordance with claim 2, wherein said closed-loop transrating path is selected for modifying the data rate of I-frames and P-frames of said video signal.

6. A method in accordance with claim 2, wherein said open-loop transrating path is selected for modifying the data rate of B-frames of said video signal.

7. A method in accordance with claim 2, wherein said closed-loop transrating path and said open loop transrating path are operable simultaneously for processing frames of multiple video signals.

8. A method in accordance with claim 7, wherein:

said closed-loop transrating path is selected for modifying the data rate of I-frames and P-frames of said video signals; and

said open-loop transrating path is selected for modifying the data rate of B-frames of said video signals.

9. A method in accordance with claim 7, wherein:

both of said open-loop transrating path and said closed loop transrating path are available for processing I-frames and P-frames of said video signals.

10. A method in accordance with claim 7, wherein:

both of said open-loop transrating path and said closed loop transrating path are available for processing B-frames of said video signals.

11. A method in accordance with claim 2, further comprising:

an additional open-loop transrating path for modifying the data rate of the video signal.

12. A method in accordance with claim 11, wherein said two open-loop transrating paths and said closed-loop transrating path are operable simultaneously.

13. A method in accordance with claim 2, wherein said open-loop transrating path comprises:

variable length decoding of said video signal to provide quantized transform coefficients;

performing an inverse quantization operation on said transform coefficients to scale said transform coefficients;

performing a quantization operation on said scaled transform coefficients to re-scale said transform coefficients to achieve a desired data rate; and

variable length encoding of said re-scaled transform coefficients to provide a video signal at said desired data rate.

15. A method in accordance with claim 1, further comprising:

providing a normal processing mode for processing said video signal when said video signal does not contain an insertion window.

16. A method in accordance with claim 15, wherein said normal processing mode comprises:

variable length decoding of said video signal to provide quantized transform coefficients;

performing an inverse quantization operation on said transform coefficients to scale said transform coefficients;

performing an inverse transformation process on said scaled transform coefficients to convert said scaled transform coefficients into pixel values;

summing the pixel values with motion-compensated pixel errors to provide error-compensated pixel values;

performing a transformation process on said error-compensated pixel values to produce error compensated transform coefficients;

performing a quantization operation on said error-compensated transform coefficients to re-scale said transform coefficients to achieve a desired data rate;

variable length encoding of said re-scaled transform coefficients to provide a video signal at said desired data rate.

17. A method in accordance with claim 16 wherein said motion-compensated pixel errors are obtained by:

obtaining errors from the quantization operation in the form of pixel errors;

storing said pixel errors;

retrieving said stored pixel errors when performing a motion compensation operation on associated pixels;

processing said pixel errors and said associated pixels to provide said motion-compensated pixel errors;

wherein said summing of said pixel values with said motion-compensated pixel errors prior to said transformation process prevents accumulation of requantization artifacts during said quantization operation.

18. A method in accordance with claim 1, wherein:

said limited processing mode comprises;

variable length decoding of said video signal to provide quantized transform coefficients;

performing an inverse quantization operation on said transform coefficients to scale said transform coefficients;

performing a quantization operation on said scaled transform coefficients to re-scale said transform coefficients to achieve a desired data rate;

variable length encoding of said re-scaled transform coefficients to provide a video signal at said desired data rate;

performing an inverse transformation process on said scaled transform coefficients to convert said scaled transform coefficients into pixel values;

adding said pixel values to motion compensated pixels retrieved from pixel memory to obtain decoded pixel values; and

storing said decoded pixel values.

19. A method in accordance with claim 1, wherein said obscured processing mode comprises:

variable length decoding of said obscured region of said video signal to provide quantized transform coefficients;

performing an inverse quantization operation on said transform coefficients to scale said transform coefficients;

performing an inverse transformation process on said scaled transform coefficients to convert said scaled transform coefficients into pixel values;

adding said pixel values to motion compensated pixels retrieved from pixel memory to obtain decoded pixel values;

storing said decoded pixel values;

processing said decoded pixel values to modify said current pixel block such that said current pixel block does not require reference to said portion of said insertion window of the video signal for decoding.

20. A method in accordance with claim 19, wherein said processing of said decoded pixel values comprises:

performing a transformation process on said decoded pixel values to reproduce said scaled transform coefficients;

performing a quantization operation on said scaled transform coefficients to re-scale said transform coefficients to achieve a desired data rate; and

variable length encoding of said re-scaled transform coefficients to provide an intra coded pixel block in place of said current pixel block at said desired data rate.

21. A method in accordance with claim 19, wherein said processing of said stored decoded pixel values comprises:

performing a motion compensation operation on said stored decoded pixel values to reconstruct the obscured region of the video signal;

performing a transformation process on said motion compensated pixel values to reproduce said scaled transform coefficients;

performing a quantization operation on said scaled transform coefficients to re-scale said transform coefficients to achieve a desired data rate; and

variable length encoding of said re-scaled transform coefficients using new motion vectors to provide a modified pixel block in said video signal in place of said current pixel block at said desired data rate.

22. A method in accordance with claim 1, wherein said insert processing mode comprises:

variable length decoding of said insertion window portion of said video signal to provide quantized transform coefficients;

performing an inverse quantization operation on said transform coefficients to scale said transform coefficients;

performing an inverse transformation process on said scaled transform coefficients to convert said scaled transform coefficients into pixel values corresponding to said insertion window portion;

adding said pixel values to motion compensated pixels retrieved from pixel memory to obtain decoded pixel values;

storing said decoded pixel values corresponding to said insertion window portion;

combining pre-stored pixel values corresponding to said overlay content with at least a portion of said stored decoded pixel values corresponding to said insertion window portion to provide modified decoded pixel values;

performing a transformation process on said modified decoded pixel values to provide modified scaled transform coefficients;

performing a quantization operation on said modified scaled transform coefficients to re-scale said modified transform coefficients to achieve a desired data rate; and

variable length encoding of said re-scaled modified transform coefficients to provide said modified video signal at said desired data rate.

23. A method in accordance with claim 22, wherein said pre-stored pixel values corresponding to said overlay content are obtained by:

variable length decoding of said overlay content signal to provide quantized transform coefficients corresponding to said overlay content;

performing an inverse quantization operation on said transform coefficients corresponding to said overlay content to scale said transform coefficients;

performing an inverse transformation process on said scaled transform coefficients to convert said scaled transform coefficients into pixel values corresponding to said overlay content; and

storing said pixel values corresponding to said overlay content.

24. A method in accordance with claim 15, further comprising transitioning from said normal processing mode to the insert processing mode when inserting said overlay content into said video signal.

25. A method in accordance with claim 24, wherein said transitioning comprises:

waiting to receive an I-frame in said video signal;

switching from said normal processing mode to said limited processing mode;

waiting to receive a next reference frame in a current portion of said video signal;

setting a window-mode to on enabling an insertion window in said video signal; and

switching to said insert processing mode when processing a region of the video signal corresponding to the insertion window; and

switching to said obscured processing mode when processing regions of said video signal external to said insertion window portion if motion vectors corresponding to said external regions reference the insertion window portion in a previously decoded frame.

26. A method in accordance with claim 15, further comprising transitioning to said normal processing mode after completing an insertion of overlay content.

27. A method in accordance with claim 26, wherein said transitioning comprises:

waiting to receive a reference frame;

setting a prime-mode to on initiating a resetting of a memory used to process said reference frame;

setting a window-mode to off to prevent creation of an insertion window in a current portion of said video signal;

waiting for a next frame of said video signal;

determining if said next frame is a B-frame;

if said next frame is a B-frame;

setting said window-mode to on enabling said insertion window in said video signal; and

waiting for further frames after said next frame until a next reference frame is received;

if said next frame is a reference frame;

setting said window-mode to off;

setting said prime mode to off; and

switching to said normal processing mode.

28. A method in accordance with claim 17, further comprising:

providing a pixel combiner;

providing a first multiplexer for selecting at least one of said transform coefficients from said inverse quantization operation or a signal from said pixel combiner; and

providing a second multiplexer for selecting at least one of said pixel errors from said quantization operation or said signal from said pixel combiner.

29. A method in accordance with claim 28, wherein said signal from said pixel combiner comprises at least one of stored overlay content pixels or main video content pixels obtained from processing of said video signal.

30. A method in accordance with claim 28, further comprising:

providing an overlay controller for controlling at least one of said first multiplexer, said second multiplexer, said pixel combiner, and said motion compensation operation.

31. A method in accordance with claim 30, further comprising:

synchronizing operation of said first multiplexer, said second multiplexer, said pixel combiner, and said motion compensation operation.

32. A method in accordance with claim 1, wherein:

said processing of said overlay content signal and said insertion window portion of said decoded video signal to produce said modified insertion window portion of said video signal comprises;

processing said insertion window portion of said video signal to obtain a corresponding first set of absolute pixel values;

storing said first set of absolute pixel values corresponding to said insertion window portion;

processing said overlay content signal to obtain a corresponding second set of absolute pixel values corresponding to said overlay content; and

storing said second set of absolute values corresponding to said overlay content;

said modified insertion window portion is produced by processing said first and second set of absolute pixel values.

33. A method in accordance with claim 1, wherein said overlay content is inserted into said video signal in place of said insertion window portion of said video signal to produce said modified insertion window portion.

34. A method in accordance with claim 1, wherein said overlay content is blended with said insertion window portion of said video signal to produce said modified insertion window portion.

35. A method in accordance with claim 1, further comprising:

providing different overlay content in one or more overlay content signals;

appending a tag to at least one of;

(a) the video signal; and

(b) at least one of said one or more overlay content signals, said tag containing identifying information; and

selecting overlay content for said inserting from one of said one or more overlay content signals in accordance with said identifying information.

14. A method for inserting overlay content into a video signal, comprising:

receiving an overlay content signal containing overlay content;

receiving a compressed video signal;

decoding said compressed video signal;

identifying an insertion window portion of said decoded video signal;

processing said overlay content signal and said insertion window portion of said decoded video signal to produce a modified insertion window portion of said video signal;

processing said video signal and said modified insertion window portion to produce a modified video signal;

providing at least one of an obscured processing mode and an insert processing mode for carrying out said processing steps;

said obscured processing mode is operable for processing an obscured region of said video signal in which motion vectors of a current pixel block of the video signal reference a portion of said insertion window;

said insert processing mode is operable for said processing of said overlay content signal and said insertion window portion of said video signal to produce said modified insertion window portion;

providing a normal processing mode for processing said video signal when the video signal does not contain an insertion window; and

in said obscured processing mode, said current pixel block is modified such that said current pixel block does not require reference to said portion of said insertion window of the video signal for decoding providing a limited processing mode for carrying out at least one of said processing steps when said normal processing mode, said obscured processing mode, and said insert processing mode are not selected.

36. Apparatus for inserting overlay content into a video signal, comprising:

means for receiving an overlay content signal containing overlay content;

means for receiving a compressed video signal;

means for decoding said compressed video signal;

means for identifying an insertion window portion of said decoded video signal;

means for processing said overlay content signal and said insertion window portion of said decoded video signal to produce a modified insertion window portion of said video signal; and

means for processing said video signal and said modified insertion window portion to produce a compressed modified video signal;

wherein;

an insert processing mode is provided for said processing of said overlay content signal and said insertion window portion of said video signal to produce said modified insertion window portion;

an obscured processing mode is provided for processing an obscured region of said video signal in which motion vectors of a current pixel block of the video signal reference a portion of said insertion window; and

a limited processing mode is provided for said processing when said obscured processing mode and said insert processing mode are not selected.