Interactive encoded content system including object models for viewing on a remote device

• US 9,042,454 B2
• Filed: 01/11/2008
• Issued: 05/26/2015
• Est. Priority Date: 01/12/2007
• Status: Expired due to Fees

• Current Assignee: ActiveVideo Networks, Inc. (CommScope Holding Company, Inc.)
• Original Assignee: ActiveVideo Networks, Inc. (CommScope Holding Company, Inc.)
• Inventors: Gordon, Donald, Pavlovskaia, Lena Y., Landau, Airan, Lennartsson, Andreas, Lawrence, Charlie, Brown, Greg, Ludvig, Edward
• Primary Examiner(s): Huq, Obaidul
• Application Number:
  US12/008,697
• Publication Number:
  US 20080170622A1
• Time in Patent Office: 2,692 Days
• Field of Search: 375/240, 375/240.01, 375/240.12, 375/240.16, 375/240.18, 375/240.26, 375/240.29
• US Class Current: 375/240.16
• CPC Class Codes: G06F 2203/04804 Transparency, e.g. transparent or translucent windows
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: H04N 19/23 with coding of regions that are present throughout a whole video segment, e.g. sprites, background or mosaic
• CPC Class Codes: H04N 19/48 using compressed domain processing techniques other than decoding, e.g. modification of transform coefficients, variable length coding [VLC] data or run-length data motion estimation in a transform domain H04N19/547; processing of decoded motion vectors H04N19/513
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/2343 involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements video transcoding H04N19/40; media packet handling at the source H04L65/762
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams
• CPC Class Codes: H04N 21/2368 Multiplexing of audio and video streams
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/4347 Demultiplexing of several video streams
• CPC Class Codes: H04N 21/4348 Demultiplexing of additional data and video streams
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content
• CPC Class Codes: H04N 21/812 involving advertisement data advertising per se G06Q30/02
• CPC Class Codes: H04N 21/8543 using a description language, e.g. Multimedia and Hypermedia information coding Expert Group [MHEG], eXtensible Markup Language [XML] information retrieval of semistructured data, the underlying structure being taken into account, e.g. mark-up language structure data G06F16/80

Abstract

A system for creating composite encoded video from two or more encoded video sources in the encoded domain. In response to user input, a markup language-based graphical layout is retrieved. The graphical layout includes frame locations within a composite frame for at least a first encoded source and a second encoded source. The system either retrieves or receives the first and second encoded sources. The sources include block-based transform encoded data. The system also includes a stitcher module for stitching together the first encoded source and the second encoded source according to the frame locations of the graphical layout to form an encoded frame. The system outputs an encoded video stream that is transmitted to a client device associated with the user. In response to further user input, the system updates the state of an object model and replaces all or a portion of one or more frames of the encoded video stream. The system may be used with MPEG encoded video.

Claims

1. A system allowing for interactivity with video content that is transmitted to a client device as compressed video content, wherein the compressed video content includes a plurality of composited video frames, the system comprising:

a processor located remotely from the client device for controlling an interactive session with the client device and maintaining state information about a first compressed graphical element within the compressed video content, wherein the state information comprises a state of the first compressed graphical element, and wherein the state is configured to be associated with a user input operation performed with respect to the first compressed graphical element; and

a stitcher responsive to the processor for stitching together, in a frequency domain, at least the first compressed graphical element and a second compressed graphical element to form a new composited video frame;

wherein when the processor receives a signal requesting a change in state of the first compressed graphical element, the processor retrieves a new compressed graphical element indicative of the state change of the first compressed graphical element and the stitcher stitches the new compressed graphical element with the second compressed graphical element in the frequency domain to form a new composited video frame defining new compressed video content to be transmitted to the client device.

2. A system according to claim 1, further comprising:

a transmitter for transmitting the new compressed video content to the requesting client device.

3. A system according to claim 1, wherein the first graphical element is one or more MPEG encoded video frames.

4. A system according to claim 1 wherein each of the graphical elements are block-based transform encoded.

5. The system of claim 1, wherein the state of the first compressed graphical element indicates an on state or an off state of the first compressed graphical element.

6. The system of claim 1, wherein the state of the first compressed graphical element indicates whether a video program within the compressed video content is playing or stopped.

7. The system of claim 1, wherein when the processor receives a signal requesting a change in state of the first compressed graphical element, the processor further executes a script.

8. The system of claim 7, wherein the script includes instructions for:

determining if a video program within the compressed video content is playing or stopped;

if the video program is playing, executing an operation such that the video program stops playing; and

if the video program is stopped, executing an operation such that the video program resumes playing.

9. The system of claim 1, wherein when the processor receives a signal requesting a change in state of the first compressed graphical element, the processor further executes a call to an external program.

10. A method for creating a custom MPEG mosaic, the method comprising:

receiving a request from a requesting client device for a custom MPEG mosaic containing a number of video sources including at least a first video source and a second video source, wherein the first and second video sources are video streams of different broadcast channels;

receiving one or more groomed MPEG content streams including the first video source and the second video source;

stitching the groomed MPEG content streams including the first video source and the second video source with a background, without transform decoding the first video source and the second video source into a spatial domain, thereby creating a sequence of MPEG mosaic frames configured to simultaneously present the first and second video sources; and

transmitting the sequence of MPEG mosaic frames to the requesting client device as an MPEG elementary stream.

11. The method of claim 10, wherein the first video source includes user selected material and the second video source includes material selected by a processing office.

12. The method of claim 11, wherein the material selected by the processing office includes one or more advertisements.

13. The method of claim 10, further comprising:

receiving a request from a requesting client device to edit the custom MPEG mosaic;

receiving an additional groomed MPEG content stream including a third video source;

replacing the first video source with the third video source by stitching the additional groomed MPEG content stream including the third video source with the one or more groomed MPEG content streams including the second video source, without transform decoding the second video source and the third video source into a spatial domain, thereby creating a new sequence of MPEG mosaic frames wherein the second and third video sources are presented simultaneously;

transmitting the new sequence of MPEG mosaic frames to the requesting client device as an MPEG elementary stream.

14. The method of claim 10, further comprising:

receiving a request from a requesting client device to edit the custom MPEG mosaic;

receiving an additional groomed MPEG content stream including a third video source;

stitching the additional groomed MPEG content stream including the third video source with the one or more groomed MPEG content streams including the first video source and the second video source, without transform decoding the first, second, and third video sources into a spatial domain, thereby creating a new sequence of MPEG mosaic frames wherein the first, second, and third video sources are presented simultaneously;

transmitting the new sequence of MPEG mosaic frames to the requesting client device as an MPEG elementary stream.

15. The method of claim 10, wherein each of the one or more groomed MPEG content streams includes a plurality of intracoded MPEG P-frames.