Compressed video signal including independently coded regions

• US 6,507,618 B1
• Filed: 04/25/2000
• Issued: 01/14/2003
• Est. Priority Date: 04/25/2000
• Status: Expired due to Term

• Current Assignee: Dolby International AB (Dolby Laboratories Incorporated)
• Original Assignee: Hewlett-Packard Company (HP Inc.)
• Inventors: Wee, Susie J., Apostolopoulos, John G.
• Primary Examiner(s): RAO, ANAND SHASHIKANT
• Application Number:
  US09/558,079
• Time in Patent Office: 994 Days
• Field of Search: 375/240-8-2
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/107 between spatial and temporal predictive coding, e.g. picture refresh
• CPC Class Codes: H04N 19/124 Quantisation
• CPC Class Codes: H04N 19/132 Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• CPC Class Codes: H04N 19/159 Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
• CPC Class Codes: H04N 19/162 User input
• CPC Class Codes: H04N 19/17 the unit being an image region, e.g. an object
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/174 the region being a slice, e.g. a line of blocks or a group of blocks
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/46 Embedding additional information in the video signal during the compression process H04N19/517, H04N19/68, H04N19/70 take precedence
• CPC Class Codes: H04N 19/55 Motion estimation with spatial constraints, e.g. at image or region borders
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 21/23424 involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
• CPC Class Codes: H04N 21/44016 involving splicing one content stream with another content stream, e.g. for substituting a video clip

Abstract

In a MPEG or other video system, regions of each frame may be independently coded for later, independent extraction directly from a compressed bit stream. An encoder/transcoder receives raw video, standard compressed video or compressed video already having independently coded regions (“ICRs”) that are to be edited in some manner. A resultant signal may be used downstream for logo insertion or other video editing or processing without having to completely decode the entirety of each frame of a video sequence; rather, only those portions upon which editing or processing is to be performed are decoded. Once modified, substitute data may be inserted back into the compressed bit stream to take the place of decoded, edited data.

Claims

1. An improvement in a representation of a signal, the signal representing at least two data frames in compressed format, the frames including an anchor frame and a dependent frame that depends upon the anchor frame for its reconstruction, the compressed format for the dependent frame including motion vectors and residuals that enable recreation of data blocks in the dependent frame from data blocks in the anchor frame, said improvement comprising:

defining first and second regions in the anchor frame which have mutually exclusive subsets of anchor frame data;

defining first and second regions in the dependent frame which have mutually exclusive subsets of dependent frame data;

constraining the motion vectors during compression of the dependent frame such that

motion vectors for the first region of the dependent frame are limited to point into the first region of the anchor frame but not into the second region of the anchor frame,

motion vectors for the second region of the dependent frame are limited to point into the second region of the anchor frame but not into the first region of the anchor frame; and

indicators observable from the compressed format for at least the dependent frame which mark the location of portions of the compressed format that relate to the first region of the dependent frame, and that mark portions of the compressed format that relate to the second region of the dependent frame, such that either of the first and second regions can be separately extracted from the compressed format and decompressed without any need to decompress non-corresponding regions of the anchor frame.

2. An improvement according to claim 1, wherein each frame represents video image data.

3. An improvement according to claim 1, wherein:

the representation is embodied as a compressed bit stream;

the indicators are observable from examination of the compressed bit stream, and separately mark portions of the compressed bit stream that relate to the first and second regions of each of the dependent frame and the anchor frame;

the signal is adapted to permit

selective extraction and spatial domain reconstruction of the first region of each of the dependent and anchor frames without any need to extract or decompress either of the second region of the dependent frame and the second region of the anchor frame, and

selective extraction and spatial domain reconstruction of the first region of each of the dependent and anchor frames without any need to extract or decompress either of the second region of the dependent frame and the second region of the anchor frame.

4. An improvement according to claim 3, wherein the indicators include slice start codes.

5. An improvement according to claim 3, wherein the indicators include entries in a region map for each frame that are adapted to permit association of corresponding regions across multiple frames and to identify position of individual regions within each frame.

6. An apparatus comprising a signal stored on machine readable media, the signal characterized by:

at least two data frames in compressed format, the frames including an anchor frame and a dependent frame that depends upon the anchor frame for its reconstruction, the compressed format for the dependent frame including motion vectors and residuals that enable recreation of data blocks in the dependent frame from data blocks in the anchor frame;

wherein the anchor frame has associated with it first and second regions in the anchor frame which have mutually exclusive subsets of anchor frame data;

wherein the dependent frame has associated with it first and second regions in the dependent frame which have mutually exclusive subsets of dependent frame data;

wherein the motion vectors for the dependent frame are constrained such that

motion vectors for the first region of the dependent frame are limited to point into the first region of the anchor frame but not into the second region of the anchor frame,

motion vectors for the second region of the dependent frame are limited to point into the second region of the anchor frame but not into the first region of the anchor frame; and

indicators in the signal for at least the dependent frame which mark portions of the signal that contain compressed data for the first region of the dependent frame, and that mark portions of the signal that contain compressed data for the second region of the dependent frame, such that either of the first and second regions can be separately extracted from the signal and decompressed without any need to decompress non-corresponding regions of the anchor frame.

7. An apparatus according to claim 6, wherein the signal is a video signal representing plural image frames.

8. An apparatus according to claim 6, wherein:

the signal is a compressed bit stream;

the indicators are observable from examination of the compressed bit stream, and separately mark portions of the compressed bit stream that relate to the first and second regions of each of the dependent frame and the anchor frame;

the signal is adapted to permit

selective extraction and spatial domain reconstruction of the first region of each of the dependent and anchor frames without any need to extract or decompress either of the second region of the dependent frame and the second region of the anchor frame, and

selective extraction and spatial domain reconstruction of the second region of each of the dependent and anchor frames without any need to extract or decompress either of the first region of the dependent frame and the first region of the anchor frame.

9. An apparatus according to claim 8, wherein the indicators include slice start codes.

10. An apparatus according to claim 8, wherein the indicators include entries in a region map for each frame that are adapted to permit association of corresponding regions across multiple frames and to identify position of individual regions within each frame.

11. A product-by-process, said product comprising a compressed image sequence stored on digital storage media wherein the image sequence includes at least one image frame representing a single visual image, the single visual image being subdivided into two independently coded regions, such that each region may be extracted and decompressed from the compressed image sequence without extracting and decompressing the other region, said product formed by a process that includes selecting within a single image frame one of a geographic area and an image object that is to be independently coded, and compressing the one such that motion vectors for any dependent frame are constrained to point only to the same object or geographic area of any anchor frame, and by encoding the one as a subset of substantially non-overlapping image slices that may be independently recognized and extracted from the compressed image sequence.

12. A product-by-process according to claim 11, wherein the process includes encoding the one of a geographic area and an image object as a first independently coded region, and encoding all other data appearing in the same image frame as a second independently coded region, wherein dependent frame motion vectors for the one are constrained to point to reference data representing the same area or object in an anchor frame, and wherein dependent frame motion vectors for the second independently coded region are constrained to not point to reference data representing the same area or object in an anchor frame.

14. A product-by-process according to claim 11, wherein:

the compressed image sequence stored on digital storage media is a compressed bitstream having a plurality of substantially non-overlapping image slices from each frame, each image slice separately detectable from the compressed bitstream; and

the image slices are formed from motion-compensated blocks of data, where each motion-compensated block corresponds to an independently coded region, and a new image slice is initiated when a next motion-compensated block to be encoded represents a different independently coded region from an immediately prior motion-compensated block.

15. A product-by-process according to claim 14, wherein the compressed image sequence is MPEG-2 compliant.

13. An apparatus comprising a compressed image sequence stored on digital media, wherein the image sequence includes at least two independently coded regions, where both independently coded regions are represented together in at least one frame of the image sequence encoded as a single data stream representing the at least one frame, the independently coded regions each adapted for select extraction and decompression from the compressed image sequence in mutually independent fashion by extracting and decoding image slices corresponding to the selected independently coded region from the compressed image sequence.

16. An apparatus according to claim 13, wherein each image slice is formed from one or more motion-compensated blocks of data, where each motion-compensated block corresponds to an independently coded region, and a new image slice is begun each time a next motion-compensated block to be encoded represents a different independently coded region from an immediately prior motion-compensated block.

17. An apparatus comprising a signal stored on machine-readable media, wherein:

said signal includes a compressed bitstream representing at least one video frame;

the compressed bitstream is formatted as a series of substantially non-overlapping image slices for each video frame, with each image slice adapted for independent detection and extraction from the compressed bitstream; and

said signal includes at least two regions that are independently encoded, with any image slice representing compressed image data for one region only, such that each region may be independently decoded from the compressed bitstream via decoding of a subset of image slices for each frame.

18. An apparatus according to claim 17, wherein said signal includes at least one dependent frame having motion compensation information with respect to an anchor frame, the motion compensation information for a given image slice of a dependent frame limited to refer only to reference data associated with the same region in a corresponding anchor frame.

19. An improvement in a compressed representation of a video signal stored on machine-readable media, where motion vectors and residuals that enable recreation of data blocks in a dependent frame from data blocks in an anchor frame, said improvement comprising:

at least two regions where motion compensation for each region is limited so as to refer to only reference data associated with the same region in the same frame or an anchor frame; and

image slicing in the compressed representation, such that each image slice includes encoded video information for a single region only;

wherein each image slice is adapted for selective extraction from the compressed representation, such that each region can be independently decoded from the compressed representation.

20. An improvement according to claim 19, further comprising mapping observable from the compressed representation that identifies a subset of slices associated with each region, such that each region may be separately extracted from the compressed representation without necessarily extracting all data for other regions by decoding only those image slices identified by the mapping as belonging to a particular region.