Image compression featuring selective re-use of prior compression data

• US 6,697,061 B1
• Filed: 01/21/1999
• Issued: 02/24/2004
• Est. Priority Date: 01/21/1999
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Schuyler, Marc P., Wee, Susie J.
• Primary Examiner(s): Zimmerman, Mark
• Assistant Examiner(s): Nguyen, Kimbinh T.
• Application Number:
  US09/235,101
• Time in Patent Office: 1,860 Days
• Field of Search: 345/418, 345/419, 348/416, 348/403, 348/423, 348/722, 375/240, 341/51
• US Class Current: 345/419
• CPC Class Codes: H04N 19/40 using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

This disclosure provides a compression system for edited images which have previously been compressed. The preferred system uses software to track and determine the nature of edits made to each image frame. Each image frame is divided into spatial regions, and codes defining the nature of changes are then stored in a set of tables called the “registry of edits.” When it is time to compress images for output, re-compression software interrogates the registry to determine whether spatial regions in the frame has been altered in a manner that undermines the integrity of the original compression data. For example, if a compressed image signal is modified in the spatial domain to add the logo of a local television station, most of each image frame will remain unchanged and the original motion vectors and residuals (or other compressed representation) from an input signal may be re-used, thereby saving substantial processing time and minimizing introduction of additional quantization errors. The preferred embodiment may be used with most any digital editor or computer to substantially reduce the processing time and resources required to provide a compressed output signal.

Claims

1. A method of editing an input image frame using an editor, an input buffer, a de-compression engine and a compression engine, said method comprising:

dividing the input image frame into plural spatial regions;

storing data from at least one of the spatial regions as compressed input data in the input buffer;

using the de-compression engine to convert data of the input image frame to the spatial domain;

editing with the editor at least one of

the input image frame,

an anchor frame upon which the input image frame depends, and

the order of the input image frame relative to an anchor frame upon which the input image depends

to thereby prepare spatial domain output data for compression; and

using the compression engine to convert the spatial domain output data to compressed output data;

wherein,

using the editing device includes creating a table having an entry for each region, where each entry represents whether predetermined

editing has been performed with respect to the associated region which affects whether the compressed input data (for the associated region) in the input buffer may be re-used as part of the compressed output data, and

using the compression engine to convert includes re-using at least some of the compressed input data from the input buffer to produce compressed output data, in dependence upon the entries in the table.

2. A method according to claim 1, wherein using the compression engine to convert includes obtaining an original motion vector from the input buffer and exactly re-using that original motion vector as part of the compressed output data, for each of multiple image tiles within a region that has not had predetermined editing performed which affects that region.

3. A method according to claim 1, wherein using the compression engine to convert includes calculating new residuals in a spatial region whose input compression representation is affected by editing, regardless of whether editing has actually resulted in changed pixel values within the region.

4. A method according to claim 1, wherein using the compression engine to convert includes for each region, determining whether editing has occurred which affects that region and, responsive to a determination that no editing has occurred, retrieving each of plural original motion vectors from the input buffer falling within the region each for exact use as a new motion vector in the compressed output data.

5. A method according to claim 4, wherein:

said method further includes subdividing each of plural input frames into plural spatial regions, each spatial region occupying identical size and position with respect to all frames;

editing includes creating a table for each of the plural input image frames, to thereby create a registry of edits, and placing a predetermined entry for each region into each table if editing has not been performed which affects the associated region; and

determining includes checking, for each region, for presence of the predetermined entry in the corresponding table.

6. A method according to claim 4, wherein:

storing data in the input buffer includes storing at least one original motion vector and corresponding original residuals; and

determining includes, responsive to a determination that no editing has occurred which affects a region, retrieving an original motion vector and corresponding original set of residuals for the region from the input buffer for use in the compressed output data.

7. A method according to claim 6, wherein:

each original motion vector stored in the input buffer is accompanied by a corresponding original set of residuals;

editing includes, for each of multiple regions, placing a first predetermined entry in a table if editing has not affected that region at all and a second predetermined entry if editing has affected that region in a manner consistent with all original motion vectors for the region, such that the original motion vectors are adapted for use to predict motion reflected by the compressed output data;

determining includes checking, for each region, for presence of a predetermined entry in the table; and

for each region

if the table contains the first predetermined entry then using each original motion vector and corresponding original set of residuals for the region from the input buffer in the compressed output data,

if the table contains the second predetermined entry then using each original motion vector to limit any motion search, and

if the table contains none of the first predetermined entry and the second predetermined entry then calculating a new motion vector by motion search not limited in dependence upon an original motion vector and calculating new residuals.

8. A method according to claim 1, wherein each entry is set to a predetermined value in response to a change in frame order which affects dependence of the input image frame relative to an anchor frame.

9. For use with an image editing system that receives a compressed input image frame and outputs a compressed output image frame, the image editing system having a de-compression engine that converts input data from a compressed format to spatial domain data, an editor that edits at least one of (a) the input image frame, (b) an anchor frame upon which the input image frame depends, and (c) order of the input image frame relative to an anchor frame upon which the input image frame depends, and a compression engine that converts the spatial domain data following editing to compressed output data, an improvement comprising:

dividing the input image frame into plural spatial regions;

during or prior to de-compression, storing at least some data for the input image frame in compressed format in an input buffer;

in response to predetermined editing, creating a table having plural entries that represent the plural spatial regions; and

for each region, compressing any spatial domain data within the region to form the compressed output data by determining from the table whether no editing has been performed within the region and,

if no editing has been performed within the region, retrieving from the input buffer at least one original vector and deriving in dependence thereon a new motion vector.

10. An improvement according to claim 9, further comprising:

for each region, for each of plural image tiles falling within the region, retrieving from the input buffer and exactly re-using an original motion vector and associated residuals.

11. An improvement according to claim 10, wherein the improvement is implemented via one of software and firmware.

12. An improvement according to claim 9, further comprising a table that stores multi-bit values representing whether associated spatial regions of an image frame have been edited.

13. An improvement according to claim 9, further comprising:

responsive to an editing process in which one of a first group of edits are performed which affect motion-vector-and-residual format data for a region, placing a first predetermined entry into the table to indicate that an original motion vector and associated original residuals from the input buffer may not be re-used in forming the image output;

responsive to an editing process in which one of a second group of edits are performed which affect motion-vector-and-residual format data within the region, but none of the first group of edits, placing a second predetermined entry into the table to indicate that original motion vectors may be re-used in deriving new motion vectors; and

in compressing data, re-using original motion vectors from the compressed input image in dependence upon the table contents.

14. An improvement according to claim 13, further comprising:

a table that stores multi-bit values representing whether associated spatial regions of an image frame have been edited; and

responsive to an editing process in which one of no editing is performed which affect motion-vector-and-residual format data within a region, placing a third predetermined entry into the table to indicate that original motion vectors and corresponding original residuals may be re-used in producing the compressed output data.

15. An improvement according to claim 9, wherein each of multiple locations within the table corresponds to a rectilinear, fixed-size spatial region of an image frame.

16. An improvement according to claim 9, further comprising creating a table for each one of plural input frames, to thereby create a registry of edits.

17. An apparatus for compressing spatial domain image frame data into motion vector and residual format output data, comprising:

an input buffer having original compression data stored therein associated with the image frame data;

a table corresponding to the image frame, the table having a number of entries, each entry corresponding to a different spatial region of the image frame; and

a compression engine that converts the spatial domain data to the motion vector and residual format, including one of hardware, software and firmware that

associates data to be converted to the motion vector and residual format output data with a spatial region,

interrogates a corresponding entry in the table and, responsive to the entry, performs at least two of

retrieving original compression data corresponding to the associated spatial region from the input buffer, and directly using the retrieved compression data as part of the output data,

retrieving an original compression data from the input buffer, and using motion vectors from the original compression data to reduce search, but calculating at least new residuals, and

calculating a new motion vector and associated residuals based on motion search which compares the spatial domain data to other image data, without using original compression data vector from the input buffer.