Using encoding cost data for segmentation of compressed image sequences

• US 6,058,210 A
• Filed: 09/15/1997
• Issued: 05/02/2000
• Est. Priority Date: 09/15/1997
• Status: Expired due to Term

• Current Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: de Queiroz, Ricardo L., Bozdagi, Gozde
• Primary Examiner(s): Bella, Matthew C.
• Assistant Examiner(s): Patel, Kanji
• Application Number:
  US08/931,098
• Time in Patent Office: 960 Days
• Field of Search: 382/232, 382/233, 382/235, 382/236, 382/238, 382/244, 382/245, 382/246, 382/248, 382/250, 382/251, 382/253, 382/173, 382/166, 382/171, 348/231, 345/501
• US Class Current: 382/232
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20052 Discrete cosine transform [DCT]
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 9/007 Transform coding, e.g. discrete cosine transform

Abstract

A method for processing compressed digital data derived from an original image sequence, the data being organized as a set of image frames, each image frame comprising a set of blocks, each block including a string of bits corresponding to an area of the original image frame in the original image sequence. A cost function is derived as a number related to the amount of bits spent to encode a block, sets of blocks, an image frame, or sets of image frames. A segmentation technique is applied to the map with cost functions. Temporal segmentation is performed by analyzing cost functions associated with each image frame. In both cases auxiliary functions can be used to improve the segmentation quality. The segmented regions of a image frame or sets of image frames can be identified, replaced, printed, or processed in special manners.

Claims

1. A method of processing compressed digital data derived from an original sequence of images, the digital data organized as a set of image frames, each image frame including a set of blocks, each block including a string of bits corresponding to an area of an image frame of the original sequence of images, comprising the steps of:

counting a number of bits for sets of at least one of said strings of bits in the compressed digital data;

deriving a cost function for each set of strings, said cost function being a number related to the number of bits in each set of strings of bits;

applying a temporal segmentation technique to the cost functions associated with strings of bits pertaining to at least two image frames represented in said compressed digital data;

identifying at least one image frame from said compressed digital data.

2. The method of claim 1, wherein the temporal segmentation is obtained as a function of the cost functions associated with the sum of bits for a whole image frame.

3. The method of claim 1, wherein the processing applied to the cost function comprises:

separating the cost function into a plurality of vectors, said vectors corresponding to each type of image frame, each type of image frame depending on a motion estimation strategy;

processing each vector with a plurality of filters;

applying local maxima detection to a combined filtered output for vector corresponding to an image frame type;

identifying any scene-cut positions as image frames corresponding to vectors'"'"' local maxima positions;

processing identified scene-cut positions for all image frame types, to thereby better represent scene-cut regions in said sequence.

4. The method of claim 1, wherein the identified image frames are replaced by precomputed image data.

5. The method of claim 1, wherein the identified image frames are extracted and printed.

6. The method of claim 1, wherein a spatial segmentation is applied to an identified image frame, said spatial segmentation identifying at least one region of the said image.

7. The method of claim 6, wherein the identified regions are replaced by precomputed image data.

8. The method of claim 6, wherein the identified regions are extracted and printed.

9. A method of processing compressed digital data derived from a sequence of images, the compressed data being organized as a set of image frames, each image frame comprising a set of blocks, each block comprising a string of bits corresponding to an area of a image frame of the original sequence, comprising the steps of:

counting a number of bits of a plurality of said strings of bits in the original sequence of images;

deriving a cost function for each set of strings of bits, said cost function being a value related to the number of bits in each set of strings of bits;

deriving an auxiliary function from the compressed data, said auxiliary function distinct from the cost function and related to the relationship of the set of image frames; and

applying a temporal segmentation technique to the cost functions in combination with the auxiliary functions associated with at least two images frames;

identifying at least one image frame.

10. The method of claim 9, wherein, the digital data is compressed using a discrete cosine transform method, each block therein having a DC component associated therewith, the auxiliary function is a map of the DC components of the blocks within a image frame.

11. The method of claim 9, wherein the digital data is compressed using a discrete cosine transform method, each block therein having a DC component associated therewith, the auxiliary function is related to a histogram of the DC component of the blocks within a image frame.

12. The method of claim 9, wherein the auxiliary function is a function related to any motion vectors associated with said image frames.

13. The method of claim 9, wherein the auxiliary function is a function related to a classification associated with said image frames, said classification provided as embedded data into the compressed data.

14. The method of claim 9, wherein the auxiliary function is a function related to a classification associated with the blocks within an image frame, said classification provided as embedded data into the compressed data.

15. The method of claim 9, wherein the segmentation technique processes the cost function associated with the cost function value for the whole image frame.

16. The method of claim 9, wherein the identified image frames are replaced by precomputed image data.

17. The method of claim 9, wherein the identified image frames are extracted and printed.

18. The method of claim 9, wherein a spatial segmentation is applied to an identified image frame, said spatial segmentation identifying at least one region of the said image.

19. The method of claim 18, wherein the identified regions are replaced by precomputed image data.

20. The method of claim 18, wherein the identified regions are extracted and printed.