System and method for frame differencing video compression and decompression with frame rate scalability

• US 5,384,598 A
• Filed: 10/20/1992
• Issued: 01/24/1995
• Est. Priority Date: 10/20/1992
• Status: Expired due to Term

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Kantner, Robert F. Jr., Rutherfoord, Charles T., Pietras, Mark A., Rodriguez, Arturo A., Hancock, Steven M.
• Primary Examiner(s): Chin, Tommy P.
• Assistant Examiner(s): Le, Vu
• Application Number:
  US07/963,791
• Time in Patent Office: 826 Days
• Field of Search: 358/133, 358/135, 358/136, 358/102, 358/86
• US Class Current: 348/384.1
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109
• CPC Class Codes: H04N 19/587 involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence

Abstract

A system and method of compressing original video data expressed in a plurality of digitally coded frames which enable decompression and playback of resulting compressed video data at one of a plurality of frame rates while maintaining temporal fidelity of the frames displayed. Compression includes selecting a plurality of rate streams for the compressed video data, including a highest rate stream including all of the frames of the original video data and a lowest rate stream including a subset of regularly spaced frames of the original video data. Then the initial frame in the original video data is spatially compressed and the resulting compressed data placed in the compressed video data. The initial frame is also saved as a base frame for all rate streams for subsequent temporal compression of the original video data. As frames are retrieved from the original video data in sequence, temporal compression based on frame differencing techniques between the retrieved frame and the base is carried out, with difference frames being stored to the compressed video data. Each difference frame is placed in the resulting compressed video data for later decompression and reproduction.

Claims

1. A method of compressing original video data expressed in a plurality of digitally coded frames to enable decompression and playback of resulting compressed video data at one of a plurality of frame rates while maintaining temporal fidelity of the frames displayed, the method comprising the steps of:

selecting a plurality of rate streams for the resulting compressed video data, including a highest rate stream including all of the frames of the original video data and a lowest rate stream including a subset of regularly spaced frames of the original video data;

spatially compressing one of the frames from the original video data as a base frame for all of the rate streams and placing the resulting compressed base frame in the resulting compressed video data;

retrieving subsequent frames from the original video data in sequence;

applying temporal compression to each of the subsequent frames retrieved from the original video data based on comparison to the base frame for each of the rate streams to which each of the subsequent frames belongs to generate a resulting difference frame; and

responsive to generation of the resulting difference frame, placing the resulting difference frame in the resulting compressed video data.

2. A method of compressing the original video data as set forth in claim 1, wherein the step of applying temporal compression further comprises:

responsive to each occurrence of generation of a difference frame using a subsequent frame from the original video data belonging to the lowest rate stream, making the frame from the original video data a new base frame for all of the rate streams.

3. A method of compressing the original video data as set forth in claim 2, wherein the step of selecting the rate streams further comprises selecting intermediate rate streams, each of the intermediate rate streams including a subset of regularly spaced frames of the original video data and including all of the frames in the lowest rate stream.

4. A method of compressing the original video data as set forth in claim 1, wherein the step of applying temporal compression to the subsequent frames retrieved from the original video data further comprises generating difference data based on comparisons of each of the subsequent frames and the base frame in each of the rate streams to which the subsequent frame belongs and summing the difference data to generate the difference frame.

5. A method of compressing the original video data as set forth in claim 4, and further comprising the step of:

subsequent to comparison of the retrieved frame to the base frame for each rate stream, making the retrieved frame the base frame for all rate streams to which the retrieved frame belongs.

6. A method of compressing the original video data as set forth in claim 5, wherein the step of selecting the rate streams further comprises selecting intermediate rate streams, each of the intermediate rate streams including a subset of regularly spaced frames of the original video data and including all of the frames in the lowest rate stream.

7. A method of compressing the original video data as set forth in claim 5, and further comprising:

after retrieval of the frame from the original video data, determining if the retrieved frame belongs to the lowest rate stream;

if the retrieved frame does not belong with to the lowest rate stream, continuing with the step of applying temporal compression; and

if the retrieved frame belongs to the lowest rate stream, omitting the step of applying temporal compression and spatially compressing the retrieved frame as the base frame for all of the rate streams and placing the resulting compressed frame in the resulting compressed video data.

8. A method of compressing the original video data as set forth in claim 7, wherein the step of selecting the rate streams further comprises selecting intermediate rate streams, each of the intermediate rate stream including a subset of regularly spaced frames of the original video data and including all of the frames in the lowest rate stream.

9. A method of compressing the video data as set forth in claim 2, and further comprising:

decompressing video data from the resulting compressed video data on a playback platform; and

displaying the decompressed video data.

10. A method of compressing the video data as set forth in claim 9, wherein the step of decompressing video data further comprises the steps of:

determining the rate streams used in generating the resulting compressed video data;

selecting one of the rate streams for reproduction based upon fastest rate stream supported by the playback platform;

decompressing the initial frame from the resulting compressed video data and saving the decompressed initial frame as the base frame in a memory for the playback platform;

decompressing a next difference frame in the selected rate stream from the resulting compressed video data;

combining the decompressed difference frame with the base frame to generate a display frame;

determining if the decompressed difference frame belongs to the lowest rate stream;

if the decompressed difference frame belongs to the lowest rate stream, storing the display frame in the memory as the base frame; and

returning to the step of decompressing a next difference frame.

11. A method of compressing the original video data as set forth in claim 5, and further comprising:

decompressing the resulting compressed video data by,

decompressing the initial frame,

storing the decompressed initial frame in the memory as a display frame,

retrieving subsequent frames from the resulting compressed video data,

decompressing each of the subsequent frames in sequence, and

updating the display frame with each decompressed subsequent frame.

12. A method of compressing the original video data as set forth in claim 7, and further comprising:

decompressing the resulting compressed video data by,

decompressing the initial frame,

storing the decompressed initial frame in the memory as a display frame,

retrieving a next subsequent frame from the resulting compressed video data,

decompressing the next subsequent frame,

determining if the next subsequent frame was spatially compressed or if the next subsequent frame is the difference frame,

if the next subsequent frame is the difference frame, updating the display frame with the difference frame,

if the next subsequent frame was a spatially compressed frame, making the spatially compressed frame the display frame upon decompression, and

returning to the step of retrieving the next subsequent frame.

13. A data processing system for compressing original video data expressed in a plurality of digitally coded frames to enable decompression and playback of resulting compressed video data at one of a plurality of frame rates while maintaining temporal fidelity of the frames displayed, comprising:

a memory for storing a base frame;

means for selecting a plurality of rate streams for the resulting compressed video data, the rate streams including a highest rate stream having all of the frames in the original video data and a lowest rate stream having a subset of regularly spaced frames in the original video data;

means for placing one of the frames from the original video data in the memory means as a base frame for all rate streams;

means for spatially compressing the base frame and placing a resulting compressed frame in the resulting compressed video data;

means for retrieving subsequent frames from the original video data in sequence;

means for applying temporal compression to the subsequent frames retrieved from the original video data based on comparison to the base frame for each of the rate streams to which the subsequent frames belong to generate a plurality of resulting difference frames; and

means for placing the resulting difference frames in the resulting compressed video data.

14. A data processing system as set forth in claim 13, wherein the means for applying temporal compression further comprises:

means responsive to generation of a difference frame using a subsequent frame from the original video data belonging to the lowest rate stream for making the subsequent frame from the original video data the base frame for all of the rate streams.

15. A data processing system as set forth in claim 14, wherein the means for selecting the rate streams further comprises means for selecting intermediate rate streams, each of the intermediate rate streams including a subset of regularly spaced frames of the original video data and including all of the frames in the lowest rate stream.

16. A data processing system as set forth in claim 13, wherein the means for applying temporal compression to the subsequent frames retrieved from the original video data further comprises means for generating difference data based on comparisons of each of the subsequent frames and the base frame in each of the rate streams to which the subsequent frame belongs and means for combining the difference data to generate the difference frame.

17. A data processing system for compressing original video data as set forth in claim 16, and further comprising:

means responsive to comparison of the subsequent frame to the base frames for each rate stream to which the frame belongs for making the frame the base frame for all rate streams to which the subsequent frame belongs.

18. A data processing system for compressing original video data as set forth in claim 17, wherein the means for selecting the rate streams further comprises means for selecting intermediate rate streams, each of the intermediate rate streams including a subset of regularly spaced frames of the original video data and including all of the frames in the lowest rate stream.

19. A data processing system for compressing original video data as set forth in claim 17, and further comprising:

means responsive to retrieval of the subsequent frame from the original video data for determining if the subsequent frame belongs to the lowest rate stream; and

means responsive to the subsequent frame belonging to the lowest rate stream for spatially compressing the subsequent frame as the base frame for all of the rate streams and for placing the resulting compressed base frame in the resulting compressed video data.

20. A data processing system as set forth in claim 19, wherein the means for selecting the rate streams further comprises means for selecting intermediate rate streams, each of the intermediate rate stream including a subset of regularly spaced frames of the original video data and including all of the frames in the lowest rate stream.

21. A data processing system as set forth in claim 14, and further comprising:

a playback platform for decompressing video data; and

means for displaying the decompressed video data.

22. A data processing system as set forth in claim 21, wherein the playback platform further comprises:

a memory;

means for accessing the resulting compressed video data;

means for determining the rate streams used in generating the resulting compressed video data;

means for selecting one of the rate streams for reproduction based upon fastest rate stream supported by playback platform;

means for decompressing the initial frame from resulting compressed video data and saving the decompressed initial frame as the base frame in the memory;

means for decompressing a next difference frame in the selected rate stream from the resulting compressed video data;

means for combining the decompressed difference frame with the base frame to generate a display frame;

means for determining if the decompressed difference frame belongs to the lowest rate stream; and

means responsive to the decompressed difference frame belonging to the lowest rate stream for storing the display frame in the memory as the base frame.

23. A data processing system as set forth in claim 21, wherein the playback platform further comprises:

a memory;

means for decompressing the initial frame;

means for storing the decompressed initial frame in the memory as a display frame;

means for retrieving subsequent frames from the resulting compressed video data;

means for decompressing each of the subsequent frames in sequence; and

means for updating the display frame with each decompressed subsequent frame.

24. A data processing system as set forth in claim 23, wherein the memory includes a video buffer for storage of the display frame.

25. A data processing system as set forth in claim 21, and further comprising:

a memory;

means for decompressing the initial frame;

means for storing the decompressed initial frame in the memory as a display frame;

means for retrieving a next subsequent frame from the resulting compressed video data;

means for decompressing the next subsequent frame;

means for determining if the next subsequent frame was spatially compressed or if the next subsequent frame is the difference frame;

means responsive to the next subsequent frame being the difference frame for updating the display frame with the difference frame; and

means responsive to the next subsequent frame having been a spatially compressed frame for storing the next subsequent frame as the display frame upon decompression.