Methods and apparatus for reducing the amount of buffer memory required for decoding MPEG data and for performing scan conversion

• US 6,243,140 B1
• Filed: 02/17/1999
• Issued: 06/05/2001
• Est. Priority Date: 08/24/1998
• Status: Expired due to Fees

• Current Assignee: Hitachi America Limited (Hitachi, Ltd.)
• Original Assignee: Hitachi America Limited (Hitachi, Ltd.)
• Inventors: Suzuki, Norihiro
• Primary Examiner(s): Eisenzopf, Reinhard J.
• Assistant Examiner(s): Yenke, Brian P.
• Application Number:
  US09/251,451
• Time in Patent Office: 839 Days
• Field of Search: 348/448, 348/451, 348/452, 348/441, 348/446, 348/458, 348/443, 348/715, 348/402.1, 348/407.1, 348/409--41, 348/416.1, 348/424.1, 375/240-2-2
• US Class Current: 348/448
• CPC Class Codes: H04N 19/137 Motion inside a coding unit, e.g. average field, frame or block difference
• CPC Class Codes: H04N 19/172 the region being a picture, frame or field
• CPC Class Codes: H04N 19/426 using memory downsizing methods
• CPC Class Codes: H04N 19/44 Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 7/012 Conversion between an interlaced and a progressive signal for graphics images G09G2310/0229

Abstract

Methods and apparatus for reducing the total amount of memory required to implement a video decoder and to perform a scan conversion operation on decoded video are described. In accordance with the present invention this is accomplished by having an interlaced to progressive (I-P) conversion circuit utilize the same frame memory used to decode the images upon which a conversion operation is performed. In this manner, the images, e.g., frames, which are buffered in the decoder are utilized by both the decoder and I-P conversion circuit thereby eliminating the need for the I-P conversion circuit to be supported with an independent frame memory. Data included in a decoder'"'"'s frame memories is used to detect moving image areas for purposes of the I-P conversion process. In a specific exemplary embodiment, one of three frames, which is nearest to a present frame, is referred for calculating frame difference signals. Both subsequent and preceding frames are used to detect motion for I-P conversion purposes. This approach eliminates the need for a separate frame memory for motion detection purposes. Using the above discussed memory saving techniques, I-P conversion can be performed in accordance with the present invention by sharing the anchor frame memories and B-frame buffer present in a conventional decoder for both decoding and I-P conversion.

Claims

1. A method of generating data representing a sequence of progressive images, from data representing a sequence of first, second and third interlaced frames, the data representing each interlaced frame including data representing a first field and a second field, the method comprising the steps of:

performing a first motion compensated interpolation operation to generate data representing a first progressive image from data representing the first interlaced frame, the motion compensated interpolation operation including making an estimate of the image motion between a portion of a field of the first interlaced frame and a corresponding portion of a subsequent interlaced frame in the sequence of interlaced frames and accessing a memory device used to store reference frame data used by a decoder to support motion compensated prediction operations to obtain from the memory device data representing said corresponding portion of a subsequent interlaced frame; and

generating, by performing a second motion compensated interpolation operation, data representing a second progressive image from data representing the second interlaced frame, the motion compensated interpolation operation including the step of making an estimate of the image motion between a portion of a field of the second interlaced frame and a corresponding portion of a preceding frame in the sequence of interlaced frames.

2. The method of claim 1, further comprising the step of:

performing a third motion compensated interpolation operation, to generate data representing a third progressive image from data representing the third interlaced frame, the motion compensated interpolation operation including the step of making an estimate of the image motion between a portion of a field of the third interlaced frame and a corresponding portion of a preceding frame in the sequence of interlaced frames.

3. The method of claim 1, wherein the subsequent frame used to generate the first progressive image is the second interlaced frame.

4. The method of claim 3, wherein the preceding frame used to generate the second progressive image is the first interlaced frame.

5. The method of claim 4, wherein the preceding frame used to generate the third progressive image is the second interlaced frame.

7. The method of claim 1, wherein the first interlaced frame is a first decoded bi-directionally coded frame, the second interlaced frame is a decoded intra-coded frame and the third interlaced frame is a second decoded bi-directionally coded frame.

6. A method of generating data representing a sequence of progressive images, from decoded data representing a sequence of first, second and third interlaced frames, the data representing each interlaced frame including data representing a first field and a second field, the method comprising the steps of:

performing a first motion compensated interpolation operation to generate data representing a first progressive image from data representing the first interlaced frame, the motion compensated interpolation operation including the step of making an estimate of the image motion between a portion of a field of the first interlaced frame and a corresponding portion of a subsequent interlaced frame in the sequence of interlaced frames;

performing a second motion compensated interpolation operation to generate data representing a second progressive image from data representing the second interlaced frame, the motion compensated interpolation operation including the step of making an estimate of the image motion between a portion of a field of the second interlaced frame and a corresponding portion of a preceding frame in the sequence of interlaced frames; and

wherein the first interlaced frame is a first decoded bi-directionally coded frame, the second interlaced frame is a decoded intra-coded frame and the third interlaced frame is a second decoded bi-directionally coded frame.

8. A method of generating data representing a sequence of progressive images, from decoded data representing a sequence of first, second and third interlaced frames, the data representing each interlaced frame including data representing a first field and a second field, the method comprising the steps of:

performing a first motion compensated interpolation operation to generate data representing a first progressive image from data representing the first interlaced frame, the motion compensated interpolation operation including the step of making an estimate of the image motion between a portion of a field of the first interlaced frame and a corresponding portion of a subsequent interlaced frame in the sequence of interlaced frames;

performing a second motion compensated interpolation operation to generate data representing a second progressive image from data representing the second interlaced frame, the motion compensated interpolation operation including the step of making an estimate of the image motion between a portion of a field of the second interlaced frame and a corresponding portion of a preceding frame in the sequence of interlaced frames; and

wherein the step of performing a second motion compensated interpolation operation includes the step of accessing a buffer memory used to store B frames when decoded frames are being re-ordered for decoder output, to obtain from the buffer memory data representing said corresponding portion of a preceding interlaced frame.

9. A system for generating data representing a sequence of progressive images from data representing a sequence of encoded interlaced images, comprising:

a memory device for storing data representing decoded interlaced images;

a decoder, coupled to said memory device, for decoding data representing the sequence of encoded interlaced images including at least one P frame or B frame using decoded image data stored in the memory device as reference data for motion compensated prediction purposes when decoding at least some predictively coded data corresponding to said at least one P frame or B frame; and

an interlaced to progressive image converter, coupled to said memory device for converting decoded interlaced image data produced by the decoder into a series of progressive images, the interlaced to progressive image converter including motion detection means for detecting image motion using data corresponding to an interlaced image from which a progressive image is being generated and at least some of the decoded image data stored in the memory device for motion compensated prediction purposes when decoding at least some predictively coded data corresponding to said at least one P frame or B frame.

10. The system of claim 9, wherein the interlaced to progressive image converter further comprises:

a motion compensated interpolator for performing motion compensated interpolation operation as a function of the motion detected by the motion detecting means.

11. The system of claim 10, wherein the memory device comprises:

a first anchor frame memory; and

a second anchor frame memory.

12. The system of claim 11, wherein the memory device further comprises a B frame memory.

13. The system of claim 11, wherein the means for detecting motion uses data corresponding to an image which occurs subsequent to the image from which a progressive image is being generated when performing at least one motion detection operation.

14. The system of claim 13,

wherein the memory device further comprises a B frame memory for storing decoded interlaced images corresponding to data originally coded as a B frame.

15. The system of claim 14, wherein at least some of the B frame data is used for motion detection purposes after said at least some decoded B frame data is output by the decoder.

16. The system of claim 13,

wherein the memory device further comprises a B frame memory of a size which is sufficient for storing decoded interlaced image data corresponding to decoded B frames for use in motion detection after said decoded B frame data is output by the decoder and supplied to the interlaced to progressive image converter.

17. A method of generating an image suitable for display using a progressive scan, from an interlaced image comprising first and second fields included in a sequence of fields, the method comprising the steps of:

performing an image decoding operation on encoded image data to generate decoded image data representing the first and second fields and at least one additional field in the sequence of fields;

storing the decoded data representing the first and second fields and the additional field in a memory device; and

using the decoded data stored in the memory device corresponding to at least one of the first, the second and the additional field as reference data when performing a motion compensated prediction operation to decode a predicitively coded frame; and

performing an interlaced to progressive image conversion processes using said decoded data, stored in the memory device, which was used as reference data when performing the motion compensated prediction operation, to generate said image suitable for display using a progressive scan.

18. The method of claim 17, wherein the step of using the decoded data stored in the memory device to generate said image includes the steps of:

estimating the amount of image motion between a portion of one of the first and second fields and the additional field; and

performing an interpolation operation as a function of the estimated amount of motion.

19. The method of claim 18,

wherein pairs of fields in the sequence of fields represent I, P and B frames arranged as a group of pictures, the first and second fields representing a frame;

and

wherein the additional field corresponds to a frame which is located in said group of pictures subsequent to the frame represented by the first and second fields.

20. A method of converting encoded data representing a predictively coded interlaced image comprising a pair of fields, into data representing a progressive image comprising a single field, the method comprising the step of:

decoding said encoded data by performing at least one motion compensated prediction operation using previously decoded data stored in a memory device as reference data, to produce decoded data representing at least a field of said interlaced image;

comparing data representing a portion of a field of said interlaced image to a corresponding portion of a field of another interlaced image included in said stored reference data to produce a motion estimate; and

performing an interpolation operation to generate data representing a progressive image from said decoded data representing at least a field of said interlaced image and previously decoded data stored in said memory device as reference data, the interpolation being performed as a function of said motion estimate.

21. A method of converting data representing a series of interlaced images each comprising a pair of fields, into data representing a series of progressive images each comprising a single field, the method comprising the step of:

comparing data representing a portion of a field of a first interlaced image to a corresponding portion of a field of a second interlaced image to produce a first motion estimate, the second interlaced image occurring in said series of interlaced images subsequent to the first interlaced image;

using interpolation to generate data representing a progressive image from data representing one of the fields of the first interlaced image, the interpolation being performed as a function of the first motion estimate;

comparing data representing a portion of a field of a third interlaced image to data representing a corresponding portion of a field of a fourth interlaced image to produce a second motion estimate, the second interlaced image occurring in said series of interlaced images prior to the fourth interlaced image; and

using interpolation to generate data representing an additional progressive image from the data representing one of the fields of the third frame, the interpolation being performed as a function of the second motion estimate.

22. The method of claim 21,

wherein the data representing the progressive image includes interpolated data and the data representing the first field of the first frame; and

wherein the data representing the additional progressive image includes interpolated data and the data representing the first field of the third frame.

23. The method of claim 21,

wherein the data representing the progressive image includes interpolated data and the data representing the second field of the first frame;

and

wherein the data representing the additional progressive image includes interpolated data and the data representing the second field of the third frame.

24. The method of claim 21, further comprising the step of:

operating a decoder to decode encoded data representing the series of images, the step of operating a decoder including the step of storing in a memory device decoded reference frame data for use in performing motion compensated prediction operations during decoding and storing decoded B frame data to buffer said data; and

wherein the step of using interpolation to generate data representing a progressive image from one of the fields of the first frame, includes the step of using the decoded data that was stored in said memory for use as reference frame data.

25. The method of claim 24, wherein the step of using the decoded data that was stored in said memory includes the step of:

performing an image motion estimate.

26. The method of claim 21, wherein the interpolation used to generate data representing a progressive image is a motion compensated interpolation operation.

27. A method of converting data representing a series of interlaced images each comprising a pair of fields, into data representing a series of progressive images each comprising a single field, the method comprising the step of:

comparing data representing a portion of a field of a first interlaced image to a corresponding portion of a field of a second interlaced image to produce a first motion estimate, the second interlaced image occurring in said series of interlaced images subsequent to the first interlaced image;

using interpolation to generate data representing a progressive image from data representing one of the fields of the first interlaced image, the interpolation being performed as a function of the first motion estimate and;

wherein each pair of fields represents an I, P, or B frame.