System and method for performing motion estimation in the DCT domain with improved efficiency

• US 5,796,434 A
• Filed: 06/07/1996
• Issued: 08/18/1998
• Est. Priority Date: 06/07/1996
• Status: Expired due to Term

• Current Assignee: Avago Technologies General IP PTE Limited (Broadcom, Inc.)
• Original Assignee: LSI Logic Corporation (Broadcom, Inc.)
• Inventors: Lempel, Mody
• Primary Examiner(s): Chin, Tommy P.
• Assistant Examiner(s): Le, Vu
• Application Number:
  US08/660,423
• Time in Patent Office: 802 Days
• Field of Search: 348/403, 348/400-402, 348/407-413, 348/415-416, 348/420, 348/699, 386/27, 386/33, 386/109, 386/111, 382/166, 382/235, 382/236, 382/239, 382/248, 382/250
• US Class Current: 348/403.1
• CPC Class Codes: G06T 7/262 using transform domain methods, e.g. Fourier domain methods
• CPC Class Codes: H04N 19/547 Motion estimation performed in a transform domain

Abstract

A system and method for estimating motion vectors between frames of a video sequence which operates in the DCT domain with improved efficiency and reduced computational requirements. The motion estimation system operates to encode a target block using pointers or motion vectors to a previously encoded block, referred to as the reference block or search block. The system first partitions the target frame into a plurality of target blocks, and DCT transforms the target blocks in the target frame. The motion estimation system then selects a candidate block from the search frame and DCT transforms the selected candidate block. The motion estimation system uses a novel method for selecting candidate blocks which allows re-use of at least a portion of the transformed values of a prior selected candidate block. After the candidate block is selected, the motion estimation system determines one or more "relevant" target blocks in the target frame which have a corresponding search window which includes the selected candidate block. The system then computes a distance between the transformed values of the selected candidate block and the transformed values of each of the determined target blocks. If a better metric is found for a respective target block, the system stores the location of the selected candidate block for the respective target block. The above steps are performed for each of the possible candidate blocks in the search frame. This method thus produces or determines "best fit" candidate blocks for each of the target blocks. The system then computes motion vectors for each of the target blocks in the target frame based on these "best fit" blocks.

Claims

1. A method for performing motion estimation between a target frame and a search frame, wherein the target frame includes a plurality of target blocks, the method comprising:

performing a frequency domain transform on the plurality of target blocks in the target frame;

selecting a candidate block from the search frame;

performing a frequency domain transform on the selected candidate block from the search frame;

determining one or more target blocks in the target frame in response to said selected candidate block;

computing a distance between the transformed values of the selected candidate block and the transformed values of each of said one or more determined target blocks, wherein said computing produces a distance for each of said one or more determined target blocks;

determining if said computed distance is a better metric than a currently stored computed distance for each of said one or more determined target blocks;

storing a location of said selected candidate block for a respective target block if said computed distance for said respective target block is a better metric than the current best computed distance for said respective target block;

wherein said selecting a candidate block from the search frame, performing a frequency domain transform on the selected candidate block from the search frame, determining one or more target blocks in the target frame, computing a distance, determining if said computed distance is a better metric, and storing a location of said selected candidate block, are performed for a plurality of candidate blocks in said search frame, wherein said performing for said plurality of candidate blocks in said search frame produces motion vectors for a plurality of said target blocks in said target frame.

2. The method of claim 1, wherein said determining one or more target blocks in the target frame comprises determining one or more target blocks in the target frame which have a corresponding search window which includes the selected candidate block.

3. The method of claim 1, wherein said selected candidate block is not located on a row or column boundary of any of said target blocks, wherein said determining one or more target blocks in the target frame comprises determining a first number of neighboring target blocks in the target frame, wherein each of said determined first number of neighboring target blocks has a corresponding search window which includes the selected candidate block.

4. The method of claim 3, wherein said selected candidate block is located on both a row and column boundary of one of said target blocks, wherein said determining one or more target blocks in the target frame comprises determining a second number of neighboring target blocks in the target frame, wherein each of said determined second number of neighboring target blocks has a corresponding search window which includes the selected candidate block;

wherein said second number is greater than said first number.

5. The method of claim 4, wherein said selected candidate block is located on one of either a row or column boundary of at least one of said target blocks, wherein said determining one or more target blocks in the target frame comprises determining a third number of neighboring target blocks in the target frame, wherein each of said determined third number of neighboring target blocks has a corresponding search window which includes the selected candidate block, wherein said search window is square, wherein said third number is greater than said first number and less than said second number.

6. The method of claim 1, wherein said computing a distance between the transformed values of the selected candidate block and the transformed values of each of said one or more determined target blocks comprises:

subtracting the transformed values of the selected candidate block from the transformed values of each of said one or more determined target blocks, wherein said subtracting produces a matrix for each of said one or more determined target blocks; and

determining the number of values in said matrices which will be quantized to zero for each of said one or more determined target blocks.

7. The method of claim 1, wherein said computing a distance between the transformed values of the selected candidate block and the transformed values of each of said one or more determined target blocks comprises:

subtracting the transformed values of the selected candidate block from the transformed values of each of said one or more determined target blocks, wherein said subtracting produces a matrix for each of said one or more determined target blocks;

quantizing said matrices for each of said one or more determined target blocks;

encoding each of said quantized matrices for each of said one or more determined target blocks into a plurality of bits; and

determining which of said encoded matrices has a least number of bits.

8. The method of claim 1, further comprising:

storing said computed distance for said respective target block if said computed distance is a better metric than the current best computed distance for said respective target block.

9. The method of claim 1, wherein said performing a frequency domain transform comprises performing a Discrete Cosine Transform (DCT).

10. The method of claim 1, wherein said selecting a candidate block from the search frame, performing a frequency domain transform on the selected candidate block from the search frame, determining one or more target blocks in the target frame, computing a distance, determining if said computed distance is a better metric, and storing a location of said selected candidate block, are performed for all candidate blocks in said search frame, wherein said performing for all candidate blocks in said search frame produces motion vectors for each of said target blocks in said target frame.

11. The method of claim 1, wherein said target frame and said search frame are comprised in an MPEG stream.

12. The method of claim 1, further comprising:

storing at least a portion of said transformed values of the selected candidate block after said performing a frequency domain transform on the selected candidate block from the search frame;

wherein said selecting a candidate block from the search frame includes selecting a candidate block which re-uses said at least a portion of said transformed values of a prior selected candidate block.

13. The method of claim 12, wherein said performing a frequency domain transform on the selected candidate block from the search frame comprises:

partitioning the selected candidate block into a plurality of sub-blocks; and

performing a frequency domain transform on each of said plurality of sub-blocks;

wherein said storing at least a portion of said transformed values of the selected candidate block comprises storing transformed values of one or more of said sub-blocks;

wherein said selecting a candidate block from the search frame includes selecting a candidate block which re-uses said stored transformed values of said one or more of said sub-blocks.

14. The method of claim 13, wherein said search frame comprises a plurality of horizontal bands, wherein each of said horizontal bands has a vertical length equal to a vertical length of said candidate blocks, wherein each of said horizontal bands includes a first portion and a second portion;

wherein said search frame further comprises a plurality of vertical strips, wherein each of said vertical strips has a horizontal width equal to a horizontal width of said candidate blocks;

wherein said selecting a candidate block from the search frame comprises;

for each vertical strip, selecting a horizontal band in the vertical strip which re-uses said second portion of a prior horizontal band.

15. The method of claim 14, wherein said search frame comprises a plurality of horizontal bands, wherein each of said horizontal bands has a vertical length equal to a length of said candidate blocks, wherein each of said horizontal bands includes an upper portion comprising one or more upper sub-blocks and a lower portion comprising one or more lower sub-blocks;

wherein said search frame further comprises a plurality of vertical strips, wherein each of said vertical strips has a horizontal width equal to a horizontal width of said candidate blocks;

wherein, for each vertical strip, said selecting a candidate block from the search frame comprises;

selecting a horizontal band in the vertical strip which re-uses a lower portion comprising a lower number of sub-blocks of a prior horizontal band, wherein said selecting is performed a plurality of times for a plurality of horizontal bands in the vertical strip;

wherein said selecting horizontal bands in the vertical strip which re-use lower portions of prior horizontal bands is performed a plurality of times for each of said vertical strips to select all of the candidate blocks in said search frame.

16. The method of claim 14, further comprising:

storing at least a portion of said transformed values of selected candidate blocks from a vertical strip;

wherein said selecting a candidate block from the search frame comprises selecting a new vertical strip which allows re-use of said at least a portion of said transformed values of said selected candidate blocks from a prior vertical strip.

17. The method of claim 12, wherein said candidate block comprises a plurality of columns and a plurality of sub-blocks, wherein said performing a frequency domain transform on the selected candidate block from the search frame includes:

performing a frequency domain transform on each of said plurality of columns of said selected candidate block;

wherein said storing at least a portion of said transformed values of the selected candidate block comprises storing transformed values of one or more of said columns;

wherein said selecting a candidate block from the search frame includes selecting a candidate block which re-uses said stored transformed values of said one or more of said columns.

18. The method of claim 17, wherein said search frame comprises a plurality of horizontal bands;

wherein said search frame further comprises a plurality of vertical strips, wherein each of said vertical strips comprises a plurality of columns;

wherein said selecting a candidate block from the search frame comprises;

selecting a vertical strip which re-uses computed columns of a prior selected vertical strip; and

for each vertical strip, selecting a horizontal band in the vertical strip.

19. The method of claim 18, wherein said selecting a horizontal band in the vertical strip comprises:

for each vertical strip, selecting a horizontal band in the vertical strip which re-uses computed portions of a prior horizontal band.

20. An MPEG encoder for performing motion estimation between a target frame and a search frame, wherein the target frame includes a plurality of target blocks, the MPEG encoder comprising:

a DCT transform block for performing a frequency domain transform on the plurality of target blocks in the target frame;

means for selecting a candidate block from the search frame, wherein the DCT block performs a frequency domain transform on the selected candidate block from the search frame;

means for determining one or more target blocks in the target frame in response to the selected candidate block;

means for computing a distance between the transformed values of the selected candidate block and the transformed values of each of said one or more determined target blocks, wherein said computing produces a distance for each of said one or more determined target blocks;

means for determining if said computed distance is a better metric than a currently stored computed distance for each of said one or more determined target blocks;

a memory for storing a location of said selected candidate block for a respective target block if said computed distance for said respective target block is a better metric than the current best computed distance for said respective target block;

wherein said means for selecting a candidate block from the search frame, said DCT block for performing a frequency domain transform on the selected candidate block from the search frame, said means for determining one or more target blocks in the target frame, said means for computing a distance, said means for determining if said computed distance is a better metric, and said memory storing a location of said selected candidate block, operate for a plurality of candidate blocks in said search frame, wherein said MPEG encoder produces motion vectors for a plurality of said target blocks in said target frame.

21. The MPEG encoder of claim 20, wherein said means for determining one or more target blocks in the target frame comprises determining one or more target blocks in the target frame which have a corresponding search window which includes the selected candidate block.

22. The MPEG encoder of claim 20, wherein said means for computing a distance between the transformed values of the selected candidate block and the transformed values of each of said one or more determined target blocks comprises:

a subtracter for subtracting the transformed values of the selected candidate block from the transformed values of each of said one or more determined target blocks, wherein said subtracter produces a matrix for each of said one or more determined target blocks; and

means for determining the number of values in said matrices which will be quantized to zero for each of said one or more determined target blocks.

23. The MPEG encoder of claim 20, wherein said means for computing a distance between the transformed values of the selected candidate block and the transformed values of each of said one or more determined target blocks comprises:

a subtracter for subtracting the transformed values of the selected candidate block from the transformed values of each of said one or more determined target blocks, wherein said subtracter produces a matrix for each of said one or more determined target blocks;

a quantizer for quantizing said matrices for each of said one or more determined target blocks;

an encoder for encoding each of said quantized matrices for each of said one or more determined target blocks into a plurality of bits; and

means for determining which of said encoded matrices has a least number of bits.

24. The MPEG encoder of claim 20, further comprising:

a memory for storing said computed distance for said respective target block if said computed distance is a better metric than the current best computed distance for said respective target block.

25. The MPEG encoder of claim 20, further comprising:

memory for storing at least a portion of said transformed values of the selected candidate block;

wherein said means for selecting a candidate block from the search frame selects a candidate block which re-uses said at least a portion of said transformed values of a prior selected candidate block.

26. A method for performing motion estimation between a target frame and a search frame, wherein the target frame includes a plurality of target blocks, the method comprising:

selecting a candidate block from the search frame;

determining one or more target blocks in the target frame in response to said selected candidate block;

computing a distance between the values of the selected candidate block and the values of each of said one or more determined target blocks, wherein said computing produces a distance for each of said one or more determined target blocks;

determining if said computed distance is a better metric than a currently stored computed distance for each of said one or more determined target blocks;

storing a location of said selected candidate block for a respective target block if said computed distance for said respective target block is a better metric than the current best computed distance for said respective target block;

wherein said selecting a candidate block from the search frame, determining one or more target blocks in the target frame, computing a distance, determining if said computed distance is a better metric, and storing a location of said selected candidate block, are performed for a plurality of candidate blocks in said search frame, wherein said performing for said plurality of candidate blocks in said search frame produces motion vectors for a plurality of said target blocks in said target frame.

27. The method of claim 26, wherein said determining one or more target blocks in the target frame comprises determining one or more target blocks in the target frame which have a corresponding search window which includes the selected candidate block.

28. The method of claim 26, wherein said selecting a candidate block from the search frame, determining one or more target blocks in the target frame, computing a distance, determining if said computed distance is a better metric, and storing a location of said selected candidate block, are performed for all candidate blocks in said search frame, wherein said performing for all candidate blocks in said search frame produces motion vectors for each of said target blocks in said target frame.

29. The method of claim 26, wherein said target frame and said search frame are comprised in an MPEG stream.