Adaptive motion estimation

• US 8,254,459 B2
• Filed: 05/03/2010
• Issued: 08/28/2012
• Est. Priority Date: 07/14/2002
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Kumar, Roger, Pun, Thomas, Nie, Xiaochun, Wu, Hsi-Jung
• Primary Examiner(s): Vo, Tung
• Application Number:
  US12/772,971
• Publication Number:
  US 20110019879A1
• Time in Patent Office: 848 Days
• Field of Search: 37524001-24029
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/533 Motion estimation using multistep search, e.g. 2D-log search or one-at-a-time search [OTS]
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

Some embodiments of the invention provide a method for adaptively performing motion estimation. In some embodiments, the method initially performs a first motion estimation operation. It then determines whether the estimated error in the result of the first motion estimation is greater than a particular threshold. If not, the method uses the results of the first motion estimation operation. Otherwise, the method selects another motion estimation operation. The method continues in this manner until it performs a motion estimation operation that results in an acceptable estimated error, or until it tries all the motion estimation operations at its disposal. When the method cannot identify a motion estimation operation that produces results with an acceptable estimated error, the method selects the result of the motion estimation operation that produced the best results.

Claims

1. A non-transitory computer readable medium storing a computer program that when executed by at least one processor performs motion estimation for an image array obtained from a current image in a sequence of video images, the computer program comprising sets of instructions for:

performing a first motion estimation process that searches for the image array in a reference image in the sequence of images (i) to identify a first motion vector that provides a first estimate of a motion of the image array between the current image and the reference image, and (ii) to obtain a first error estimate for the first estimate of the motion of the image array;

when the first error estimate is greater than a threshold value, performing a second different motion estimation process that searches for the image array in the reference image (i) to identify a second motion vector that provides a second estimate of the motion of the image array between the current image and the reference image, and (ii) to obtain a second error estimate for the second estimate of the motion of the image array;

when the second error estimate is greater than the threshold value, performing a third motion estimation process to obtain a third estimate of the motion of the image array between the current image and the reference image; and

encoding said current image based the motion of the image array.

2. The non-transitory computer readable medium of claim 1, wherein the computer program further comprises a set of instructions for specifying the first estimate of the motion of the image array as a final estimate of the motion of the image array when the first error estimate is less than or equal to the threshold value.

3. The non-transitory computer readable medium of claim 1, wherein the set of instructions for performing the first motion estimation process comprises sets of instructions for:

comparing the image array obtained from the current image with a set of different reference image arrays that are obtained from the reference image, wherein the image array obtained from the current image is a current-image array and each image array obtained from the reference image is a reference-image array;

identifying a particular reference-image array that is least different from the current-image array based on the comparison; and

identifying the first motion vector based on a position of the current-image array in the current image and a position of the particular reference-image array in the reference image.

4. The non-transitory computer readable medium of claim 3, wherein the computer program further comprises a set of instructions for computing the first error estimate by aggregating an absolute value of a difference between corresponding image values in the current-image array and the particular reference-image array.

5. The non-transitory computer readable medium of claim 3, wherein the set of instructions for comparing the current-image array and the set of different reference-image arrays comprises a set of instructions for aggregating an absolute value of a difference between corresponding image values in the current-image array and the particular reference-image array.

6. The non-transitory computer readable medium of claim 1, wherein the image array specifies image values for a two-dimensional set of pixels in the current image.

7. The non-transitory computer readable medium of claim 6, wherein the set of pixels is a macroblock that has a width of 16 pixels and a height of 16 pixels.

8. The non-transitory computer readable medium of claim 1, wherein the computer program further comprises a set of instructions for specifying the second estimate of the motion of the image array as a final estimate of the motion of the image array when the second error estimate is less than or equal to the threshold value.

9. The non-transitory computer readable medium of claim 1, wherein each of the motion estimation processes performs at least one search in at least one reference image to obtain the corresponding estimate of the motion of the image array.

10. The non-transitory computer readable medium of claim 1, wherein the motion estimation processes are sorted from an expected fastest to an expected slowest, wherein the computer program further comprises sets of instructions for iteratively selecting each of the motion estimation processes according to the sorted order to obtain the corresponding estimate of the motion of the image array and to obtain the corresponding error estimate.

11. The non-transitory computer readable medium of claim 10, wherein the set of instructions for performing the third motion estimation process comprises a set of instructions for obtaining a third error estimate for the third estimate of the motion of the image array, wherein the computer program further comprises a set of instructions for specifying the third estimate of the motion of the image array as a specified estimate of the motion of the image array when the third error estimate is less than or equal to the threshold value.

12. The non-transitory computer readable medium of claim 10, wherein the computer program further comprises a set of instructions for selecting a lowest error estimate from each of the error estimates produced from performing each of the motion estimation processes as a selected estimate of the motion of the image array when none of the error estimates is less than or equal to the threshold value.

13. A non-transitory computer readable medium storing a computer program that when executed by at least one processor performs motion-estimation for an image array obtained from a current image in a sequence of video images, the computer program comprising sets of instructions for:

iteratively;

selecting a particular motion-estimation searching process from a list of at least three different motion-estimation searching processes that are sorted from an expected fastest to an expected slowest;

performing the particular motion-estimation searching process that searches for the image array in a reference image in the sequence of video images (i) to identify a motion vector that provides an estimate of a motion of the image array and (ii) to identify a motion-estimation error for the estimate of the motion of the image array, the iterative selecting and performing terminated when the identified motion-estimation error meets a same threshold; and

encoding said current image based the motion of the image array.

14. The non-transitory computer readable medium of claim 13, wherein each particular motion-estimation searching process produces a motion estimation, wherein the computer program further comprises a set of instructions for defining a particular motion estimation that meets the same threshold as a specified estimate of a motion of a set of pixels in an image of the video.

15. The non-transitory computer readable medium of claim 13, wherein the set of instructions for selecting the particular motion-estimation searching process comprises a set of instructions for selecting, during each iteration, an expected-fastest motion-estimation searching process in the list that has not been selected.

16. The non-transitory computer readable medium of claim 13, wherein the list of motion-estimation searching processes comprises a third motion-estimation searching process that is different from the first and second processes.

17. The non-transitory computer readable medium of claim 16, wherein the first motion-estimation searching process is designated as a faster process than the second motion-estimation searching process, and the second motion-estimation searching process is designated as a faster process than the third motion-estimation searching process.

18. A method of performing motion estimation for an image array obtained from a current image in a sequence of video images, the method comprising:

performing a first motion estimation process that searches for the image array in a reference image in the sequence of images (i) to identify a first motion vector that provides a first estimate of a motion of the image array between the current image and the reference image, and (ii) to obtain a first error estimate for the first estimate of the motion of the image array;

when the first error estimate is greater than a threshold value, performing a second different motion estimation process that searches for the image array in the reference image (i) to identify a second motion vector that provides a second estimate of the motion of the image array between the current image and the reference image, and (ii) to obtain a second error estimate for the second estimate of the motion of the image array;

when the second error estimate is greater than the threshold value, performing a third motion estimation process to obtain a third estimate of the motion of the image array between the current image and the reference image; and

encoding said current image based the motion of the image array.

19. The method of claim 18, wherein the first motion estimation process is designated as a faster process than the second motion estimation process, and the second motion estimation process is designated as a faster process than the third motion estimation process.

20. The method of claim 18 further comprising receiving a sorted list comprising a plurality of motion estimation processes, the sorted list comprising the first motion estimation process that, based on prior analysis, is expected to be a fastest process in the plurality of motion estimation processes and a last motion estimation process that, based on the prior analysis, is expected to be a slowest process in the plurality of motion estimation processes, wherein performing the motion estimation processes comprises iteratively performing one or more of the plurality of motion estimation processes according to the sorted list.

21. A method of performing motion-estimation for an image array obtained from a current image in a sequence of video images, the method comprising:

iteratively;

selecting a particular motion-estimation searching process from a list of at least three different motion-estimation searching processes that are sorted from an expected fastest to an expected slowest;

performing the particular motion-estimation searching process that searches for the image array in a reference image in the sequence of video images (i) to identify a motion vector that provides an estimate of a motion of the image array and (ii) to identify a motion-estimation error for the estimate of the motion of the image array, the iterative selecting and performing terminated when the identified motion-estimation error meets a same threshold; and

encoding said current image based the motion of the image array.

22. The method of claim 21, wherein each particular motion-estimation searching process produces a motion estimation, the method further comprising defining a particular motion estimation that meets the same threshold as a specified estimate of a motion of a set of pixels in an image of the video.

23. The method of claim 21, wherein selecting the particular motion-estimation searching process comprises selecting, during each iteration, an expected-fastest motion-estimation searching process in the list that has not been selected.

24. The method of claim 21, wherein the list of motion-estimation searching processes comprises a third motion-estimation searching process that is different from the first and second processes.

25. The method of claim 24, wherein the first motion-estimation searching process is designated as a faster process than the second motion-estimation searching process, and the second motion-estimation searching process is designated as a faster process than the third motion-estimation searching process.