External vectors in a motion estimation system

• US 8,929,451 B2
• Filed: 07/31/2012
• Issued: 01/06/2015
• Est. Priority Date: 08/04/2011
• Status: Active Grant

• Current Assignee: Imagination Technologies Limited (Imagination Technologies Group Ltd.)
• Original Assignee: Imagination Technologies Limited (Imagination Technologies Group Ltd.)
• Inventors: Fishwick, Steven, Morphet, Stephen
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): PONTIUS, JAMES M
• Application Number:
  US13/563,525
• Publication Number:
  US 20130101041A1
• Time in Patent Office: 889 Days
• Field of Search: 375/240.16
• US Class Current: 375/240.16
• CPC Class Codes: H04N 19/513 Processing of motion vectors
• CPC Class Codes: H04N 19/521 for estimating the reliability of the determined motion vectors or motion vector field, e.g. for smoothing the motion vector field or for correcting motion vectors
• CPC Class Codes: H04N 19/56 Motion estimation with initialisation of the vector search, e.g. estimating a good candidate to initiate a search
• CPC Class Codes: H04N 19/61 in combination with predictive coding

Abstract

A method and apparatus for motion estimation in a sequence of video images is provided. Each field or frame in a sequence of video images is sub-divided into a plurality of blocks. Each block in each video field or frame has assigned to it a set of candidate motion vectors. The vector which produces a best match to a block in a previous field or frame, from the set of candidate motion vectors, is assigned to that block thus forming the motion vector field for the current video field or frame using the selected vector. The set of candidate motion vectors assigned to a block include one or more candidate vectors derived from an external source vector field and each such vector from an external source vector field is assigned a weighting that biases towards or away from the selection of that vector.

Claims

1. A machine-implemented method for motion estimation in a sequence of video images, comprising:

subdividing each field or frame of a sequence of video images into a plurality of blocks;

assigning to each block in each video field or frame a set of candidate motion vectors;

for each block in a current video field or frame, selecting a vector that produces a best match to a block in a previous video field or frame, from the set of candidate motion vectors assigned to that block; and

forming a motion vector field for the current video field or frame using the selected vector for each block,

wherein the set of candidate motion vectors assigned to a block include one or more candidate vectors derived from an external source vector field,

and wherein each candidate motion vector from an external source is assigned a weighting value, that biases towards or away from the selection of that candidate motion vector.

2. A method according to claim 1, wherein an external source vector field is assigned a weighting value that biases towards or away from the selection of all candidate motion vectors from the external source vector field.

3. A method according to claim 1, wherein the weighting value that biases towards or away from the selection of a candidate motion vector from an external source vector field is derived from the weighting value assigned to the candidate motion vector and from the weighting value assigned to the vector field from the external source.

4. A method according to claim 1, wherein the weighting value that biases towards or away from the selection of a candidate motion vector is provided by the external source.

5. A method according to claim 1, wherein the weighting value that biases towards or away from the selection of a candidate motion vector is determined from the external source.

6. A method according to claim 5, wherein the weighting value that biases towards or away from the selection of a candidate motion vector is derived from the number of pixels in the block in a current video field or frame that are co-located with pixels used in the generation of the motion vector candidate provided by the external source.

7. A method according to claim 2, wherein the weighting value that biases towards or away from the selection of candidate motion vectors from the external source vector field is provided by the external source.

8. A method according to claim 2, wherein the weighting value that biases towards or away from the selection of candidate motion vectors from the external source vector field is determined from the external source.

9. A method according to claim 1, wherein the weighting value that biases towards or away from the selection of candidate motion vectors from the external source is derived from the similarity of the vector field formed from the selected vectors for each block to the external source vector field.

10. A method according to claim 1, wherein the weighting value that biases towards or away from the selection of candidate motion vectors from the external source is derived from the similarity of a region of vectors in the vector field formed from the selected vectors for each block to a co-located region of vectors in the external source vector field.

11. A method according to claim 1, wherein the weighting value that biases towards or away from the selection of candidate motion vectors from the external source is derived from the similarity of a set of recently selected vectors to the co-located vectors in the external source vector field.

12. A method according to claim 9, wherein the weighting value that biases towards or away from the selection of candidate motion vectors from the external source is derived from the historical similarity of vector fields previously formed from the selected vectors for each block to the external source vector fields.

13. A method according to claim 9, wherein the similarity of vector fields is determined by accumulating a measure of similarity of the vectors within the fields, and similarity of vectors is derived from the difference in magnitude and direction of the vectors.

14. A method according to claim 9, wherein the similarity of vector fields is determined by entropic measures.

15. A method according to claim 1, wherein the one or more candidate motion vectors derived from an external source vector field are derived from a motion estimator elsewhere in the system.

16. A method according to claim 1, wherein the one or more candidate motion vectors derived from an external source vector field are derived from a source elsewhere in the system that is not a motion estimator.

17. A method according to claim 1, wherein the one or more candidate motion vectors derived from an external source vector field are derived from vectors supplied by a computer graphics engine.

18. A method according to claim 1, wherein the one or more candidate motion vectors derived from an external source vector field are derived from vectors supplied by an overlay compositing engine.

19. A method according to claim 1, wherein a best match is defined as the vector that represents the true motion of objects in the scene.

20. A method according to claim 1, wherein a best match is defined as the vector that produces the closest match between areas of pixel data in the current and previous video fields or frames.

21. Apparatus for motion estimation in a sequence of video images comprising:

a source of pluralities of blocks, each plurality from a respective subdivided field or frame of a sequence of video images;

a generator of internal motion vector candidates;

one or more external sources of candidate motion vectors;

a memory coupled to the generator of internal motion vector candidates and to the one or more external sources of candidate motion vectors, and storing a motion vector candidate list, comprising internal motion vector candidates and one or more motion vectors from the one or more external sources of candidate motion vectors;

a selector configured, for each block in a current video field or frame, to select a vector that produces a best match to a block in a previous video field or frame of the sequence of video images, from a set of candidate motion vectors assigned to that block from the motion vector candidate list; and

one or more of a deinterlacer and a frame rate converter configured to form a motion vector field for the current video field or frame using the selected vector for each block, wherein

each candidate motion vector from the external source is assigned a weighting value that biases towards or away from the selection of that candidate motion vector.

22. Apparatus according to claim 21, wherein an external source vector field is assigned a weighting value that biases towards or away from the selection of all candidate motion vectors from the external source vector field.

23. Apparatus according to claim 21, wherein the weighting value that biases towards or away from the selection of a candidate motion vector is derived from the weighting value assigned to the candidate motion vector and from the weighting value assigned to the vector field from the external source.

24. Apparatus according to claim 21, wherein the weighting value that biases towards or away from the selection of a candidate motion vector is provided by the external source.

25. Apparatus according to claim 21, wherein the weighting value that biases towards or away from the selection of a candidate motion vector is determined from the external source.

26. Apparatus according to claim 25, wherein the weighting value that biases towards or away from the selection of a candidate motion vector is derived from the number of pixels in the block in a current video field or frame that are co-located with pixels used in the generation of the motion vector candidate provided by the external source.

27. Apparatus according to claim 21, wherein the weighting value that biases towards or away from the selection of candidate motion vectors from the external source is derived from the similarity of the vector field formed from the selected vectors for each block to the external source vector field.

28. Apparatus according to claim 21, wherein the weighting value that biases towards or away from the selection of candidate motion vectors from the external source is derived from the similarity of a region of vectors in the vector field formed from the selected vectors for each block to a co-located region of vectors in the external source vector field.

29. Apparatus according to claim 21, wherein the weighting value that biases towards or away from the selection of candidate motion vectors from the external source is derived from the similarity of a set of recently selected vectors to the co-located vectors in the external source vector field.

30. Apparatus according to claim 29, wherein the weighting value that biases towards or away from the selection of candidate motion vectors from the external source is derived from the historical similarity of vector fields previously formed from the selected vectors for each block to the external source vector fields.

31. Apparatus according to claim 29, wherein the similarity of vector fields is determined by accumulating a measure of similarity of the vectors within the fields, and similarity of vectors is derived from the difference in magnitude and direction of the vectors.

32. Apparatus according to claim 29, wherein the similarity of vector fields is determined by entropic measures.

33. Apparatus according to claim 21, wherein the candidate motion vector derived from an external source is derived from a motion estimator elsewhere in the system.

34. Apparatus according to claim 21, wherein the candidate motion vector derived from an external source is derived from a source elsewhere in the system that is not a motion estimator.

35. Apparatus according to claim 21, wherein the candidate motion vector derived from an external source is derived from vectors supplied by a computer graphics engine.

36. Apparatus according to claim 21, wherein the candidate motion vector from an external source is derived from vectors supplied by an overlay compositing engine.