Robust camera pan vector estimation using iterative center of mass

• US 8,019,124 B2
• Filed: 10/23/2008
• Issued: 09/13/2011
• Est. Priority Date: 12/23/2003
• Status: Active Grant

• Current Assignee: Tamiras Per Pte Ltd LLC (Intellectual Ventures LLC)
• Original Assignee: Tamiras Per Pte Ltd LLC (Intellectual Ventures LLC)
• Inventors: Nair, Hari N.
• Primary Examiner(s): Le; Vu
• Assistant Examiner(s): Abdi; Amara
• Application Number:
  US12/257,204
• Publication Number:
  US 20090086103A1
• Time in Patent Office: 1,055 Days
• Field of Search: 382/107, 348/669, 348/416, 348/699
• US Class Current: 382/107
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: H04N 5/145 Movement estimation for video coding H04N19/51

Abstract

Methods and apparatus, including computer program products, implementing and using techniques for camera pan vector estimation, are disclosed. A camera model is provided for representing motion vectors based on a plurality of parameters. A distribution of the motion vectors is generated. A cluster in the distribution of motion vectors is identified. A center of mass of the identified cluster is determined. The camera pan vector is provided based upon the determined center of mass.

Claims

1. A method for generating motion vectors, the method comprising:

identifying a motion vector of an image at a first resolution;

using the identified motion vector to generate a first test vector and a second test vector for the image at a second resolution, the second resolution of higher resolution than the first resolution;

determining one or more camera model parameters at the second resolution of the image based on the first test vector, the second test vector, and known position locations in the image; and

generating a first motion vector and a second motion vector for the known position locations in the image at the second resolution based on the one or more camera model parameters.

2. The method of claim 1, further comprising:

determining further camera model parameters for further known position locations in the image based on the first motion vector and the second motion vector; and

generating a third motion vector and a fourth motion vector for the further known position locations in the image based on the further camera model parameters.

3. The method of claim 1, wherein the one or more camera model parameters include at least one of a horizontal translation parameter, vertical translation parameter, camera rotation parameter, zoom out parameter, and zoom in parameter.

4. The method of claim 1, further comprising:

generating a plurality of motion vectors recursively using the first motion vector and the second motion vector as initial inputs.

5. The method of claim 4, wherein a majority of the plurality of motion vectors form a cluster.

6. The method of claim 5, further comprising:

determining horizontal and vertical translation camera model parameters based on a center of mass of the cluster.

7. The method of claim 5, further comprising:

discarding any motion vectors of the plurality of motion vectors that are not part of the cluster.

8. The method of claim 1, wherein the first motion vector and the second motion vector each include values ranging from a positive maximum value to a negative maximum value.

9. The method of claim 8, wherein the absolute value of the negative maximum value is the same as the absolute value of the positive maximum value.

10. The method of claim 8, wherein the first motion vector and the second motion vector form a grid of motion vector values.

11. The method of claim 10, wherein the grid of motion vector values has a size based on a sum of the absolute value of the negative maximum value of the first motion vector and the positive maximum value of the first motion vector and a sum of the absolute value of the negative maximum value of the second motion vector and the positive maximum value of the second motion vector.

12. The method of claim 4, wherein said determining one or more camera model parameters includes categorizing each of the plurality of motion vectors into a bin configured to represent a possible value of each of the plurality of motion vectors.

21. The method of claim 1, wherein the identified motion vector has a minimum sum of absolute differences of pixels positioned relative to pixels in a previous motion vector from a previous image frame.

22. The method of claim 21, wherein said identifying a motion vector comprises:

projecting the previous motion vector from the previous image frame at the first resolution;

providing a plurality of motion vectors in the image frame; and

comparing a position of pixels in each of the plurality of motion vectors in the image frame to a position of pixels in the previous motion vector from the previous image frame to identify the motion vector in the image frame having the minimum sum of absolute differences of pixels positioned relative to pixels in the previous motion vector from the previous image frame.

13. An apparatus comprising:

a motion vector selector configured to identify a motion vector of an image at a first resolution;

a camera model parameter calculator configured to use the identified motion vector to generate a first test vector and a second test vector for the image at a second resolution, the second resolution of higher resolution than the first resolution, and to determine one or more camera model parameters based on the first test vector, the second test vector, and known position locations in the image at the second resolution; and

a camera model motion vector generator configured to generate a first motion vector and a second motion vector for the known position locations in the image at the second resolution based on the one or more camera model parameters.

14. The apparatus of claim 13, wherein the camera model motion vector generator is further configured to recursively generate a plurality of motion vectors for further known position locations in the image based on further camera model parameters calculated from previously generated motion vectors.

15. The apparatus of claim 14, wherein a majority of the plurality of motion vectors form a cluster.

16. The apparatus of claim 15, wherein the camera model parameter calculator is further configured to determine horizontal and vertical translation camera model parameters based on a center of mass of the cluster.

17. The apparatus of claim 15, wherein the camera model motion vector generator is further configured to discard any motion vectors of the plurality of motion vectors that are not part of the cluster.

23. The apparatus of claim 13, wherein the identified motion vector has a minimum sum of absolute differences of pixels positioned relative to pixels in a previous motion vector from a previous image frame.

24. The apparatus of claim 23, wherein said identifying a motion vector comprises:

projecting the previous motion vector from the previous image frame at the first resolution;

providing a plurality of motion vectors in the image frame; and

comparing a position of pixels in each of the plurality of motion vectors in the image frame to a position of pixels in the previous motion vector from the previous image frame to identify the motion vector in the image frame having the minimum sum of absolute differences of pixels positioned relative to pixels in the previous motion vector from the previous image frame.

25. The apparatus of claim 24, wherein the one or more camera model parameters include at least one of a horizontal camera pan parameter, vertical camera pan parameter, zoom in parameter, zoom out parameter, and camera rotation parameter.

18. A tangible computer-readable medium having instructions stored thereon that, in response to execution by a computing device, cause the computing device to perform operations comprising:

identifying a motion vector of an image at a first resolution;

using the identified motion vector to generate a first test vector and a second test vector for the image at a second resolution, the second resolution of higher resolution than the first resolution;

determining one or more camera model parameters at the second resolution of the image based on the first test vector, the second test vector, and known position locations in the image at the second resolution; and

generating a first motion vector and a second motion vector for the known position locations in the image at the second resolution based on the camera model parameters.

19. The tangible computer-readable medium of claim 18, having further instructions stored thereon that, in response to execution by the computing device, cause the computing device to perform operations comprising:

generating a plurality of motion vectors recursively using the first motion vector and the second motion vector as initial inputs.

20. The tangible computer-readable medium of claim 19, having further instructions stored thereon that, in response to execution by the computing device, cause the computing device to perform operations comprising:

determining a cluster from a majority of the plurality of motion vectors;

determining horizontal and vertical translation camera model parameters based on a center of mass of the cluster; and

discarding any motion vectors of the plurality of motion vectors that are not part of the cluster.

26. The tangible computer-readable medium of claim 18, wherein the identified motion vector has a minimum sum of absolute differences of pixels positioned relative to pixels in a previous motion vector from a previous image frame.

27. The tangible computer-readable medium of claim 26, wherein the one or more camera model parameters include at least one of a horizontal camera pan parameter, vertical camera pan parameter, zoom in parameter, zoom out parameter, and camera rotation parameter.