Hand jitter reduction compensating for rotational motion

• US 7,970,239 B2
• Filed: 09/25/2006
• Issued: 06/28/2011
• Est. Priority Date: 01/19/2006
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Li, Jingqiang, Jiang, Xiaoyun, Quan, Shuxue
• Primary Examiner(s): Mehta; Bhavesh M
• Assistant Examiner(s): Thirugnanam; Gandhi
• Application Number:
  US11/534,935
• Publication Number:
  US 20070172150A1
• Time in Patent Office: 1,737 Days
• Field of Search: 382/154, 382/289, 382296-298
• US Class Current: 382/289
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06V 10/30 Noise filtering
• CPC Class Codes: G06V 10/507 Summing image-intensity values; Histogram projection analysis

Abstract

The registration of images comprising segmenting an image in a frame into a set of sectors which forms a circle. Generating a plurality of sets of projections in a base frame, wherein each set of projections is generated from any sector amongst the set of sectors from the base frame. Also generating a plurality of sets of projections in a movement frame, wherein each set of projections is generated from any sector amongst the set of sectors from the movement frame. Then summing each set of projections, from any sector amongst the set of sectors from the base frame and summing each set of projections from any sector amongst the set of sectors from the movement frame. Furthermore, comparing a set of each sum of projections from the base frame with a set of each sum of projections from the movement frame, and generating a rotation angle estimate to add to the base frame.

Claims

1. An apparatus configured to process images, comprising:

a region of interest locator for segmenting an image in a frame into a set of sectors which forms a circle;

a projection generator configured to generate a plurality of projections in any sector from the set of sectors;

a rotational integrator configured to generate a plurality of sums, wherein a sum is formed by adding the plurality of projections in the any sector;

a memory for storing a first plurality of sums generated by the rotational integrator and a second plurality of sums generated by the rotational integrator;

a projection correlator configured to compare the first plurality of sums with the second plurality of sums and configured to generate a set of projection correlation error (pce) values;

a pce value index selector configured to select a minimum shift value determined by selecting a shift value that corresponds to the minimum pce value amongst the set of pce values;

a look-up-table for mapping the minimum shift value to a rotation angle estimate between a first frame and a second frame; and

a frame registrator configured to generate an updated base image-component comprising a first adder to add the rotation angle estimate to an image-component, wherein the frame registrator is further configured to generate an early exit signal comprising a detector for detecting a sign-bit output of a comparator for comparing the rotation angle estimate to a shift threshold.

2. The apparatus of claim 1, wherein the projection correlator comprises a shifter for shift aligning the set of the sum of projections from the base frame with the set of the sum of projections from the movement frame, where shift aligning includes shifting the position between two input vectors for each position between the two vectors.

3. The apparatus of claim 2, wherein the first plurality of sums is a first vector and the second plurality of sums is a second vector.

4. The apparatus of claim 3, wherein the projection correlator comprises a subtractor for performing a subtraction operation between the first vector and the second vector to generate a projection correlation error (pce) vector.

5. The apparatus of claim 4, wherein a norm of the pce vector is taken to generate a pce value.

6. The apparatus of claim 5, wherein the norm is an L1 norm.

7. The apparatus of claim 1, wherein the first frame and the second frame are selected based on a parity of a number of total frames to be processed in a row from a frame flow-tree, wherein the frame flow-tree includes a tree of frames where leaf nodes include unregistered frames and other nodes include registered frames that are based on their respective child nodes.

8. The apparatus claim 1, wherein the at least one set of projections from the first frame and the at least one set of projections from the second frame are low pass filtered before the plurality of sums is generated.

9. The apparatus of claim 1, wherein the at least one set of projections from the first frame and the at least one set of projections from the second frame are interpolated.

10. The apparatus of claim 1, wherein the frame registrator is further configured to generate at least one additional updated base image-component comprising at least one adder to add an image-component to the rotation angle estimate.

11. A method of registration of images comprising:

segmenting an image in a frame into a set of sectors which forms a circle;

generating a plurality of projections in any sector from the set of sectors in a first frame;

generating a plurality of projections in any sector from the set of sectors in a second frame;

generating a first plurality of sums, wherein a sum is formed by adding the plurality of projections in the any sector in the first frame;

generating a second plurality of sums, wherein a sum is formed by adding the plurality of projections in the any sector in the second frame;

comparing the first plurality of sums with the second plurality of sums, to generate a set of projection correlation error (pce) values;

selecting a minimum shift value determined by selecting a shift value that corresponds to the minimum pce value amongst the set of pce values;

mapping the minimum shift value to a rotation angle estimate between the first frame and the second frame; and

registering the second frame by adding the rotation angle estimate to the image-components of the first frame, wherein the registering stops when the rotation angle estimate is smaller than a threshold.

12. The method of claim 11, wherein the comparing further comprises taking the first plurality of sums and shift aligning them with the second plurality of sums, where shift aligning includes shifting the position between two input vectors for each position between the two vectors.

13. The method of claim 12, wherein the first plurality of sums is a vector and the second plurality of sums is a second vector.

14. The method of claim 13, wherein the first vector and the second vector are subtracted from each other to generate a pce vector.

15. The method of claim 14, wherein a norm of the pce vector is taken to generate a pce value.

16. The method of claim 15, wherein the norm is an L1 norm.

17. The method of claim 11, wherein the first frame and the second frame are selected based on a parity of a number of total frames to be processed in a row from a frame flow-tree, wherein the frame flow-tree includes a tree of frames where leaf nodes include unregistered frames and other nodes include registered frames that are based on their respective child nodes.

18. The method of claim 11, wherein the plurality of projections in any sector from the set of sectors in a first frame and the plurality of projections in any sector from the set of sectors in a second frame are low pass filtered.

19. The method of claim 11, wherein the plurality of projections in any sector from the set of sectors in a first frame and the plurality of projections in any sector from the set of sectors in a second frame are interpolated.

20. The method of claim 11, wherein the first frame and the second frame comprises pixels.

21. The method of claim 11, wherein the first frame and the second frame comprises spatial-frequency transformed pixels.

22. A non-transitory computer-readable medium embodying a set of instructions, wherein the set of instructions when executed by one or more processors comprises:

computer-readable program code means for segmenting an image in a frame into a set of sectors which forms a circle;

computer-readable program code means for generating a plurality of projections in any sector from the set of sectors in a first frame;

computer-readable program code means for generating a plurality of projections in any sector from the set of sectors in a second frame;

computer-readable program code means for generating a first plurality of sums, wherein a sum is formed by adding the plurality of projections in the any sector in the first frame;

computer-readable program code means for generating a second plurality of sums, wherein a sum is formed by adding the plurality of projections in the any sector in the second frame;

computer-readable program code means for comparing the first plurality of sums with the second plurality of sums, to generate a set of projection correlation error (pce) values;

computer-readable program code means for selecting a minimum shift value determined by selecting a shift value that corresponds to the minimum pce value amongst the set of pce values;

computer-readable program code means for mapping the minimum shift value to a rotation angle estimate between the first frame and the second frame; and

computer-readable program code means for registering the second frame by adding the rotation angle estimate to the image-components of the first frame, wherein the registering stops when the rotation angle estimate is smaller than a threshold.

23. The computer-readable medium of claim 22, wherein the comparing further comprises taking the first plurality of sums and shift aligning them with the second plurality of sums, where shift aligning includes shifting the position between two input vectors for each position between the two vectors.

24. The computer-readable medium of claim 23, wherein the first plurality of sums is a vector and the second plurality of sums is a second vector.

25. The computer-readable medium of claim 24, wherein the first vector and the second vector are subtracted from each other to generate a pce vector.

26. The computer-readable medium of claim 25, wherein a norm of the pce vector is taken to generate a pce value.

27. The computer-readable medium of claim 26, wherein the norm is an L1 norm.

28. The computer-readable medium of claim 22, wherein the first frame and the second frame are selected based on a parity of a number of total frames to be processed in a row from a frame flow-tree, wherein the frame flow-tree includes a tree of frames where leaf nodes include unregistered frames and other nodes include registered frames that are based on their respective child nodes.

29. The computer-readable medium of claim 22, wherein the plurality of projections in any sector from the set of sectors in a first frame and the plurality of projections in any sector from the set of sectors in a second frame are low pass filtered.

30. The computer-readable medium of claim 22, wherein the plurality of projections in any sector from the set of sectors in a first frame and the plurality of projections in any sector from the set of sectors in a second frame are interpolated.

31. The computer-readable medium of claim 22, wherein the first frame and the second frame comprises pixels.

32. The computer-readable medium of claim 22, wherein the first frame and the second frame comprises spatial-frequency transformed pixels.

33. An apparatus configured to process images, comprising:

means for segmenting an image in a frame into a set of sectors which forms a circle;

means for generating a plurality of projections in any sector from the set of sectors in a first frame;

means for generating a plurality of projections in any sector from the set of sectors in a second frame;

means for generating a first plurality of sums, wherein a sum is formed by adding the plurality of projections in the any sector in the first frame;

means for generating a second plurality of sums, wherein a sum is formed by adding the plurality of projections in the any sector in the second frame;

means for comparing the first plurality of sums with the second plurality of sums, to generate a set of projection correlation error (pce) values;

means for selecting a minimum shift value determined by selecting a shift value that corresponds to the minimum pce value amongst the set of pce values;

means for mapping the minimum shift value to a rotation angle estimate between the first frame and the second frame; and

means for registering the second frame by adding the rotation angle estimate to the image-components of the first frame, wherein the registering stops when the rotation angle estimate is smaller than a threshold.

34. The apparatus of claim 33, wherein the means for comparing further comprises taking the first plurality of sums and shift aligning them with the second plurality of sums, where shift aligning includes shifting the position between two input vectors for each position between the two vectors.