HAND JITTER REDUCTION FOR COMPENSATING FOR LINEAR DISPLACEMENT

• US 20070236579A1
• Filed: 09/25/2006
• Published: 10/11/2007
• Est. Priority Date: 01/19/2006
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Qi, Yingyong, Li, Jingqiang, Noyes, Ying Xie
• Granted Patent: US 8,019,179 B2
• Time in Patent Office: Days
• US Class Current: 348/208.990
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations

Abstract

The registration of images comprising generating a plurality of projections from a base frame and generating a plurality of projections from a movement frame. Comparing a set of projections from the base frame, with a second set of projections from the movement frame, and generating a global motion vector estimate to add to the base frame.

Claims

1. An apparatus operable to compensate for linear displacement comprising:

a projection generator configured to generate a first plurality of projections from a base frame and a second plurality of projections from a movement frame;

a projection correlator configured to compare a first set of projections amongst the first plurality of projections with a second set of projections amongst the second plurality of projections, and generate a set of projection correlation error (pce) values;

a pce value index selector configured to select at least one minimum shift value determined by selecting a shift value that corresponds to the at least one minimum pce value amongst the set of pce values, to generate a component of a global motion vector estimate;

a frame registrator configured to generate an updated base image-component comprising a first adder to add an image-component to the global motion vector estimate;

a first comparator for detecting a first sign-bit output of a first minimum shift value with a first threshold;

a second comparator for detecting a second sign-bit output of a second minimum shift value with a second threshold; and

an early terminator configured to generate an early exit signal based on a combination of the first sign-bit output and the second sign-bit output.

2. The apparatus of claim 1, wherein the projection correlator comprises a shifter for shift aligning the first set of projections with the second set of projections.

3. The apparatus of claim 2, wherein the first set of the projections is a first vector and the second set of projections is a second vector.

4. The apparatus of claim 3, wherein the projection correlator comprises a subtractor for performing a subtraction operation between the first vector and the second vector to generate a projection correlation error (pce) vector.

5. The apparatus of claim 4 wherein a norm of the pce vector is taken to generate a pce value.

6. The apparatus of claim 5 wherein the norm is an L1 norm.

7. The apparatus of claim 1, wherein the base frame and the movement frame is selected based on a parity of a number of total frames to be processed in a row from a frame flow-tree.

8. The apparatus claim 1, wherein the first set of projections and the second set of projections are low pass filtered prior to the generation of the set of pce values.

9. The apparatus of claim 1, wherein the first set of projections and the second set of projections are interpolated.

10. The apparatus of claim 1, wherein the frame registrator is further comprises at least one adder to add an image-component to the global motion vector estimate to generate an updated base image-component.

11. A method of frame registration of images comprising:

generating a first plurality of projections from a first frame;

generating a second plurality of projections from a second frame;

comparing a first set of projections amongst the first plurality of projections with a second set of projections amongst the second plurality of projections, and generate a set of projection correlation error (pce) values;

selecting the first frame and second frame based on a parity based of a number of total frames to be processed in a row from a frame flow-tree;

generating a component of a global motion vector estimate by selecting a minimum pce value from among the set of pce values ; and

adding the estimated global motion vector to the first frame.

12. The method of claim 11, wherein the comparing further comprises taking the first set of projections and shift aligning them with the second set of projections.

13. The method of claim 12, wherein the first set of projections is a first vector and the second set of projections is a second vector.

14. The method of claim 13, wherein the comparing further comprises performing a subtraction operation between the first vector and the second vector to generate a pce vector.

15. The method of claim 14, wherein a norm of the pce vector is taken to generate a pce value.

16. The method of claim 15, wherein the norm is an L1 norm.

17. The method of claim 11, wherein the generating the global motion vector estimate comprises selecting a first minimum pce value from amongst a first set of pce values to determine a first minimum shift value, and selecting a second minimum pce value from amongst a second set of pce values to determine a second minimum shift value.

18. The method of claim 17, wherein the registration stops when the first minimum shift value is smaller than a first shift threshold and the second minimum shift value is smaller than a second shift threshold.

19. The method of claim 11, wherein at least one set of projections from the first frame and at least on set of projections from the second frame are low pass filtered.

20. The method of claim 11, wherein at least one set of projections from the first frame and at least one set of projections from the second frame are interpolated.

21. The method of claim 11, wherein the first frame and the second frame comprises pixels.

22. The method of claim 11, wherein the first frame and the second frame comprises spatial-frequency transformed pixels.

23. A computer-readable medium embodying a set of instructions, wherein the set of instructions when executed by one or more processors comprises:

computer-readable program code means for generating a first plurality of projections from a first frame;

computer-readable program code means for generating a second plurality of projections from a second frame;

computer-readable program code means for comparing a first set of projections amongst the first plurality of projections with a second set of projections amongst the second plurality of projections, and generate a set of projection correlation error (pce) values;

computer-readable program code means for selecting the first frame and second frame based on a parity based of a number of total frames to be processed in a row from a frame flow-tree;

computer-readable program code means for generating a component of a global motion vector estimate by selecting a minimum pce value from among the set of pce values ; and

computer-readable program code means for adding the estimated global motion vector to the first frame.

24. The computer-readable medium of claim 23, wherein the computer-readable program code means for comparing further comprises taking the first set of projections and shift aligning them with the second set of projections.

25. The computer-readable medium of claim 24, wherein the first set of projections is a first vector and the second set of projections is a second vector.

26. The computer-readable medium of claim 25, wherein the first vector and the second vector are subtracted from each other to generate a pce vector.

27. The computer-readable medium of claim 26, wherein a norm of the pce vector is taken to generate a pce value.

28. The computer-readable medium of claim 27, wherein the norm is an L1 norm.

29. The computer-readable medium of claim 23, wherein the computer-readable program code means for generating the global motion vector estimate comprises selecting a first minimum pce value from amongst a first set of pce values to determine a first minimum shift value, and selecting a second minimum pce value from amongst a second set of pce values to determine a second minimum shift value.

30. The computer-readable medium of claim 29, wherein registration stops when the first minimum shift value is smaller than a first shift threshold and the second minimum shift value is smaller than a second shift threshold.

31. The computer-readable medium of claim 23, wherein at least one set of projections from the first frame and at least one set of projections from the second frame are low pass filtered.

32. The computer-readable medium of claim 23, wherein at least on set of projections from the first frame and at least one set of projections from the second frame are decimated.

33. The method computer-readable medium of claim 23, wherein the first frame and the second frame comprises pixels.

34. The computer-readable medium of claim 23, wherein the first frame and the second frame comprises spatial-frequency transformed pixels.

35. An apparatus comprising:

means for generating a first plurality of projections from a first frame;

means for generating a second plurality of projections from a second frame;

means for comparing a first set of projections amongst the first plurality of projections with a second set of projections amongst the second plurality of projections, and generate a set of projection correlation error (pce) values;

means for selecting the first frame and second frame based on a parity based of a number of total frames to be processed in a row from a frame flow-tree;

means for generating a component of a global motion vector estimate by selecting a minimum pce value from among the set of pce values ; and

means for registration of a frame based on adding an image-component to the global motion vector estimate.

36. The apparatus of claim 35, wherein the means for comparing further comprises taking the first set of projections and shift aligning them with the second set of projections.

37. The apparatus of claim 36, wherein the first set of projections is a first vector and the second set of projections is a second vector.

38. The apparatus of claim 37, wherein the first vector and the second vector are subtracted from each other to generate a pce vector.

39. The apparatus of claim 38, wherein a norm of the pce vector is taken to generate a pce value.

40. The apparatus of claim 39, wherein the norm is an L1 norm.

41. The apparatus of claim 35, wherein the means for generating the global motion vector estimate comprises selecting a first minimum pce value from amongst a first set of pce values to determine a first minimum shift value, and selecting a second minimum pce value from amongst a second set of pce values to determine a second minimum shift value.

42. The apparatus of claim 41, wherein the means for registration stops when the first minimum shift value is smaller than a first shift threshold and the second minimum shift value is smaller than a second shift threshold.

43. The apparatus of claim 35, wherein at least one set of projections from the first frame and at least on set of projections from the second frame are low pass filtered.

44. The apparatus of claim 35, wherein at least one set of projections from the first frame and at least one set of projections from the second frame are interpolated.

45. The apparatus of claim 35, wherein at least one set of projections from the first frame and at least one set of projections from the second frame are decimated.

46. The apparatus of claim 35, wherein the first frame and the second frame comprises pixels.

47. The apparatus of claim 35, wherein the first frame and the second frame comprises spatial-frequency transformed pixels.

48. A method of global motion vector estimation in a mobile unit comprising:

generating a global motion vector between two captured frames;

checking a parity for a total number of frames processed in generation of the global motion vector;

using previously registered frames to select a base frame or a movement frame; and

generating at least one additional updated base image-component comprising at least one adder to add an image-component to generate the global motion vector estimate.

49. The method of claim 48, wherein the global motion vector is based on registering more than one pair of base and movement frames.