Method and apparatus for compensating for jitter in a digital video image

• US 6,628,711 B1
• Filed: 07/02/1999
• Issued: 09/30/2003
• Est. Priority Date: 07/02/1999
• Status: Expired due to Term

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Motorola, Inc. (Motorola Solutions, Inc.)
• Inventors: Zhang, Xing, Mathew, Reji
• Primary Examiner(s): Kelley, Chris
• Assistant Examiner(s): Parsons, Charles E
• Application Number:
  US09/347,082
• Time in Patent Office: 1,551 Days
• Field of Search: 375/240.16, 375/240.12, 348/208.99, 348/208.13, 348/208.6, 348/208.4
• US Class Current: 375/240.12
• CPC Class Codes: H04N 19/527 Global motion vector estimation
• CPC Class Codes: H04N 19/61 in combination with predictive coding
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/6815 by distinguishing pan or tilt from motion
• CPC Class Codes: H04N 23/682 Vibration or motion blur correction
• CPC Class Codes: H04N 5/21 Circuitry for suppressing or minimising disturbance, e.g. moiré or halo

Abstract

Apparatus (10) for compensating for jitter in a digital image forming part of a video sequence of such digital images includes a first motion estimation unit (13) having an input for receiving sensed image data and an output for providing a first plurality of motion vectors, each corresponding to one of a plurality of image blocks making up the digital image. A jitter estimation unit (14) determines a jitter vector from the first plurality of motion vectors for the digital image and a jitter compensation unit (15) adjusts the digital image using the jitter vector to compensate for jitter in the digital image. A second motion estimation unit (16) provides a second plurality of motion vectors, each corresponding to one of a plurality of image blocks making up the jitter compensated digital image. The second plurality of motion vectors are determined by utilizing the first plurality of motion vectors and the jitter vector. The second plurality of motion vectors can be used in coding the jitter compensated image for storage and/or onward transmission.

Claims

1. A method of compensating for jitter in a digital image forming part of a video sequence of such digital images, the method comprising the steps of:

providing a first plurality of motion vectors each corresponding to one of a plurality of image blocks making up the digital image,

determining a jitter vector from the first plurality of motion vectors for the digital image and from a second plurality of motion vectors provided for a previous digital image in the video sequence;

adjusting the digital image using the jitter vector to compensate for jitter in the digital image to produce a jitter compensated digital image;

providing a second plurality of motion vectors including the steps of, for each image block in the jitter compensated digital image,

determining a corresponding best matching image block in the digital image; and

utilizing the motion vector from the first plurality of motion vectors for the corresponding best matching image block and the jitter vector to determine a jitter-compensated motion vector for the image block in the jitter compensated digital image.

2. A method of compensating for jitter in a digital image according to claim 1, wherein the step of providing a first plurality of motion vectors includes the steps of, for each of the plurality of image blocks:

determining a corresponding best matching image block in the previous digital image;

determining a displacement value between the location of the image block in the digital image and the location of the corresponding best matching image block in the previous digital image; and

utilizing the displacement value for the image block to determine a motion vector for that image block.

3. A method of compensating for jitter in a digital image according to claim 1, wherein the step of determining a jitter vector utilizes a plurality of motion vectors for at least one previous digital image.

4. A method of compensating for jitter in a digital image according to claim 1, wherein the step of determining a jitter vector utilizes a plurality of motion vectors for at least one subsequent digital image.

5. A method of compensating for jitter in a digital image according to claim 1, wherein the plurality of motion vectors relate to at least one component of jitter.

6. A method of compensating for jitter in a digital image according to claim 1, wherein the steps of providing a plurality of motion vectors, determining a jitter vector and adjusting the digital image are carried out for at least two separate components of jitter.

7. A method of compensating for jitter in a digital image according to claim 1, wherein the digital image is received from an image sensing device.

8. A method of compensating for jitter in a digital image according to claim 7, wherein the received digital image is corrected using the jitter vectors.

9. A method of compensating for jitter in a digital image according to claim 8, wherein the motion vectors are encoded for at least one of storage and onward transmission.

10. A method of compensating for jitter in a digital image according to claim 8, wherein the step of adjusting the digital image includes the step of:

cropping a received digital image to produce a smaller digital image where the location of the smaller digital image within the received digital image is dependent on the jitter vector.

11. A method of compensating for jitter in a digital image according to claim 10, further comprising the step of upsampling the smaller digital image to produce a digital image of a desired size.

12. Apparatus for compensating for jitter in a digital image forming part of a video sequence of such digital images, the apparatus comprising:

a first motion estimation unit having an input for receiving sensed image data and an output for providing a first plurality of motion vectors, each corresponding to one of a plurality of image blocks making up the digital image;

a jitter estimation unit coupled to the first motion estimation unit for determining a jitter vector from the first plurality of motion vectors for the digital image;

a jitter compensation unit coupled to the jitter estimation unit for adjusting the digital image using the jitter vector to compensate for jitter in the digital image to produce a jitter compensated digital image;

a second motion estimation unit having a first input coupled to the first motion estimating unit for receiving the first plurality of motion vectors, a second input coupled to the jitter estimation unit for receiving the jitter vector and a third input coupled to the jitter compensating unit for receiving the jitter compensated digital image, the second motion estimation unit having an output for providing a second plurality of motion vectors, each corresponding to one of a plurality of image blocks making up the jitter compensated digital image.

13. Apparatus for compensating for jitter in a digital image according to claim 12, further comprising a motion compensation unit coupled to the second motion estimation unit for adjusting the jitter compensated digital image according to the second plurality of motion vectors to provide a motion compensated digital image.

14. Apparatus for compensating for jitter in a digital image according to claim 13, further comprising a comparator device coupled to the motion compensation unit and to the jitter compensation unit for comparing the jitter compensated digital image with the motion compensated digital image to provide a difference image.

15. Apparatus for compensating for jitter in a digital image according to claim 14, further comprising an encoding device coupled to the comparator device for encoding the difference image, the apparatus having a first output providing an encoded digital image.

16. Apparatus for compensating for jitter in a digital image according to claim 15, wherein the apparatus has a second output coupled to the output of the second motion estimation unit providing the second plurality of motion vectors.