BLOCK-BASED IMAGE STABILIZATION

• US 20090123082A1
• Filed: 11/12/2007
• Published: 05/14/2009
• Est. Priority Date: 11/12/2007
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Atanssov, Kalin, Li, Hsiang-Tsun, Hwang, Hau, Chiu, Chinchuan Andrew
• Granted Patent: US 8,600,189 B2
• Time in Patent Office: Days
• US Class Current: 382/236
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 5/002 Denoising; Smoothing noise processing or correction adapted to be used in an image pickup device containing and electronic image sensor H04N23/81, H04N25/60 - H04N25/67
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 7/223 using block-matching
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory

Abstract

This disclosure describes image stabilization techniques for devices with image capture capabilities. An image capture device may capture two or more images and combine the image using the techniques described in this disclosure. In particular, the image capture device may compute motion vectors for a plurality of blocks of pixels of one of the images. In cases, the image capture device may also interpolate or extrapolate motion vectors for individual pixels or sub-blocks of pixels using the block motion vectors. The image capture device may then average the first and second images by averaging each of the pixels of the first image with pixels of the second image that correspond to a location indicated by the plurality of motion vectors. The techniques may be particularly effective in reducing blur in image information resulting from certain movements during image capture or use of certain image capture technologies.

Claims

1. A method for processing digital image data comprising:

partitioning a first image of a scene of interest into a plurality of blocks of pixels;

computing, for each of the plurality of blocks, a motion vector that represents the offset between the block of the first image and a corresponding block of pixels within a second image of the scene of interest;

averaging pixel values for each of the pixels of the blocks of the first image with pixel values of corresponding pixels of the second image based on the motion vector of the respective block to which the pixels belong; and

storing the averaged pixel values.

2. The method of claim 1, further comprising:

determining, for each of the plurality of blocks, motion vectors for each of the pixels of the block using the motion vector computed for the block to which the pixel belongs and at least one other motion vector computed for one other block of the image, and

wherein averaging the pixel values comprises averaging the pixel values for each of the pixels of the blocks of the first image with pixel values of corresponding pixels of the second image at locations indicated by the respective motion vectors of the pixels.

3. The method of claim 2, wherein determining motion vectors for each of the pixels comprises determining motion vectors for each of the pixels using one of extrapolation and interpolation based on the motion vector computed for the block to which the pixel belongs and at least one other motion vector computed for one other block of the image.

4. The method of claim 1, further comprising:

determining, for each of the plurality of blocks, motion vectors for sub-blocks of pixels of the block using the motion vector computed for the block to which the sub-block belongs and at least one other motion vector computed for one other block of the image, and

wherein averaging the pixel values comprises averaging the pixel values for each of the sub-blocks of the first image with pixel values of corresponding sub-blocks of the second image at locations indicated by the respective motion vectors of the sub-blocks.

5. The method of claim 1, wherein computing a motion vector comprises:

computing, for each of the plurality of blocks, at least one of horizontal projections and vertical projections for the block; and

computing, for each of the plurality of blocks, a motion vector based on the at least one of the horizontal and vertical projections for the block.

6. The method of claim 1, further comprising capturing the first image and the second image, wherein at least one of the first image and the second image is captured during an image preview.

7. The method of claim 1, wherein partitioning the first image into a plurality of blocks of pixels comprises partitioning the first image into a plurality of overlapping blocks of pixels.

8. A computer-program product for processing digital image data comprising a computer readable medium having instructions thereon, the instructions comprising:

code for partitioning a first image of a scene of interest into a plurality of blocks of pixels;

code for computing, for each of the plurality of blocks, a motion vector that represents the offset between the block of the first image and a corresponding block of pixels within a second image of the scene of interest;

code for averaging pixel values for each of the pixels of the blocks of the first image with pixel values of corresponding pixels of the second image based on the motion vector of the respective block to which the pixels belong; and

code for storing the averaged pixel values.

9. The computer-program product of claim 8, further comprising:

code for determining, for each of the plurality of blocks, motion vectors for each of the pixels of the block using the motion vector computed for the block to which the pixel belongs and at least one other motion vector computed for one other block of the image, and

wherein code for averaging the pixel values comprises code for averaging the pixel values for each of the pixels of the blocks of the first image with pixel values of corresponding pixels of the second image at locations indicated by the respective motion vectors of the pixels.

10. The computer-program product of claim 9, wherein code for determining motion vectors for each of the pixels comprises code for determining motion vectors for each of the pixels using one of extrapolation and interpolation based on the motion vector computed for the block to which the pixel belongs and at least one other motion vector computed for one other block of the image.

11. The computer-program product of claim 8, further comprising:

determining, for each of the plurality of blocks, motion vectors for sub-blocks of pixels of the block using the motion vector computed for the block to which the sub-block belongs and at least one other motion vector computed for one other block of the image, and

wherein averaging the pixel values comprises averaging the pixel values for each of the sub-blocks of the first image with pixel values of corresponding sub-blocks of the second image at locations indicated by the respective motion vectors of the sub-blocks.

12. The computer-program product of claim 8, wherein code for computing a motion vector comprises:

code for computing, for each of the plurality of blocks, at least one of horizontal projections and vertical projections for the block; and

code for computing, for each of the plurality of blocks, a motion vector based on the at least one of the horizontal and vertical projections for the block.

13. The computer-program product of claim 8, further comprising code for capturing the first image and the second image, wherein at least one of the first image and the second image is captured during an image preview.

14. The computer-program product of claim 8, wherein code for partitioning the first image comprises code for partitioning the first image into a plurality of overlapping blocks of pixels.

15. An apparatus for processing digital image data comprising:

an image processor to process the image data, the image processor including;

a block partitioner to partition a first image of a scene of interest into a plurality of blocks of pixels;

a motion vector module that computes, for each of the plurality of blocks, a motion vector that represents the offset between the block of the first image and a corresponding block of pixels within a second image of the scene of interest; and

pixel averaging module that averages pixel values for each of the pixels of the blocks of the first image with pixel values of corresponding pixels of the second image based on the motion vector of the respective block to which the pixels belong; and

a memory to store the averaged pixel values.

16. The apparatus of claim 15, wherein:

the motion vector module determines, for each of the plurality of blocks, motion vectors for each of the pixels of the block using the motion vector computed for the block to which the pixel belongs and at least one other motion vector computed for one other block of the image, and

the pixel averaging module averages the pixel values for each of the pixels of the blocks of the first image with pixel values of corresponding pixels of the second image at locations indicated by the respective motion vectors of the pixels.

17. The apparatus of claim 16, wherein the motion vector module determines motion vectors for each of the pixels using one of extrapolation and interpolation based on the motion vector computed for the block to which the pixel belongs and at least one other motion vector computed for one other block of the image.

18. The apparatus of claim 15, wherein the motion vector module computes, for each of the plurality of blocks, at least one of horizontal projections and vertical projections for the block and computes, for each of the plurality of blocks, a motion vector based on the at least one of the horizontal and vertical projections for the block.

19. The apparatus of claim 15, further comprising an image sensor to capture the first image and the second image, wherein the image sensor captures at least one of the first image and the second image during an image preview.

20. The apparatus of claim 15, wherein the block partitioner partitions the first image into a plurality of overlapping blocks of pixels.

21. The apparatus of claim 15, wherein the apparatus is incorporated within a wireless communication device, the wireless communication device including:

an encoding module that encodes the averaged image; and

a transmitter that transmits the encoded image.

22. An apparatus for processing digital image data, the device comprising:

means for partitioning a first image of a scene of interest into a plurality of blocks of pixels;

means for computing, for each of the plurality of blocks, a motion vector that represents the offset between the block of the first image and a corresponding block of pixels within a second image of the scene of interest;

means for averaging pixel values for each of the pixels of the blocks of the first image with pixel values of corresponding pixels of the second image based on the motion vector of the respective block to which the pixels belong; and

means for storing the averaged pixel values.

23. The apparatus of claim 22, wherein:

the determining means determines, for each of the plurality of blocks, motion vectors for each of the pixels of the block using the motion vector computed for the block to which the pixel belongs and at least one other motion vector computed for one other block of the image, and

the averaging means averages the pixel values for each of the pixels of the blocks of the first image with pixel values of corresponding pixels of the second image at locations indicated by the respective motion vectors of the pixels.

24. The apparatus of claim 23, wherein the determining means determines motion vectors for each of the pixels using one of extrapolation and interpolation based on the motion vector computed for the block to which the pixel belongs and at least one other motion vector computed for one other block of the image.

25. The apparatus of claim 22, wherein:

the determining means determines, for each of the plurality of blocks, motion vectors for sub-blocks of pixels of the block using the motion vector computed for the block to which the sub-block belongs and at least one other motion vector computed for one other block of the image, and

the averaging means averages the pixel values for each of the sub-blocks of the first image with pixel values of corresponding sub-blocks of the second image at locations indicated by the respective motion vectors of the sub-blocks.