SYSTEM AND METHOD TO SELECTIVELY COMBINE IMAGES

• US 20100157079A1
• Filed: 12/19/2008
• Published: 06/24/2010
• Est. Priority Date: 12/19/2008
• Status: Abandoned Application

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Li, Hsiang-Tsun, Hwang, Hau, Atanassov, Kalin M.
• Application Number:
  US12/340,522
• Publication Number:
  US 20100157079A1
• Time in Patent Office: Days
• US Class Current: 348/222.100
• CPC Class Codes: G06T 2207/10144 Varying exposure
• CPC Class Codes: G06T 2207/10148 Varying focus
• CPC Class Codes: G06T 2207/10152 Varying illumination
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 3/14 Transformations for image registration, e.g. adjusting or mapping for alignment of images
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/683 performed by a processor, e.g. controlling the readout of an image memory
• CPC Class Codes: H04N 23/951 by using two or more images to influence resolution, frame rate or aspect ratio

Abstract

Systems and methods to selectively combine images are disclosed. In a particular embodiment, an apparatus includes a registration circuit configured to generate a set of motion vector data based on first image data corresponding to a first image and second image data corresponding to a second image. The apparatus includes a combination circuit to selectively combine the first image data and adjusted second image data that corresponds to the second image data adjusted according to the motion vector data. The apparatus further includes a control circuit to control the combination circuit to generate third image data.

Claims

1. A method comprising:

receiving first image data corresponding to a first image and second image data corresponding to a second image;

adjusting the second image data by applying a set of motion vector data corresponding to offsets between portions of the second image data with respect to corresponding portions of the first image data to produce adjusted second image data; and

generating third image data by selectively combining first values from the first image data and second values from the adjusted second image data at least partially based on comparing a first characteristic of the first image data to a second characteristic of the second image data.

2. The method of claim 1, wherein the selective combining includes a coarse combining operation and a fine combining operation.

3. The method of claim 2, wherein the coarse combining operation is performed on macroblocks of the adjusted second image data and wherein the fine combining operation is performed on pixels of the first image data and of the second adjusted image data.

4. The method of claim 2, wherein the coarse combining operation includes selectively discarding one or more macroblocks of the adjusted second image data when a difference between the one or more macroblocks of the adjusted second image data and corresponding macroblocks of the first image data exceeds a selectable threshold value.

5. The method of claim 2, wherein the fine combining operation includes filtering the first image data and the adjusted second image data to compare filtered image characteristics, and wherein the first values and the second values are selectively combined based on the filtered image characteristics.

6. The method of claim 1, wherein the first characteristic and the second characteristic are indicative of a focus condition, a movement of an object within the first image and the second image, or an exposure.

7. The method of claim 1, wherein the first values and the second values are selectively combined at least partially based on comparing a filtered portion of the first image data to a filtered portion of the second image data.

8. The method of claim 1, wherein the first image data and the second image data include successive frames captured at an image sensor of a camera, wherein the first image data includes a first set of macroblocks corresponding to the first image, wherein the second image data includes a second set of macroblocks corresponding to the second image, and wherein adjusting the second image data includes applying the set of motion vector data to the second set of macroblocks.

9. The method of claim 1, wherein the set of motion vector data is determined via a hierarchical registration process.

10. The method of claim 9, wherein the hierarchical registration process includes:

determining a first alignment offset between a first portion of the first image and a first portion of the second image; and

determining a second alignment offset between a second portion of the first image and a second portion of the second image based on the first alignment offset, wherein the second portion of the first image is within the first portion of the first image.

11. The method of claim 1, wherein pixels within macroblocks of the adjusted second image data are discarded when an image is adjusted to compensate for hand jitter reduction.

12. The method of claim 1, wherein the third image data adjusts the first image data and the second image data to compensate for lateral chromatic aberrations.

13. A method comprising:

determining a first set of motion vectors corresponding to an offset between each block of a first set of blocks of first image data and a corresponding block of a first set of blocks of second image data;

upsampling a motion vector of the first set of motion vectors corresponding to a particular block of the first set of blocks of the second image data to apply the motion vector to a second set of blocks of the second image data, the second set of blocks of the second image data included within the particular block;

determining a second set of motion vectors corresponding to an offset between each of a second set of blocks of the first image data and a corresponding block of the second set of blocks of the second image data after applying the motion vector to the second set of blocks;

applying the second set of motion vectors to the second image data to generate adjusted second image data; and

selectively combining the first image data and the adjusted second image data to produce third image data.

14. The method of claim 13, wherein the third image data has at least one of a greater depth of field than the first image data, less noise than the first image data, or a greater dynamic range than the first image data.

15. The method of claim 13, wherein the second set of motion vectors is determined based on a selectable search range that is indicated via a control input.

16. The method of claim 15, wherein the selectable search range limits the offset between each of the second set of blocks of the first image data and the corresponding block of the second set of blocks of the second image data after applying the motion vector to the second set of blocks.

17. An apparatus comprising:

a registration circuit configured to generate a set of motion vector data based on first image data corresponding to a first image and second image data corresponding to a second image;

a combination circuit coupled to selectively combine the first image data and adjusted second image data, wherein the adjusted second image data corresponds to the second image data after being adjusted according to the motion vector data; and

a control circuit to control the combination circuit to generate third image data.

18. The apparatus of claim 17, wherein the registration circuit includes a coarse registration circuit and a fine registration circuit.

19. The apparatus of claim 17, wherein the combination circuit includes a coarse combination circuit and a fine combination circuit.

20. The apparatus of claim 17, wherein the registration circuit comprises:

a motion vector generation circuit having a first input coupled to accept the first image data, a second input coupled to accept the second image data, and a motion vector generation output;

a motion vector upsampling circuit having an input coupled to accept the motion vector generation output and providing a motion vector upsampling output; and

a macro block motion vector refining circuit having a first input coupled to accept the first image data, a second input coupled to accept the second image data, a third input coupled to accept the motion vector upsampling output, and having a macro block motion vector output and a macro block difference output.

21. The apparatus of claim 17, wherein the combination circuit comprises:

a macro block image alignment circuit having a first input coupled to receive the first image data and a second input coupled to receive the second image data, a third input coupled to receive a macro block motion vector signal, an aligned first image output and an aligned second image output;

a block difference discriminator circuit having a first input coupled to receive the aligned first image output, a second input coupled to receive the aligned second image output, a third input coupled to receive a macro block difference signal, and having a first image difference output and a second image difference output; and

a mean pixel difference circuit having a first input coupled to the block difference discriminator circuit and configured to provide a pixel difference output signal to indicate a mean pixel difference result.

22. The apparatus of claim 21, further comprising:

a first filter coupled between the block difference discriminator circuit and the mean pixel difference circuit having a first input coupled to receive the first image difference output and an output providing a filtered first image difference output; and

a second filter coupled between the block difference discriminator circuit and the mean pixel difference circuit having a first input coupled to receive the second image difference output and having an output to provide a filtered second image difference output.

23. The apparatus of claim 17, further comprising:

a camera lens; and

circuitry coupled to the camera lens to generate the first image data and to generate the second image data.

24. The apparatus of claim 23, further comprising a processor coupled to a wireless transceiver to send and receive data via an antenna.

25. The apparatus of claim 23, wherein the apparatus is a personal digital assistant (“

PDA”

).

26. An apparatus comprising:

a registration means for generating a set of motion vector data based on first image data corresponding to a first image and second image data corresponding to a second image;

a combination means for selectively combining the first image data and adjusted second image data, wherein the adjusted second image data corresponds to the second image data after being adjusted according to the motion vector data; and

a control means for controlling the combination means to generate third image data.

27. The apparatus of claim 26, further comprising:

a camera lens; and

circuitry coupled to the camera lens to generate the first image data and the second image data.