Method and apparatus for camera shake effect image stabilization

• US 9,179,060 B2
• Filed: 09/27/2007
• Issued: 11/03/2015
• Est. Priority Date: 09/27/2007
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Safaee-Rad, Reza, Aleksic, Milivoje
• Primary Examiner(s): FLOHRE, JASON A
• Application Number:
  US11/862,621
• Publication Number:
  US 20090086036A1
• Time in Patent Office: 2,959 Days
• Field of Search: 348/208.1, 348/208.12, 348/352, 348/354, 348/355
• US Class Current: 1/1
• CPC Class Codes: H04N 1/2145 of a sequence of images for selection of a single frame before final recording, e.g. from a continuous sequence captured before and after shutter-release
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/57 Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image
• CPC Class Codes: H04N 23/673 based on contrast or high frequency components of image signals, e.g. hill climbing method

Abstract

A method and apparatus for camera shake effect image stabilization determines a most favorable image sharpness metric out of image sharpness metrics from a plurality of images that were captured at a same lens position. A final image is selected based on the most favorable image sharpness metric.

Claims

1. A method for storing a stabilized image, comprising:

autofocusing a lens through a series of lens positions until a final lens position is determined;

determining image sharpness metric information for a set of images captured at the final lens position by calculating a sharpness metric value for each pixel in a plurality of regions of interest in the set of images, wherein each region of interest comprises a plurality of adjacent pixels, wherein each pixel comprises information captured from the lens;

selecting a final image frame from the set as the stabilized image based on the most favorable image sharpness metric information;

encoding the final image frame to create an encoded final image frame;

processing at least one image frame from the set for display before encoding the final image frame;

displaying the at least one image frame to a display; and

storing the encoded final image frame in a memory linked to a camera.

2. The method of claim 1, wherein selecting the final image frame includes:

comparing two image sharpness metric informations associated with two image frames in the set and determining the most favorable image sharpness metric information between the two image frames.

3. The method of claim 1, further comprising:

if the determining of the image sharpness metric information is performed on a first image frame of the set, then storing the image frame and associated image sharpness metric information as a most favorable image sharpness metric;

comparing a next image sharpness metric information of the set to the most favorable image sharpness metric information and determining a new most favorable image sharpness metric;

updating the most favorable image sharpness metric information and the next image sharpness metric; and

comparing each updated next sharpness metric information of the set to each updated most favorable image sharpness metric information until the set is empty.

4. The method of claim 1, further comprising capturing the set of images during a camera shake condition, and wherein a quantity of images in the set to be compared is determined by a user.

35. The method of claim 1, wherein the encoded final image frame corresponds to an image frame having a reduced camera shake effect relative to the set of images.

36. The method of claim 1, wherein the sharpness metric is a squared laplacian sharpness metric.

38. The method of claim 1, further comprising scanning pixel information across a horizontal dimension of an image in the set of images such that all regions of interest in a row are processed simultaneously.

39. The method of claim 1, wherein each region of interest comprises an unprocessed value for each of the plurality of adjacent pixels.

5. A method for selecting a stabilized image, comprising:

autofocusing a lens through a series of lens positions until a final lens position is determined, based at least in part, on auto-focus image statistics logic;

capturing a set of images during a camera shake condition at the final lens position;

dividing each image of the set into a plurality of regions of interest, wherein each region of interest comprises a plurality of adjacent pixels;

determine an image sharpness metric information for each image of the set by performing statistical analysis on each region of interest, based on at least in part, on the auto-focus image statistics logic, by calculating a sharpness metric value for each pixel in the plurality of regions of interest in the set of images, wherein each pixel comprises information captured from the lens; and

selecting a final image from the set as the stabilized image based on a most favorable image sharpness metric.

6. The method of claim 5, wherein selecting the final image includes:

comparing two image sharpness metrics associated with two images in the set and determining a most favorable image sharpness metric information between the two images; and

storing the most favorable image sharpness metric information and the associated image.

7. The method of claim 5, further comprising:

if the determining of the image sharpness metric information is performed on a first image of the set, then storing the image and associated image sharpness metric information as a most favorable image sharpness metric;

comparing a next image sharpness metric information of the set to the most favorable image sharpness metric information and determining a new most favorable image sharpness metric;

updating the most favorable image sharpness metric information and the next image sharpness metric;

comparing each updated next sharpness metric information of the set to each updated most favorable image sharpness metric information until the set is empty; and

storing the image associated with a final most favorable image sharpness metric information as a stabilized image.

8. The method of claim 5, wherein a quantity of images in the set to be compared is determined by a user.

9. A circuit in a camera for selecting a stabilized image, comprising:

autofocusing logic operative to autofocus a lens through a series of lens positions until a final lens position is determined;

image capturing logic operative to capture a set of images during a camera shake condition at the final lens position;

an image sharpness metric circuit operative to determine image sharpness metrics for each image frame of the set of images by calculating a sharpness metric value for each pixel in a plurality of regions of interest in the set of images, wherein each region of interest comprises a plurality of adjacent pixels, wherein each pixel comprises information captured from the lens;

memory operative to store the captured images and associated image sharpness metrics from the image sharpness metric circuit;

image stabilization logic operative to select a final image frame from the set as the stabilized image based on a most favorable image sharpness metric;

encoding logic operative to encode the final image frame as a final encoded image frame; and

processing logic operative to process at least one image frame from the set for display before the encoding logic encodes the final image frame, wherein the at least one image frame is displayed to a display.

10. The circuit in claim 9, wherein a first address in the memory is operative to store a new image sharpness metric information and a new associated image from the image sharpness metric circuit and a second address in the memory is operative to store a current most favorable image sharpness metric information and an associated image.

11. The circuit in claim 9, wherein the image stabilization logic, comprises:

comparator logic operative to compare the image sharpness metrics stored in the first and second addresses in memory; and

decision logic operative to output a signal indicating which address in memory contains the most favorable image sharpness metric.

12. The circuit in claim 9, wherein the image sharpness metric circuit, comprises a statistical block operative to analyze image pixel information of an image and generate a statistical image sharpness metric information from the image pixel information.

13. An apparatus in a camera for selecting a stabilized image comprising:

autofocusing logic operative to autofocus a lens through a series of lens positions until a final lens position is determined;

an image sharpness metric circuit operative to determine image sharpness metrics for each image frame of a set of images that were captured at the final lens position by calculating a sharpness metric value for each pixel in a plurality of regions of interest in the set of images, wherein each region of interest comprises a plurality of adjacent pixels, wherein each pixel comprises information captured from the lens;

memory operative to store the captured images and associated image sharpness metrics from the image sharpness metric circuit;

image stabilization logic operative to select a final image frame from the set as the stabilized image based on a most favorable image sharpness metric;

a display operative to display the stabilized image frame;

encoding logic operative to encode the final image frame as a final encoded image frame; and

processing logic operative to process at least one image frame from the set for display before the encoding logic encodes the final image frame, wherein the at least one image frame is displayed to a display.

14. The apparatus of claim 13, wherein the apparatus is a handheld device which communicates with a radio telephone subsystem.

15. The apparatus of claim 13, wherein:

a first address in the memory is operative to store a new image sharpness metric information and a new associated image from the image sharpness metric circuit and a second address in the memory is operative to store a current most favorable image sharpness metric information and an associated image.

16. The apparatus of claim 13, wherein the image stabilization logic includes:

comparator logic operative to compare the image sharpness metrics stored in the first and second addresses in memory; and

decision logic operative to output a signal indicating which address in memory contains the most favorable image sharpness metric.

17. The apparatus of claim 13, wherein the image sharpness metric circuit comprises a statistical block operative to analyze image pixel information of an image and generate a statistical image sharpness metric information from the image pixel information.

18. The apparatus of claim 13, further comprising image capturing logic operative to capture the set of images during a camera shake condition.

19. A mobile device having a camera, comprising:

means for autofocusing a lens through a series of lens positions until a final lens position is determined;

means for determining image sharpness metric information for each image frame of a set of images that were captured at the final lens position by calculating a sharpness metric value for each pixel in a plurality of regions of interest in the set of images, wherein each region of interest comprises a plurality of adjacent pixels, wherein each pixel comprises information captured from the lens;

means for selecting a final image frame from the set as a stabilized image based on most favorable image sharpness metric information;

means for encoding the final image frame to create an encoded final image;

means for processing at least one image frame from the set for display before encoding the final image frame;

means for displaying the at least one image frame to a display; and

means for storing the encoded final image frame in a memory linked to the camera.

20. The mobile device of claim 19, wherein the means for selecting the final image frame includes:

means for comparing two image sharpness metric informations associated with two images in the set and determining the most favorable image sharpness metric information between the two images.

21. The mobile device of claim 19, further comprising:

means for storing the image and associated image sharpness metric information as a most favorable image sharpness metric for a first image of the set;

means for comparing a next image sharpness metric information of the set to the most favorable image sharpness metric information and determining a new most favorable image sharpness metric;

means for updating the most favorable image sharpness metric information and the next image sharpness metric; and

means for comparing each updated next sharpness metric information of the set to each updated most favorable image sharpness metric information until the set is empty.

22. The mobile device of claim 19 further comprising means for capturing the set of images during a camera shake condition and means for determining from a user input a quantity of images in the set to be compared.

37. The mobile device of claim 19, wherein the mobile device comprises a mobile phone.

23. A mobile device, comprising:

means for autofocusing a lens through a series of lens positions until a final lens position is determined;

means for capturing a set of images during a camera shake condition at the final lens position;

means for dividing each image of the set into a plurality of regions of interest;

means for reusing auto-focus image statistics logic to determine an image sharpness metric information for each image of the set by performing statistical analysis on each region of interest by calculating a sharpness metric value for each pixel in each region of interest, wherein each pixel comprises information captured from the lens; and

means for selecting a final image from the set as a stabilized image based on a most favorable image sharpness metric.

24. The mobile device of claim 23, wherein the means for selecting the final image includes:

means for comparing two image sharpness metrics associated with two images in the set and determining a most favorable image sharpness metric information between the two images; and

means for storing the most favorable image sharpness metric information and the associated image.

25. The mobile device of claim 23, further comprising:

means for storing the image and associated image sharpness metric information as a most favorable image sharpness metric for a first image of the set;

means for comparing a next image sharpness metric information of the set to the most favorable image sharpness metric information and determining a new most favorable image sharpness metric;

means for updating the most favorable image sharpness metric information and the next image sharpness metric;

means for comparing each updated next sharpness metric information of the set to each updated most favorable image sharpness metric information until the set is empty; and

means for storing the image associated with a final most favorable image sharpness metric information as a stabilized image.

26. The mobile device of claim 23, further comprising means for determining from a user input a quantity of images in the set to be compared.

27. A non-transitory computer-readable media comprising instructions executable by a processor to perform a method in a camera for storing a stabilized image, the computer-readable media comprising:

instructions for autofocusing a lens through a series of lens positions until a final lens position is determined;

instructions for determining image sharpness metric information for each image of a set of images that were captured at the final lens position by calculating a sharpness metric value for each pixel in a plurality of regions of interest in the set of images, wherein each region of interest comprises a plurality of adjacent pixels, wherein each pixel comprises information captured from the lens;

instructions for selecting a final image frame from the set as the stabilized image based on most favorable image sharpness metric information;

instructions for encoding the final image frame to create an encoded final image;

instructions for processing at least one image frame from the set for display before encoding the final image frame;

instructions for displaying the at least one image frame to a display; and

instructions for storing the encoded final image frame in a memory linked to the camera.

28. The non-transitory computer-readable media of claim 27, wherein instructions for selecting the final image frame includes instructions for comparing two image sharpness metric informations associated with two images in the set and determining the most favorable image sharpness metric information between the two images.

29. The non-transitory computer-readable media of claim 27, further comprising:

instructions for storing the image frame and associated image sharpness metric information as a most favorable image sharpness metric for a first image frame of the set;

instructions for comparing a next image sharpness metric information of the set to the most favorable image sharpness metric information and determining a new most favorable image sharpness metric;

instructions for updating the most favorable image sharpness metric information and the next image sharpness metric; and

instructions for comparing each updated next sharpness metric information of the set to each updated most favorable image sharpness metric information until the set is empty.

30. The non-transitory computer-readable media of claim 27 further comprising instructions for capturing the set of images during a camera shake condition and instructions for determining from a user input a quantity of images in the set to be compared.

31. A non-transitory computer-readable media comprising instructions executable by a processor to perform a method for selecting a stabilized image, the computer-readable media comprising:

instructions for autofocusing a lens through a series of lens positions until a final lens position is determined;

instructions for capturing a set of images during a camera shake condition at the final lens position;

instructions for dividing each image of the set into regions of interest;

instructions for reusing auto-focus image statistics logic to determine an image sharpness metric information for each image of the set by performing statistical analysis on each region of interest and by calculating a sharpness metric value for each pixel in each region of interest, wherein each pixel comprises information captured from the lens; and

instructions for selecting a final image from the set as the stabilized image based on a most favorable image sharpness metric.

32. The non-transitory computer-readable media of claim 31, wherein instructions for selecting the final image comprises:

instructions for comparing two image sharpness metrics associated with two images in the set and determining a most favorable image sharpness metric information between the two images; and

instructions for storing the most favorable image sharpness metric information and the associated image.

33. The non-transitory computer-readable media of claim 31, further comprising:

instructions for storing the image and associated image sharpness metric information as a most favorable image sharpness metric for a first image of the set;

instructions for comparing a next image sharpness metric information of the set to the most favorable image sharpness metric information and determining a new most favorable image sharpness metric;

instructions for updating the most favorable image sharpness metric information and the next image sharpness metric;

instructions for comparing each updated next sharpness metric information of the set to each updated most favorable image sharpness metric information until the set is empty; and

instructions for storing the image associated with a final most favorable image sharpness metric information as a stabilized image.

34. The non-transitory computer-readable media of claim 31 further comprising instructions for determining from a user input a quantity of images in the set to be compared.