Method for providing image alignment feedback for panorama (composite) images in digital cameras using edge detection

• US 7,289,147 B2
• Filed: 02/03/2004
• Issued: 10/30/2007
• Est. Priority Date: 02/03/2004
• Status: Active Grant

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Webb, Christopher
• Primary Examiner(s): Vu; Ngoc-Yen
• Assistant Examiner(s): Peterson; Christopher K
• Application Number:
  US10/771,103
• Publication Number:
  US 20050179787A1
• Time in Patent Office: 1,365 Days
• Field of Search: 348 36- 39, 348/222.1, 348/252, 382/168, 382/172
• US Class Current: 348/252
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2622 Signal amplitude transition in the zone between image portions, e.g. soft edges
• CPC Class Codes: H04N 5/2625 for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect sequence generated by event triggered capturing H04N7/188

Abstract

Digital cameras and methods that provide improved image alignment for panorama image capture. The present invention identifies features that will likely be used by a stitching algorithm, and highlights those features for a user by superimposing them on top of a “liveview” image presented on a display of the digital camera. By applying an edge detection algorithm to the previously captured image, the user can select and display those features of the previous image that are relevant to aligning the camera for the subsequent image.

Claims

1. A digital camera system comprising:

a lens;

an image sensor for sensing an image viewed by the lens;

a display for displaying the image sensed by the image sensor;

a volatile storage device for storing the image sensed by the image sensor;

processing circuitry coupled to the display, image sensor, and nonvolatile storage device; and

a processing algorithm that runs on the processing circuitry that implements an edge detection filter that;

captures an image that is to be used as part of a panorama image sequence;

processes the captured image using the edge detection filter;

applies the edge detection filter to a portion of the captured image that will overlap with a subsequent image of the panorama sequence;

computes a histogram of resultant values derived from the edge detection filter to identify a threshold value that is related to a number of edges of the processed image;

alters the threshold value such that the number of displayed edges will be decreased;

extracts image pixels near the edges of the captured image; and

superimposes the extracted pixels over the liveview image of the subsequent image.

2. The system recited in claim 1 wherein the processing algorithm further reduces the percentage of pixels that are identified as edges.

3. The system recited in claim 1 wherein the processing algorithm further reduces the radius around those edges that are sampled to display on the liveview image.

4. The system recited in claim 1 wherein the processing algorithm alters the threshold value through reduction of the threshold value such that the number of displayed edges will be decreased.

5. The system recited in claim 1 wherein the processing algorithm promotes a better composition of the image through alteration of the threshold value to decrease the number of displayed edges.

6. The system recited in claim 1 wherein the processing algorithm decreases clutter obscuring the liveview image through alteration of the threshold value to decrease the number of displayed edges.

7. The system recited in claim 1 wherein the processing algorithm alters a level of obscuration of the liveview image through alteration of the radius around those edges that are sampled to display on the liveview image.

8. A method for use in a camera having a display for displaying a liveview image, a lens, an image sensor, a nonvolatile storage device, a volatile storage device, and processing circuitry, the method comprising the steps of:

capturing an image that is to be used as part of a panorama image sequence;

processing the captured image using an edge detection filter;

applying the edge detection filter to a portion of the captured image that will overlap with a subsequent image of the panorama sequence;

computing a histogram of resultant values derived from the edge detection filter to identify a threshold value that is related to a number of edges of the processed image;

altering the threshold value such that the number of displayed edges will be decreased;

extracting image pixels near the edges of the captured image; and

superimposing the extracted pixels over the liveview image.

9. The method recited in claim 8 further comprising the step of reducing the percentage of pixels that are identified as edges.

10. The method recited in claim 8 further comprising the step of reducing the radius around those edges that are sampled to display on the liveview image of the subsequent image.

11. The method recited in claim 8 wherein the step of altering the threshold value such that the number of displayed edges will be decreased comprises the step of reducing the threshold value such that the number of displayed edges will be decreased.

12. The method recited in claim 8 wherein the step of altering the threshold value such that the number of displayed edges will be decreased comprises the step of promoting a better composition of the image.

13. The method recited in claim 8 wherein the step of altering the threshold value such that the number of displayed edges will be decreased comprises the step of decreasing clutter obscuring the liveview image.

14. The method recited in claim 8 further comprising the step of altering a level of obscuration of the liveview image through alteration of the radius around those edges that are sampled to display on the liveview image.

15. A digital camera system comprising:

a lens;

an image sensor for sensing an image viewed by the lens;

a display for displaying the image sensed by the image sensor;

a volatile storage device for storing the image sensed by the image sensor, processing circuitry coupled to the display, image sensor, and nonvolatile storage device; and

a processing algorithm that runs on the processing circuitry that implements an edge detection filter that;

captures an image that is to be used as part of a panorama image sequence;

processes the captured image using the edge detection filter;

applies the edge detection filter to a portion of the captured image that will overlap with a subsequent image of the panorama sequence;

computes a histogram of resultant values derived from the edge detection filter to identify a threshold value that is related to a number of edges of the processed image;

alters the threshold value such that the number of displayed edges will be increased;

extracts image pixels near the edges of the captured image; and

superimposes the extracted pixels over the liveview image of the subsequent image.

16. The system recited in claim 15 wherein the processing algorithm alters the threshold value through increase of the threshold value such that the number of displayed edges will be increased.

17. The system recited in claim 15 wherein the processing algorithm increases a number of alignment features through increase of the number of displayed edges.

18. A method for use in a camera having a display for displaying a liveview image, a lens, an image sensor, a nonvolatile storage device, a volatile storage device, and processing circuitry, the method comprising the steps of:

capturing an image that is to be used as part of a panorama image sequence;

processing the captured image using an edge detection filter;

applying the edge detection filter to a portion of the captured image that will overlap with a subsequent image of the panorama sequence;

computing a histogram of resultant values derived from the edge detection filter to identify a threshold value that is related to a number of edges of the processed image;

altering the threshold value such that the number of displayed edges will be increased;

extracting image pixels near the edges of the captured image; and

superimposing the extracted pixels over the liveview image.

19. The method recited in claim 18 wherein the step of altering the threshold value such that the number of displayed edges will be increased comprises the step of increasing the threshold value such that the number of displayed edges will be increased.

20. The method recited in claim 18 wherein the step of altering the threshold value such that the number of displayed edges will be increased comprises the step of increasing a number of alignment features through increase of the number of displayed edges.