Photographing device and method for obtaining photographic image having image vibration correction

• US 20040239775A1
• Filed: 04/26/2004
• Published: 12/02/2004
• Est. Priority Date: 05/30/2003
• Status: Active Grant

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Washisu, Koichi
• Granted Patent: US 7,468,743 B2
• Time in Patent Office: Days
• US Class Current: 348/239
• CPC Class Codes: H04N 23/68 for stable pick-up of the scene, e.g. compensating for camera body vibrations
• CPC Class Codes: H04N 23/6811 based on the image signal
• CPC Class Codes: H04N 23/684 performed by controlling the image sensor readout, e.g. by controlling the integration time
• CPC Class Codes: H04N 23/6842 by controlling the scanning position, e.g. windowing
• CPC Class Codes: H04N 23/6845 by combination of a plurality of images sequentially taken
• CPC Class Codes: H04N 23/70 Circuitry for compensating brightness variation in the scene
• CPC Class Codes: H04N 23/74 by influencing the scene brightness using illuminating means
• CPC Class Codes: H04N 23/741 by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors

Abstract

A photographing device comprises a detecting unit for extracting at least one characteristic point in each image of multiple images and detecting movement of the characteristic point on another image as to a characteristic point on a reference image; a coordinate converting unit for performing coordinates conversion of the other image so as to match the characteristic point on the other image with the characteristic point on the reference image based on the detected results of the detecting unit; a synthesizing unit for synthesizing the reference image and the other image subjected to coordinates conversion by the coordinate converting unit; a determining unit for determining whether or not the characteristic points from the multiple images acquired by the image sensor can be extracted by the detecting unit; and a warning unit for warning users when the determining unit cannot extract characteristic points. A user-friendly photographing device can thus be provided.

Claims

1. A photographing device having an image sensor for performing photoelectric conversion of a subject for synthesizing multiple images sequentially acquired by output signals from said image sensor so as to obtain a photography image, said photographing device comprising:

a detecting unit for extracting at least one characteristic point in each image of said multiple images and detecting movement of the characteristic point on another image as to a characteristic point on a reference image;

a coordinate converting unit for performing coordinates conversion of said other image so as to match the characteristic point on said other image with the characteristic point on said reference image based on the detected results of said detecting unit;

a synthesizing unit for synthesizing said reference image and said other image subjected to coordinates conversion by said coordinate converting unit;

a determining unit for determining whether or not at least one characteristic point from one of the images acquired by said image sensor prior to photographing can be extracted by said detecting unit; and

a warning unit for warning a user in the event that said determining unit cannot extract a characteristic point.

2. A photographing device having an image sensor for performing photoelectric conversion of a subject for synthesizing multiple images sequentially acquired by output signals from said image sensor so as to obtain a photography image, said photographing device comprising:

a detecting unit for extracting at least one characteristic point in each image of said multiple images and detecting movement of the characteristic point on another image as to a characteristic point on a reference image;

a coordinate converting unit for performing coordinates conversion of said other image so as to match the characteristic point on said other image with the characteristic point on said reference image based on the detected results of said detecting unit;

a synthesizing unit for synthesizing said reference image and said other image subjected to coordinates conversion by said coordinate converting unit;

a determining unit for determining whether or not movement of the characteristic point from the multiple images acquired by said image sensor prior to photographing can be detected by said detecting unit; and

a warning unit for warning a user in the event that said determining unit cannot detect movement of the characteristic points.

3. A photographing device having an image sensor for performing photoelectric conversion of a subject for synthesizing multiple images sequentially acquired by output signals from said image sensor so as to obtain a photography image, said photographing device comprising:

a detecting unit for extracting at least one characteristic point in each image of said multiple images and detecting movement of the characteristic point on another image as to a characteristic point on a reference image;

a coordinate converting unit for performing coordinates conversion of said other image so as to match the characteristic point on said other image with the characteristic point on said reference image based on the detected results of said detecting unit;

a synthesizing unit for synthesizing said reference image and said other image subjected to coordinates conversion by said coordinate converting unit; and

a control unit for determining whether or not characteristic points from multiple images acquired by said image sensor can be extracted by said detecting unit, and prohibiting an image from being obtained by synthesizing the multiple images in the event that characteristic points cannot be extracted.

4. A photographing device according to claim 3;

wherein a first photographing mode for obtaining a photography image by synthesizing the sequentially obtained multiple images, and a second photographing mode for obtaining a photography image by acquiring a single image can be set; and

wherein said control unit sets said second photographing mode in the event that characteristic points cannot be extracted.

5. A photographing device according to claim 4;

wherein said control unit sets an exposure time of an image in said second photographing mode longer than each exposure time of the plurality of images in said first photographing mode at the time of said second photographing mode being set.

6. A photographing device having an image sensor for performing photoelectric conversion of a subject for synthesizing multiple images sequentially acquired by output signals from said image sensor so as to obtain a photography image, said photographing device comprising:

a detecting unit for extracting at least one characteristic point in each image of said multiple images and detecting movement of the characteristic point on another image as to a characteristic point on a reference image;

a coordinate converting unit for performing coordinates conversion of said other image so as to match the characteristic point on said other image with the characteristic point on said reference image based on the detected results of said detecting unit;

a synthesizing unit for synthesizing said reference image and said other image subjected to coordinates conversion by said coordinate converting unit; and

a control unit for determining whether or not movement of the characteristic points from multiple images acquired by said image sensor can be detected by said detecting unit, and prohibiting an image from being obtained by synthesizing the multiple images in the event that movement of the characteristic points cannot be detected.

7. A photographing device according to claim 6;

wherein a first photographing mode for obtaining a photography image by synthesizing the sequentially obtained multiple images, and a second photographing mode for obtaining a photography image by acquiring a single image can be set; and

wherein said control unit sets said second photographing mode in the event that movement of the characteristic points cannot be detected.

8. A photographing device according to claim 7, wherein said control unit sets an exposure time of an image in said second photographing mode longer than each exposure time of the plurality of images in said first photographing mode at the time of said second photographing mode being set.

9. A photographing device having an image sensor for performing photoelectric conversion of a subject for synthesizing multiple images sequentially acquired by output signals from said image sensor so as to obtain a photography image, said photographing device comprising:

an extracting unit for extracting at least one characteristic point in at least one predetermined region on each image of said multiple images;

a detecting unit for detecting movement of the characteristic point on another image of said multiple images as to a characteristic point on a reference image;

a coordinate converting unit for performing coordinates conversion of said other image so as to match the characteristic point on said other image with the characteristic point on said reference image based on the detected results of said detecting unit;

a synthesizing unit for synthesizing said reference image and said other image subjected to coordinates conversion by said coordinate converting unit; and

a control unit for changing said predetermined region in accordance with a distance up to a subject.

10. A photographing device according to claim 9, wherein said control unit determines whether or not the distance up to the subject is the predetermined distance or shorter, and a first region positioned at a central portion on the image is selected as a region to extract at least one characteristic point in the event that the distance up to the subject is the predetermined distance or shorter, on the other hand, a second region other than said first region in the event that the distance up to the subject is longer than the predetermined distance.

11. A photographing device according to claim 9 including multiple focusing areas within a subject observing screen;

wherein said control unit determines whether or not the distance up to the subject is the predetermined distance or shorter, and a first region positioned including a focusing area corresponding to a primary subject of said multiple focusing areas is selected as a region to extract at least one characteristic point in the event that the distance up to the subject is the predetermined distance or shorter, on the other hand, a second region other than said first region in the event that the distance up to the subject is longer than the predetermined distance.

12. A photographing device having an image sensor for performing photoelectric conversion of a subject capable of setting a first photographing mode for obtaining for synthesizing multiple images sequentially acquired by output signals from said image sensor so as to obtain a photography image, and a second photographing mode for acquiring single image so as to obtain a photography image, said photographing device comprising:

a detecting unit for extracting at least one characteristic point in each image of said multiple images and detecting movement of the characteristic point on another image as to a characteristic point on a reference image;

a coordinate converting unit for performing coordinates conversion of said other image so as to match the characteristic point on said other image with the characteristic point on said reference image based on the detected results of said detecting unit;

a synthesizing unit for synthesizing said reference image and said other image subjected to coordinates conversion by said coordinate converting unit;

a light-measuring unit for measuring light with emphasis on luminance of the background region of an image; and

a selecting unit for selecting any one photographing mode of said first photographing mode and said second photographing mode based on the photometry results by said light-measuring unit.

13. A photographing device according to claim 12, further comprising an information output unit for outputting information regarding the selected photographing mode by said selecting unit.

14. A method for synthesizing multiple images sequentially acquired by output signals from an image sensor for performing photoelectric conversion of a subject;

wherein at least one characteristic point in each image of said multiple images is extracted, and movement of the characteristic point on another image of said multiple images as to a characteristic point on a reference image is detected;

coordinates conversion of said other image is performed so as to match the characteristic point on said other image with the characteristic point on said reference image based on the detected results;

said reference image and said other image subjected to coordinates conversion are synthesized;

a determination is made regarding whether or not the characteristic points can be extracted from the images acquired by said image sensor prior to photographing, or whether or not movement of the characteristic points can be detected; and

warning of a user is performed in the event that this determination is made that the characteristic points cannot be extracted, or the movement of the characteristic points is not detected.

15. A method for synthesizing multiple images sequentially acquired by output signals from an image sensor for performing photoelectric conversion of a subject;

wherein at least one characteristic point in each image of said multiple images is extracted, and movement of the characteristic point on another image of said multiple images as to a characteristic point on a reference image is detected;

coordinates conversion of said other image is performed so as to match the characteristic point on said other image with the characteristic point on said reference image based on the detected results;

said reference image and said other image subjected to coordinates conversion are synthesized;

a determination is made regarding whether or not the characteristic points can be extracted from the multiple images acquired by said image sensor prior to photographing, or whether or not movement of the characteristic points can be detected; and

synthesizing the multiple images so as to obtain a photography image is prohibited in the event that this determination is made that the characteristic points cannot be extracted, or the movement of the characteristic points is not detected.

16. A method according to claim 15, wherein a first photographing mode for obtaining a photography image by synthesizing the sequentially obtained multiple images, and a second photographing mode for obtaining a photography image by acquiring a single image can be set, and said second photographing mode is set in the event that the characteristic points cannot be extracted or in the event that movement of the characteristic points cannot be detected.

17. A method for synthesizing multiple images sequentially acquired by output signals from an image sensor for performing photoelectric conversion of a subject;

wherein at least one characteristic point in each image of said multiple images is extracted, and movement of the characteristic point on another image of said multiple images as to a characteristic point on a reference image is detected;

coordinates conversion of said other image is performed so as to match the characteristic point on said other image with the characteristic point on said reference image based on the detected results;

said reference image and said other image subjected to coordinates conversion are synthesized; and

said predetermined region is changed in accordance with a distance up to the subject.

18. A method for synthesizing multiple images sequentially acquired by output signals from an image sensor for performing photoelectric conversion of a subject, wherein at least one characteristic point in each image of said multiple images is extracted, and movement of the characteristic point on another image of said multiple images as to a characteristic point on a reference image is detected;

coordinates conversion of said other image is performed so as to match the characteristic point on said other image with the characteristic point on said reference image based on the detected results;

said reference image and said other image subjected to coordinates conversion are synthesized;

light is measured with emphasis on luminance of the background region of an image; and

any one photographing mode of a first photographing mode for synthesizing the multiple images sequentially acquired by output signals from said image sensor so as to obtain a photography image and a second photographing mode for acquiring a single image so as to obtain a photography image is selected based on the photometry results.