Panoramic image generation in digital photography

• US 6,243,103 B1
• Filed: 05/28/1997
• Issued: 06/05/2001
• Est. Priority Date: 05/28/1996
• Status: Expired due to Term

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Hatori, Kenji, Yano, Kotaro, Katayama, Tatsushi, Takiguchi, Hideo
• Primary Examiner(s): Zimmerman, Mark
• Assistant Examiner(s): Vo, Cliff N.
• Application Number:
  US08/864,480
• Time in Patent Office: 1,469 Days
• Field of Search: 345/435, 345/418, 345/433, 345/434, 345/439, 382/284
• US Class Current: 345/634
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2624 for obtaining an image which is composed of whole input images, e.g. splitscreen

Abstract

An image synthesizing system comprising a digital camera and a personal computer. When a first image is picked up by the camera, the personal computer takes the image sensed by the camera and a reduced image of the picked-up image. While the camera is moving, the camera sequentially sends reduced images to the personal computer, and the personal computer inputs the reduced images. The personal computer determines whether an image which is currently sensed by the camera is suitable for being synthesized into a panoramic image or not on the basis of an overlapping area between the first reduced image and a current reduced image, and a determined result is notified to a user.

Claims

1. An image sensing apparatus comprising:

image pick-up means for sensing an object and obtaining image signals;

signal processing means for outputting image signals representing first and second picked-up images of the object sensed at different image view positions in response to a request signal from an image synthesizing device, and confirmation image signals obtained by reducing the first and second picked-up images, to the image synthesizing device;

determination means for automatically determining an overlapping state of the confirmation image signals; and

displaying means for simultaneously displaying, on a monitor screen, the confirmation image signals corresponding to the first and second images in such a manner that the overlapping state of the first and second images is expressed and an image generated by synthesizing the first and second picked-up images,

wherein the second picked-up image is updated at a predetermined interval.

2. An image synthesizing apparatus comprising:

reception means for receiving an image signal representing a picked-up image from an image pick-up device and receiving a confirmation image signal representing a reduced image of the picked-up image;

determination means for automatically determining relative positions between first and second confirmation images received by said reception means;

synthesizing means for synthesizing first and second images received corresponding to the first and second confirmation images, respectively, on the basis of the relative positions determined by said determination means; and

displaying means for simultaneously displaying the first and second confirmation images and a synthesized image of the first and second images in such a manner that the relative positions of the first and second confirmation images are expressed and a synthesized image of the first and second images,

wherein the second confirmation image is updated at a predetermined interval.

3. The image synthesizing apparatus according to claim 2, wherein said determination means determines the relative positions by determining a first overlapping area between the first and second confirmation images.

4. The image synthesizing apparatus according to claim 3, wherein said synthesizing means comprises:

means for detecting a second overlapping area between the first and second images on the basis of the relative positions; and

means for combining the first and second images on the basis of the detected second overlapping area.

5. The image synthesizing apparatus according to claim 3 further comprising judging means for judging whether or not the first overlapping area is appropriate.

6. The image synthesizing apparatus according to claim 4 further comprising judging means for judging whether or not the second overlapping area is appropriate.

7. The image synthesizing apparatus according to claim 6 further comprising notification means for notifying a user of a judged result by said judging means.

8. The image synthesizing apparatus according to claim 6 further comprising notification means for notifying a user of a judged result by said judging means.

9. The image synthesizing apparatus according to claim 4, wherein said synthesizing means comprises:

means for roughly obtaining the second overlapping area by magnifying the first overlapping area;

means for extracting pixels which have an edge property as corresponding points in the roughly obtained second overlapping area; and

means for obtaining mapping relationship between the first and second images on the basis of a set of extracted corresponding points,

wherein said synthesizing means combines the first and second images by transforming either the first image or the second image on the basis of the mapping relationship.

10. The image synthesizing apparatus according to claim 3, wherein said determination means for determining the first overlapping area comprises:

means for identifying a first predetermined number of pixels in the first confirmation image;

means for identifying a second predetermined number of pixels, corresponding to the first predetermined number, in the second confirmation image; and

means for determining position relationship between the first and second confirmation images by comparing pixel values of the first predetermined number of pixels and of the second predetermined number of pixels.

11. The image synthesizing apparatus according to claim 10 further comprising:

means for calculating an overlapping amount of overlapping pixels between the first and second confirmation images on the basis of the determined position relationship of the first and second reduced images; and

means for judging whether the ratio of the overlapping amount to an area of either the first confirmation image or the second confirmation image is less than a first threshold.

12. The image synthesizing apparatus according to claim 10 further comprising notification means for notifying that the ratio is less than the first threshold to a user as a timing for picking up an image to be synthesized.

13. The image synthesizing apparatus according to claim 12, wherein, when the overlapping amount is less than a second threshold which is smaller than the first threshold, said notification means notifies the user that it is inappropriate timing for picking up an image to be synthesized.

14. An image synthesizing system which synthesizes two images comprising:

an image rearing apparatus having;

image pick-up means for sensing an object and obtaining an image; and

signal processing means for generating and outputting a reduced image obtained by reducing the image in response to a request signal, and outputting the original image,

an image synthesizing apparatus having;

reception means for receiving an image signal representing a reduced image of the picked-up image and receiving an image signal representing the picked-up image from the image sensing apparatus;

determination means for automatically determining and storing relative positions between first and second reduced images received by said reception means; and

synthesizing means for synthesizing first and second images, corresponding to the first and second reduced images, received by said reception means on the basis of the relative positions determined by and stored in said determination means; and

displaying means for simultaneously displaying the first and second confirmation images and a synthesized image of the first and second images in such a manner that the relative positions between the first and second reduced images are expressed and a synthesized image of the first and second images,

wherein the second reduced image is updated at a predetermined interval.

15. The image synthesizing system according to claim 14, wherein the image sensing apparatus and the image synthesizing apparatus are connected via a general-use interface capable of transmitting an image at a high speed.

16. The image synthesizing system according to claim 15, wherein said general-use interface is a bidirectional parallel interface or a SCSI interface.

17. An image synthesizing apparatus comprising:

reception means for receiving an image signal of a picked-up image from an image pick-up device;

generation means for generating first and second reduced images on the basis of first and second images received by said reception means;

determination means for determining relative positions between the first and second reduced images on the basis of an overlapping region between the first and second images;

displaying means for simultaneously displaying, on a monitor screen, the first and second reduced images in such a manner that the relative positions between the first and second reduced images are expressed on the basis of the relative positions determined by said determination means; and

synthesizing means for, in response to the displaying means, synthesizing the first and second images on the basis of the relative positions determined by said determination means,

wherein said second image is updated at a predetermined interval.

18. An image synthesizing apparatus comprising:

generation means for generating a reduced image of an image sensed by an image pick-up device;

determination means for automatically determining relative positions between first and second reduced images generated by said generation means on the basis of an overlapping region between the first and second reduced images;

judging means for judging whether an overlap condition for determining if it is appropriate to synthesize first and second images which are original images of the first and second reduced images is satisfied or not on the basis of the relative positions between the first and second reduced images determined by said determination means; and

notification means for notifying a user of a judged result.