Picture image forming apparatus

• US 6,005,987 A
• Filed: 10/16/1997
• Issued: 12/21/1999
• Est. Priority Date: 10/17/1996
• Status: Expired due to Fees

• Current Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Sharp Kabushiki Kaisha (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Akagi, Hiroshi, Nakamura, Mitsuaki, Kitamura, Yoshihiro
• Primary Examiner(s): Boudreau, Leo H.
• Assistant Examiner(s): PATEL, KANJIBHAI B
• Application Number:
  US08/951,713
• Time in Patent Office: 796 Days
• Field of Search: 382/294, 382/154, 382/300, 382/178, 382/284, 382/285, 382/282, 382/293, 382/296, 345/114, 345/419, 348/42, 348/50, 348/49, 348/185, 348/189, 348/190, 348/222, 348/399, 348/565
• US Class Current: 382/294
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 13/167 Synchronising or controlling image signals
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/296 Synchronisation thereof; Control thereof
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 2013/0088 Synthesising a monoscopic image signal from stereoscopic images, e.g. synthesising a panoramic or high resolution monoscopic image
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

In forming a panoramic picture image by compositing picture images dividedly imaged, the invention corrects deviations of images of an object at overlap regions caused by a difference in fields of view of imaging means or movement of an object to be imaged in which a picture image forming apparatus includes a picture image inputting device, a frame memory for storing divided picture images inputted from the picture image inputting device, a picture image position calculating unit for calculating positions of compositing the divided picture images, a picture image parallax sampling and processing unit for sampling parallax information among picture images from picture images including parallaxes in the overlap regions among contiguous divided picture images, an intermediate picture image forming and processing unit for forming a plurality of intermediate picture images from the picture images including the parallaxes and a picture image composition and processing unit for compositing a panoramic picture image from the divided picture images where the deviations of images in the overlap regions are corrected by the picture image parallax sampling and processing unit and the panoramic picture images are composited performing an interpolation by the intermediate picture images formed by the intermediate picture image forming and processing unit.

Claims

1. A picture image forming apparatus comprising:

imaging means for obtain a pair of divided picture images having a parallax by imaging an object and a periphery of the object so as to include a same part of the object in imaging regions for the pair of divided picture images;

calculating means for calculating overlap regions of the respective divided picture images including the images of the object,

parallax sampling means for sampling the parallax from the images of the object in the two overlap regions,

intermediate picture image forming means for forming intermediate picture images to be provided by imaging the object from points between observing points from which the respective divided picture images are imaged on the basis of the images of the object in the overlap regions of the pair of divided picture images and the sampled parallax, and

picture image composition means for providing a composite picture image by interpolating the parallax of the images of the object in the two overlap regions by the intermediate picture images and compositing the pair of divided picture images such that the overlap regions overlap.

2. The picture image forming apparatus of claim 1, wherein, when there are a plurality of objects having different distances from the imaging means to the objects, the parallax sampling means corrects parallaxes of images of the respective objects such that the parallax of the image of any one of the objects among the parallaxes sampled from the images of the respective objects, is nullified.

3. The picture image forming apparatus of claim 1, wherein the intermediate picture image forming means of the picture image forming apparatus forms a plurality of the intermediate picture images and the respective intermediate picture images are provided with observing points different from each other to provide the respective intermediate picture images by imaging the object.

4. The picture image forming apparatus of claim 1, wherein the calculating means of the picture image forming apparatus calculates differences in angle of the images of the object in the two divided picture images and subjects the divided picture images to a rotational transformation such that the calculated differences in angle cancel each other.

5. The picture image forming apparatus of claim 1, wherein the calculating means of the picture image forming apparatus calculates differences in size of the images of the object in the divided picture images and the two divided picture images are magnified or reduced such that the calculated differences in size cancel each other.

6. The picture image forming apparatus of claim 1, wherein the calculating means calculates differences in angle and size of the images of the object in the two divided picture images and subjects the divided picture images to rotational transformation and magnification or reduction such that the calculated differences in angle and size cancel each other, respectively.

7. A picture image forming apparatus comprising:

imaging means for providing a pair of divided picture images having parallaxes in images of objects by imaging the same objects respectively from two different observing points;

calculating means for calculating overlap regions including the images of the objects in the respective divided picture images;

parallax sampling means for sampling the parallaxes from the images of the objects in the overlap regions;

joint line setting means for setting joint lines on a profile of the image of one of the objects most proximate to the imaging means in the respective overlap regions, on the basis of the sampled parallaxes; and

picture image composition means for providing a composite picture image by compositing the pair of divided picture images such that the set joint lines overlap.

8. The picture image forming apparatus of claim 7, wherein the joint line setting means of the picture image forming apparatus sets joint lines on a profile most proximate to rims of the divided picture images among the profiles of the images of the objects in the respective overlap regions.

9. The picture image forming apparatus of claim 7, wherein the joint line setting means of the picture image forming apparatus calculates candidates of a plurality of joint lines by using respective different joint line determining methods and one of the candidates of joint lines is selected and determined to be the joint line.

10. The picture image forming apparatus of claim 7, wherein the parallax sampling means corrects the parallaxes of the images of the respective objects such that a least parallax among the parallaxes sampled from the images of the respective objects is nullified.

11. A storage medium capable of being read by a computer, storing a program for allowing the computer to execute the processings of:

inputting a pair of divided picture images having a parallax by imaging an object and a periphery of the object so as to include a same part of the object in predetermined imaging regions;

calculating overlap regions of the respective divided picture images including the images of the object;

sampling the parallax from the images of the object in the two overlap regions;

forming intermediate picture images to be provided by imaging the object from points between the observing points from which the respective divided picture images are imaged on the basis of the images of the object in the overlap regions of the pair of divided picture images and the sampled parallax; and

providing a composite picture image by interpolating the parallax of the images of the object in the two overlap regions by the intermediate picture images and compositing the pair of divided picture images such that the overlap regions overlap.

12. A storage medium capable of being read by a computer, storing a program for allowing the computer to execute the processings of:

inputting a pair of divided picture images having parallaxes in images of objects by imaging the same objects respectively from two different observing points,

calculating overlap regions including the images of the objects in the respective divided picture images,

sampling the parallaxes from the images of the objects in the overlap regions,

setting joint lines on a profile of the image of one of the objects most proximate to the observing points in the respective overlap regions based on the sampled parallaxes, and

providing a composite picture image by compositing the pair of divided picture images such that the set joint lines overlap.