Method and system for forming surrounding seamless bird-view image

• US 8,242,895 B2
• Filed: 02/08/2010
• Issued: 08/14/2012
• Est. Priority Date: 12/30/2009
• Status: Active Grant

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Cheng, Kuo-Hsiang, Wu, Tung-Chuan, Tseng, Din-Chang, Shen, Hsin-Liang
• Primary Examiner(s): Mehmood, Jennifer
• Assistant Examiner(s): GIRMA, FEKADESELASS
• Application Number:
  US12/701,833
• Publication Number:
  US 20110156887A1
• Time in Patent Office: 918 Days
• Field of Search: None
• US Class Current: 340/425.5
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06V 10/16 using multiple overlapping images; Image stitching
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A method and system for forming a surrounding seamless bird-view image by processing a plurality of surrounding images associated with a carrier captured by different imaging devices is provided in the present invention. During the process, the surrounding images are converted into corresponding bird-images respectively and then an overlapping area corresponding to each two adjacent bird-view images is determined. After that, distances between the pixel in the overlapping area and the boundary of the overlapping area in each adjacent bird-view image is calculated so as to determine a weighting value for each corresponding pixel in each adjacent bird-view image thereby forming an overlapping image, wherein each pixel in the overlapping image is determined according to the weighting value and information of the corresponding pixel in each adjacent image. After determining the overlapping image for each two adjacent bird-view image, it is capable of forming the surrounding seamless bird-view image.

Claims

1. A method for forming surrounding seamless bird-view image, comprising the steps of:

providing a plurality of surrounding images;

performing a viewpoint transformation process upon the plural surroundings for transforming the same into a plurality of corresponding bird-view images;

identifying an overlapped region from any two adjacent bird-view images;

performing a weight calculation process for multiple times so as to obtain an overlapped image from the overlapped regions of any two adjacent bird-view images, as each weight calculation process further comprises the step of;

measuring a distance between each pixel in the overlapped region and the boundary of the overlapped region in one of the corresponding two adjacent bird-view image and another distance between the same pixel and the boundary of the overlapped region in another corresponding two adjacent bird-view image;

determining a first weighting value for each pixel in the overlapped region of one of the corresponding two adjacent bird-view image while determining a second weighting value for the corresponding pixel of another adjacent bird-view image according to the measured distances of the pixel; and

performing a weighted feature calculation on an image feature of the corresponding pixels in the overlapped regions of the two adjacent bird-view images by the use of the first weighting value and the second weighting value so as to obtain an overlapped feature for the corresponding pixel;

and

combining the plural overlapped images and the portions of the plural transformed bird-view images that are not overlapped into a surrounding seamless bird-view image.

2. The method of claim 1, wherein the feature includes brightness and color.

3. The method of claim 1, wherein the identifying of overlapped region further comprises the steps of:

determining a reference plane;

searching the two adjacent bird-view images along the reference plane for acquiring at least one pair of corresponding points from the two adjacent bird-view images; and

adjusting the two adjacent bird-view images for minimizing an error associating with the at least one pair of corresponding points.

4. The method of claim 3, wherein each of the corresponding points of the two adjacent bird-view images is a specific object in its corresponding bird-view image selected from the group consisting of:

lines, patterns and texts.

5. The method of claim 3, wherein the adjusting of the two adjacent bird-view images is performed in a manner selected from the group consisting of:

enabling the two adjacent bird-view images to rotate relative to each other, scaling the two adjacent bird-view images with respect to each other, shifting the two adjacent bird-view images relative to each other, and the combination thereof.

6. The method of claim 3, wherein the error is substantially the mean square of the characteristic values respectively associating to the at least one pair of the corresponding points in the two adjacent bird-view images.

7. The method of claim 6, wherein the characteristic value is a value selected from the group consisting of:

a brightness value and a color value.

8. The method of claim 1, further comprising the step of:

transmitting the surrounding seamless bird-view image to a display unit disposed inside a carrier for displaying.

9. The method of claim 1, wherein the plurality surrounding images are obtained by the use of a plurality of imaging devices as the plural imaging devices are mounted on a carrier at different positions thereof while focusing in different directions around the carrier.

10. The method of claim 1, wherein the carrier is a vehicle.

11. A system for forming surrounding seamless bird-view image, comprising:

a plurality of imaging units, respectively mounted on a carrier at different positions thereof, each being used for generating a surrounding image;

a control unit, further comprising;

an image receiving interface, for receiving the plural surrounding images;

a calculation unit, for performing a process upon the received plural surrounding images so as to form a surrounding seamless bird-view image accordingly;

a memory module, coupled to the calculation unit; and

an image output interface, coupled to the calculation unit and a display unit, for outputting the surrounding seamless bird-view image; and

the display unit, disposed inside the carrier while coupling to the control unit, for receiving and displaying the surrounding seamless bird-view image;

wherein after receiving the plural surrounding images, the control unit is enabled to perform an operation comprising the steps of;

performing a viewpoint conversion process upon the plural surroundings for transforming the same into a plurality of corresponding bird-view images;

identifying an overlapped region from any two adjacent bird-view images;

performing a weight calculation process so as to obtain an overlapped image from the overlapped regions of any two adjacent bird-view images as the weight calculation process further comprises the steps of;

measuring a distance between each pixel in the overlapped region and the boundary of the overlapped region in one of the corresponding two adjacent bird-view image and another distance between the same pixel and the boundary of the overlapped region in another corresponding two adjacent bird-view image;

determining a first weighting value for each pixel in the overlapped region of one of the corresponding two adjacent bird-view image while determining a second weighting value for the corresponding pixel of another adjacent bird-view image according to the measured distances of the pixel; and

performing a weighted feature calculation on an image feature of the corresponding pixels in the overlapped regions of the two adjacent bird-view images by the use of the first weighting value and the second weighting value so as to obtain an overlapped feature for the corresponding pixels; and

combining the plural overlapped images and the portions of the plural transformed bird-view images that are not overlapped into a surrounding seamless bird-view image.

12. The system of claim 11, wherein the plural imaging devices are mounted on the carrier respectively at the front side, rear side, left side and right side thereof.

13. The system of claim 11, wherein the carrier is a vehicle.

14. The system of claim 11, wherein the identifying of the overlapped region performed by the control unit further comprising the steps of:

determining a reference plane;

searching the two adjacent bird-view images along the reference plane for acquiring at least one pair of corresponding points from the two adjacent bird-view images; and

adjusting the two adjacent bird-view images for minimizing an error associating with the at least one pair of corresponding points.

15. The system of claim 14, wherein each of the corresponding points of the two adjacent bird-view images is a specific object in its corresponding bird-view image selected from the group consisting of:

lines, patterns and texts.

16. The system of claim 14, wherein the adjusting of the two adjacent bird-view images is performed in a manner selected from the group consisting of:

enabling the two adjacent bird-view images to rotate relative to each other, scaling the two adjacent bird-view images with respect to each other, shifting the two adjacent bird-view images relative to each other, and the combination thereof.

17. The system of claim 14, wherein the error is substantially the mean square of the characteristic values respectively associating to the at least one pair of the corresponding points in the two adjacent bird-view images.

18. The system of claim 17, wherein the characteristic value is a value selected from the group consisting of:

a brightness value and a color value.

19. The system of claim 11, wherein the control unit further comprises:

a power interface, for receiving power from a voltage source; and

a carrier signal interface, for receiving signals generated from the carrier.

20. The system of claim 11, wherein each imaging device is a device selected from the group consisting of:

a camera and a camcorder.

21. The system of claim 11, wherein the memory module further comprises:

a static random access memory and a flash memory.