Method and system for generating surrounding seamless bird-view image with distance interface

• US 8,446,471 B2
• Filed: 03/25/2010
• Issued: 05/21/2013
• Est. Priority Date: 12/31/2009
• Status: Active Grant

• Current Assignee: Chuang XIN SHAN XING Limited
• Original Assignee: Industrial Technology Research Institute
• Inventors: Wu, Tung-Chuan, Huang, Li-Kung, Tseng, Din-Chang, Shen, Hsin-Liang, Cheng, Kuo-Hsiang
• Primary Examiner(s): Follansbee, John
• Assistant Examiner(s): Woolcock, Madhu
• Application Number:
  US12/731,419
• Publication Number:
  US 20110157361A1
• Time in Patent Office: 1,153 Days
• Field of Search: 348/148, 348/218.1, 382/104, 382/284, 340/932.2
• US Class Current: 348/148
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G09G 2380/14 Electronic books and readers

Abstract

The present invention provides a method and system for generating a surrounding seamless bird-view image with distance assistance interface, which process a plurality of surrounding images associated with a carrier captured by different image sensing devices. After that, an assistant marking is determined according to the relationship between acquired image space and the real space and thereby is formed on the surrounding seamless bird-view image so as to assist the driver in the carrier for avoiding collision or events affecting the safety being occurred.

Claims

1. A method for generating surrounding seamless bird-view image with distance interface, comprising the steps of:

providing a plurality of surrounding images;

converting the plural surrounding images so as to combine the plural converted surrounding images into a surrounding seamless bird-view image; and

generating an auxiliary image containing marking aids for distance estimation in the surrounding seamless bird-view image,

wherein converting the plural surrounding images comprises the steps of;

performing a viewpoint transformation process upon the plural surrounding images for transforming the plural surrounding images into a plurality of corresponding bird-view images,

identifying a plurality of overlapped regions from pairs of adjacent bird-view images, the overlapped regions having boundaries along lines at which the pairs of adjacent bird-view images begin to overlap,

performing a weight calculation process for each overlapped region so as to obtain a plurality of overlapped images respectively corresponding to each overlapped region, and

combining the plural overlapped images and the portions of the plural transformed bird-view images that are not overlapped into the surrounding seamless bird-view image, and

wherein for each overlapped region, the weight calculation process comprises the steps of;

measuring a first distance along a first direction between each pixel in the overlapped region and a first one of the boundaries of the overlapped region and a second distance between that pixel and a second one of the boundaries of the overlapped region,

determining a first weighting value and a second weighting value for each pixel in the overlapped region according to the measured first and second distances of the respective pixel, and

performing a weighted feature calculation on an image feature of each pixel in the overlapped region by use of the first weighting value and the second weighting value so as to obtain an overlapped feature for each pixel in the overlapped region.

2. The method of claim 1, wherein the weight calculation process is performed on an image feature, and wherein the image feature is a feature selected from the group comprising:

brightness and color.

3. The method of claim 1, wherein the identifying of overlapped region further comprises the steps of:

determining a reference plane;

performing a search in the two adjacent bird-view images along the reference plane for acquiring at least one pair of corresponding points from the two adjacent bird-view images; and

adjusting the two adjacent bird-view images for minimizing an error associating with the at least one pair of corresponding points.

4. The method of claim 3, wherein each of the corresponding points of the two adjacent bird-view images is a specific object in its corresponding bird-view image selected from the group comprising:

lines, patterns and texts.

5. The method of claim 3, wherein the adjusting of the two adjacent bird-view images is performed in a manner selected from the group comprising:

enabling the two adjacent bird-view images to rotate relative to each other, scaling the two adjacent bird-view images with respect to each other, shifting the two adjacent bird-view images relative to each other, and any combination thereof.

6. The method of claim 3, wherein the error is a mean square of the image feature respectively associating to the at least one pair of the corresponding points in the two adjacent bird-view images.

7. The method of claim 6, wherein the characteristic value is a value selected from the group comprising:

a brightness value and a color value.

8. The method of claim 1, wherein further comprising the step of:

transmitting the surrounding seamless bird-view image along with the auxiliary image containing marking aids for distance estimation to a display unit for displaying.

9. The method of claim 1, wherein the plurality surrounding images are obtained by use of a plurality of image sensing devices as the plural image sensing devices are mounted on a carrier at different positions thereof while focusing in different directions around the carrier.

10. The method of claim 9, wherein the carrier is a vehicle.

11. The method of claim 1, wherein each marking aid in the auxiliary image is an icon selected from the group consisting:

a scale line and a frame.

12. A system for generating surrounding seamless bird-view image with distance interface, comprising:

a plurality of imaging units, respectively mounted on a carrier at different positions thereof, for generating plural surrounding images;

a control unit for;

performing a viewpoint transformation process upon the plural surrounding images for transforming the plural surrounding images into a plurality of corresponding bird-view images,

identifying a plurality of overlapped regions from pairs of adjacent bird-view images, the overlapped regions having boundaries along lines at which the pairs of adjacent bird-view images begin to overlap,

performing a weight calculation process for each overlapped region so as to obtain a plurality of overlapped images respectively corresponding to each overlapped region, and

combining the plural overlapped images and portions of the plural transformed bird-view images that are not overlapped into the surrounding seamless bird-view image, and

a display unit, for receiving the surrounding seamless bird-view image and an auxiliary signal so as to display the surrounding seamless bird-view image while forming marking aids contained in the auxiliary image signal on the surrounding seamless bird-view image,

wherein for each overlapped region, the weight calculation process is performed by;

measuring a first distance along a first direction between each pixel in the overlapped region and a first one of the boundaries of the overlapped region and a second distance between that pixel and a second one of the boundaries of the overlapped region,

determining a first weighting value and a second weighting value for each pixel in the overlapped region according to the measured first and second distances of the respective pixel, and

performing a weighted feature calculation on an image feature of each pixel in the overlapped region by use of the first weighting value and the second weighting value so as to obtain an overlapped feature for each pixel in the overlapped region.

13. The system of claim 12, wherein the plurality of imaging units are mounted on the carrier respectively at a front side, a rear side, a left side and a right side thereof.

14. The system of claim 12, wherein the control unit further comprises:

an image receiving interface, for receiving the plural surrounding images;

a calculation unit, for performing a process upon the received plural surrounding images so as to form the surrounding seamless bird-view image accordingly;

a memory module, coupled to the calculation unit; and

an image output interface, coupled to the calculation unit and the display unit, for outputting the surrounding seamless bird-view image to the display unit.

15. The system of claim 12, wherein the carrier is a vehicle.

16. The system of claim 12, wherein each marking aid in the auxiliary image is an icon selected from the group consisting:

a scale line and a frame.

17. The system of claim 12, wherein the display unit further comprises a human-machine interface.

18. The system of claim 17, wherein the human-machine interface is provided for enabling a user to alter formats of the marking aids.

19. The system of claim 12, wherein the control unit further comprises:

a power interface, for receiving power from a voltage source; and

a carrier signal interface, for receiving signals generated from the carrier.

20. The system of claim 12, wherein each imaging unit is a device selected from the group comprising:

a camera and a camcorder.

21. The system of claim 14, wherein the memory module further comprises:

a static random access memory and a flash memory.