Image synthesis system for a vehicle and the manufacturing method thereof

• US 8,150,210 B2
• Filed: 04/11/2008
• Issued: 04/03/2012
• Est. Priority Date: 01/18/2008
• Status: Active Grant

• Current Assignee: National Chiao Tung University (Government of The Republic of China)
• Original Assignee: National Chiao Tung University (Government of The Republic of China)
• Inventors: Chen, Yong-Sheng, Liu, Yu-Chih, Lin, Kai-Ying
• Primary Examiner(s): Cunningham, Gregory F
• Application Number:
  US12/101,325
• Publication Number:
  US 20090175492A1
• Time in Patent Office: 1,453 Days
• Field of Search: 340/426.23, 340/435, 340/436, 340/437, 340/937, 340/995.17, 348/47, 348/143, 348/147, 348/148, 348/149, 348/159, 348/584, 348/E5.03, 358/3.04, 358/3.24, 358/450, 382/104, 382/276, 382/284, 710/15
• US Class Current: 382/284
• CPC Class Codes: B60R 2300/102 using 360 degree surveillance camera system
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06V 10/16 using multiple overlapping images; Image stitching
• CPC Class Codes: G06V 10/24 Aligning, centring, orientation detection or correction of the image
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

The present invention discloses an image synthesis system for a vehicle to provide the driver with a downward-facing image of the car'"'"'s 360° surrounding view. The system includes: a first camera, which is used to shoot a first image of the periphery of the vehicle; a second camera, which is used to shoot a second image of the periphery of said vehicle, wherein the second image and the first image have an overlap region; an image processing device comprising a defining component and a synthesis component, which is used to synthesize the first image and the second image and output a third image; a display device, which is used to display the third image.

Claims

1. An image synthesis system for a vehicle comprising:

a first camera disposed on the vehicle and shooting at least a first image of at least a first portion of an outward periphery of said vehicle;

a second camera disposed on the vehicle and shooting at least a second image of at least a second portion of the outward periphery of said vehicle, wherein the second image and the first image have an overlapping region;

an image processing device comprising;

a defining component operable to define the first image into a plurality of first points each having a first residual error, and defining the second image into a plurality of second points each having a second residual error; and

,

a synthesis component operable to synthesize the first image and the second image and output a third image, wherein each point of the overlapping region of the third image is selected from one of the plurality of first points and a spatially corresponding one of the plurality of second points responsive, at least in part, to a determination of which point has lower residual error to thereby be shown in the third image; and

a display device for displaying at least the third image.

2. According to the system as claimed in claim 1, wherein the third image has at least a seam between the first image and second image.

3. According to the system as claimed in claim 2, wherein at least one object is present on the seam and the edges of the object in the first image are misaligned to the edges in the second image, and the image processing device further comprises a seam registration component for aligning the edges of the object on the seam.

4. According to the system as claimed in claim 3, the image processing device further comprises a deformation component for deforming the first image and the second image by propagating the alignment of the seam to the rest of the first image and the second image.

5. According to the system as claimed in claim 1, wherein the image processing device further comprises a transformation component for transforming the first image and second image into a downward-facing image.

6. An image synthesis method for a vehicle, the method comprising:

shooting at least a first image of at least a first portion of an outward periphery of said vehicle by a first camera disposed on the vehicle;

shooting at least a second image of at least a second portion of the outward periphery of said vehicle by a second camera disposed on the vehicle, wherein the second image and the first image have an overlapping region;

processing the image comprising;

defining the first image into a plurality of first points each having a first residual error, and defining the second image into a plurality of second points each having a second residual error; and

synthesizing the first image and the second image and outputting a third image, wherein each point of the overlapping region of the third image is selected from one of the plurality of first points and a spatially corresponding one of the plurality of second points responsive, at least in part, to a determination of which point has lower residual error to thereby be shown in the third image; and

displaying at least the third image.

7. According to the method as claimed in claim 6, wherein the third image has at least a seam between the first image and second image.

8. According to the method as claimed in claim 7, wherein at least one object is present on the seam and the edges of the object in the first image are misaligned to the edges in the second image, and the method further comprises aligning the edges of the object on the seam.

9. According to the method as claimed in claim 8, the method further comprises deforming the first image and the second image by propagating the alignment of the seam to the rest of the first image and the second image.

10. According to the method as claimed in claim 6, further comprising transforming the first image and second image into a downward-facing image.

11. An image synthesis device for synthesizing at least a first image and at least a second image, wherein the first image is taken by a first camera disposed on a vehicle, and the second image is taken by a second camera disposed on the vehicle, and the second image and the first image have an overlapping region, the image synthesis device comprising:

a defining component for defining the first image into a plurality of first points each having a first residual error, and defining the second image into a plurality of second points each having a second residual error; and

a synthesis component for synthesizing the first image and the second image and output a third image, wherein each point of the overlapping region of the third image is selected from one of the plurality of first points and a spatially corresponding one of the plurality of second points, responsive, at least in part, to a determination of which point has lower residual error to thereby be shown in the third image.

12. According to the device as claimed in claim 11, wherein the third image has at least a seam between the first image and second image.

13. According to the device as claimed in claim 12, wherein at least one object is present on the seam and the edges of the object in the first image are misaligned to the edges in the second image, and the device further comprises a seam registration component for aligning the edges of the object on the seam.

14. According to the device as claimed in claim 13, further comprising a deformation component for deforming the first image and the second image by propagating the alignment of the seam to the rest of the first image and the second image.

15. According to the device as claimed in claim 11, further comprising a transformation component for transforming the first image and second image into a downward-facing image.

16. An image synthesis method for synthesizing at least a first image and at least a second image, wherein the first image is taken by a first camera disposed on a vehicle, and the second image is taken by a second camera disposed on the vehicle, and the second image and the first image have an overlapping region, the image synthesis method comprising:

defining the first image into a plurality of first points each having a first residual error, and defining the second image into a plurality of second points each having a second residual error; and

synthesizing at least the first image and the second image and outputting at least a third image, wherein each point of the overlapping region of the third image is selected from one of the plurality of first points and a spatially corresponding one of the plurality of second points, responsive, at least in part, to a determination of which point has lower residual error to thereby be shown in the third image.

17. According to the method as claimed in claim 16, wherein the third image has at least a seam between the first image and second image.

18. According to the method as claimed in claim 17, wherein at least one object is present on the seam and the edges of the object in the first image are misaligned to the edges in the second image, and the method further comprising aligning the edges of the object on the seam.

19. According to the method as claimed in claim 18, further comprising deforming the first image and the second image by propagating the alignment of the seam to the rest of the first image and the second image.

20. According to the method as claimed in claim 16, further comprising transforming the first image and second image into a downward-facing image.