Method and system for generating a surround view

• US 10,075,634 B2
• Filed: 12/26/2012
• Issued: 09/11/2018
• Est. Priority Date: 12/26/2012
• Status: Active Grant

• Current Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman International Industries Incorporated (Samsung Electronics Co. Ltd.)
• Inventors: He, Jia, Weyrich, Norman
• Primary Examiner(s): Ustaris, Joseph
• Assistant Examiner(s): Adrovel, William
• Application Number:
  US14/655,685
• Publication Number:
  US 20150334301A1
• Time in Patent Office: 2,085 Days
• Field of Search: None
• CPC Class Codes: B60R 2011/004 outside the vehicle
• CPC Class Codes: B60R 2300/00 Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/306 using a re-scaling of images
• CPC Class Codes: B60R 2300/402 Image calibration
• CPC Class Codes: B60R 2300/60 characterised by monitoring and displaying vehicle exterior scenes from a transformed perspective
• CPC Class Codes: B60R 2300/602 with an adjustable viewpoint
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint
• CPC Class Codes: G06T 2207/30248 Vehicle exterior or interior
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G08G 1/168 Driving aids for parking, e.g. acoustic or visual feedback on parking space
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/282 for generating image signals corresponding to three or more geometrical viewpoints, e.g. multi-view systems
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A method and system for generating a surround view are provided. The method may include: establishing a surround surface; obtaining a plurality of images of surroundings; and projecting the images onto the surround surface based on a projection relationship between points on the surround surface and pixels on the images to generate a surround view, where the projection relationship may change with heights of the points on the surround surface. An improved projection effect may be obtained.

Claims

1. A method for generating a surround view, comprising:

establishing a surround surface;

obtaining one or more images of surroundings; and

generating a surround view by projecting the one or more images onto the surround surface based on a projection relationship between a point on the surround surface and a pixel on the images, wherein the projection relationship changes with a height of the point on the surround surface from a bottom surface of the surround surface, and wherein the projection relationship is derived at least by applying a translation matrix from a world coordinate system to a camera coordinate system, where the translation matrix used for each projected pixel is weighted differently based on a height of the projected pixel,

wherein the projection relationship is obtained based on an equation;

2. The method according to claim 1, wherein the weighting factor W is equal to 1 when the height of the point relative to the bottom surface or a lowest tangent plane of the surround surface is equal to 0.

3. The method according to claim 1, wherein the weighting factor W is equal to 0 when the height of the point is greater than a height of the origin of the world coordinate system from the bottom surface of the surround surface.

4. The method according to claim 1, wherein the weighting factor W is greater than 0 and less than 1 when the height of the point is between 0 and the height of the origin of the world coordinate system.

5. The method according to claim 4, wherein the weighting factor W reduces with increasing of the height of the point.

6. A system for generating a surround view, comprising:

a processing device, configured to;

establish a surround surface,

obtain one or more images of surroundings, and

project the one or more images onto the surround surface based on a projection relationship between a point on the surround surface and a pixel on the images to generate a surround view, where the projection relationship changes with a height of the point on the surround surface from a bottom surface of the surround surface, wherein the projection relationship is derived at least by applying a translation matrix from a world coordinate system to a camera coordinate system, where the translation matrix used for each projected pixel is weighted differently based on a height of the projected pixel; and

a display device for showing the surround view,

wherein the projection relationship between the images and the surround surface is obtained based on an equation;

7. The system according to claim 6, wherein the weighting factor W is equal to 1 when the height of the point relative to a lowest tangent plane of the surround surface is equal to 0.

8. The system according to claim 6, wherein the weighting factor W is equal to 0 when the height of the point is greater than a height of the origin of the world coordinate system from the bottom surface of the surround surface.

9. The system according to claim 6, wherein the weighting factor W is greater than 0 and less than 1 when the height of the point is between 0 and the height of the origin of the world coordinate system.

10. The system according to claim 9, wherein the weighting factor W reduces with increasing of the height of the point.

11. A system for generating a surround view of an object, comprising:

one or more cameras for acquiring one or more images of surroundings of the object, and

a processing device, configured to;

establish a surround surface of the object; and

generate a surround view by projecting the one or more images onto the surround surface based on a projection relationship between a point on the surround surface in a world coordinate system and a pixel on the images in a camera coordinate system, wherein the projection relationship includes a rotation matrix and a translation matrix, and the translation matrix is weighted based on a height of the point on the surround surface from a bottom surface of the surround surface, wherein the projection relationship is derived at least by applying the translation matrix from the world coordinate system to the camera coordinate system,

wherein the projection relationship between the images and the surround surface is obtained based on an equation;

12. The system according to claim 11, wherein the weighting factor W is equal to 1 when the height of the point relative to the bottom surface or a lowest tangent plane of the surround surface is equal to 0.

13. The system according to claim 11, wherein the weighting factor W is equal to 0 when the height of the point is greater than a height of the origin of the world coordinate system from the bottom surface of the surround surface.

14. The system according to claim 11, wherein the weighting factor W reduces with increasing of the height of the point when the height of the point is between 0 and the height of the origin of the world coordinate system.