Surround view system

• US 10,076,997 B2
• Filed: 07/31/2012
• Issued: 09/18/2018
• Est. Priority Date: 08/05/2011
• Status: Active Grant

• Current Assignee: Harman Becker Automotive Systems GmbH (Samsung Electronics Co. Ltd.)
• Original Assignee: Harman Becker Automotive Systems GmbH (Samsung Electronics Co. Ltd.)
• Inventors: Quast, Johannes, Scholl, Kay-Ulrich, Gassmann, Bernd
• Primary Examiner(s): Yenke, Brian
• Assistant Examiner(s): Satti, Humam
• Application Number:
  US13/563,408
• Publication Number:
  US 20130033602A1
• Time in Patent Office: 2,240 Days
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 1/28 with an adjustable field of view
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/602 with an adjustable viewpoint
• CPC Class Codes: B60R 2300/607 from a bird's eye viewpoint

Abstract

A system operable to generate a surround view based on image data, where the image data may include image data associated with surroundings of an object, such as a vehicle. The system may also be operable to project the surround view inversely onto a bowl-shaped projection surrounding the object. Further, the system may be operable to generate a virtual user view via a virtual camera position, where the virtual camera position is on a first horizontal ellipse that is about the bowl-shaped projection at a first height. Also, the system is operable to determine a viewing direction from the virtual camera position that points at a location on a second horizontal ellipse that is about the bowl-shaped projection at a second height. The second height may be lower than the first height and the second horizontal ellipse may be smaller than the first horizontal ellipse.

Claims

1. A surround view system comprising:

an image processing device including a processor and memory storing instructions executable to;

generate a surround view based on image data collected from image sensors, where the surround view projects the image data inversely on a bowl-shaped projection surrounding an object; and

generate a virtual user view from a virtual camera position at a viewing direction, where the virtual camera position is on a first horizontal ellipse parallel to the ground at a first height above the ground, and the viewing direction directs from the virtual camera position to a point located on a second horizontal ellipse having a major axis and a minor axis that is parallel to the ground at a second height above the ground, where the major axis is longer than the minor axis, where the second height is lower than the first height, where the second horizontal ellipse is smaller than the first horizontal ellipse, and where each of the first horizontal ellipse and the second horizontal ellipse is smaller than the bowl-shaped projection at associated respective vertical locations of the bowl-shaped projection and each of the first horizontal ellipse and the second horizontal ellipse extends around the object; and

a control device including one or more operating elements to change the viewing direction and to change the virtual camera position at least in a horizontal direction,

where the virtual camera position and the viewing direction are derived from the first and second horizontal ellipses, and where the image processing device is operable to;

maintain the virtual camera position and to change the viewing direction by moving the point located on the second horizontal ellipse in response to a command of changing the viewing direction.

2. The surround view system according to claim 1, where the instructions are further executable to:

maintain the virtual camera position and to change the viewing direction, so that an opening angle of the viewing direction is within a predefined maximum opening angle, wherein the opening angle of the viewing direction is defined by a projection of a direct line of the virtual camera position to a center of the second horizontal ellipse and by a direct line of the center of the second horizontal ellipse to the point located on the second horizontal ellipse, and wherein, when changing the viewing direction, the viewing direction follows points located on the second horizontal ellipse to continue directing from the virtual camera position to the second horizontal ellipse.

3. The surround view system according to claim 2, where the instructions are further executable to:

adjust the viewing direction of the virtual camera position on the first horizontal ellipse to maintain the opening angle within the predefined maximum opening angle.

4. The surround view system according to claim 1, where the instructions are further executable to determine a perimeter of the second horizontal ellipse so that the perimeter of the second horizontal ellipse corresponds to a perimeter of the object, and where a size of the first horizontal ellipse is within a range of ten to thirty percent greater than a size of the second horizontal ellipse.

5. The surround view system according to claim 1, where:

the instructions are further executable to provide a starting object view in which the virtual camera position is behind the object at a point of the first horizontal ellipse that is located on a first axis of the first horizontal ellipse, and

where the viewing direction points towards a location on the second horizontal ellipse that is located on a first axis of the second horizontal ellipse, where the first axis of the first horizontal ellipse is a major axis of the first horizontal ellipse and the first axis of the second horizontal ellipse is the major axis of the second horizontal ellipse.

6. The surround view system according to claim 1, where the instructions are further executable to position the first horizontal ellipse and the second horizontal ellipse coaxially and parallel to each other and wherein the first height of the first horizontal ellipse is within a range of four and a half feet to seven and a half feet above the ground and the viewing direction is tilted downward toward the second horizontal ellipse.

7. The surround view system according to claim 1, where the control device contains a turning element that is a part of a vehicle electronic system used for controlling another device in a vehicle, the turning element operable to be:

turned in a first direction, to indicate a change of the virtual camera position in a first horizontal direction; and

turned in a second, different direction, to indicate a change of the virtual camera position in a second horizontal direction, where the object remains centered with respect to the virtual user view as the virtual camera position changes in each of the first and second horizontal directions.

8. The surround view system according to claim 1, where:

the instructions are further executable to change the virtual camera position vertically from a first height to a second height above the object, and

where a perimeter of the second horizontal ellipse and a perimeter of the first horizontal ellipse are based on the second height above the object.

9. The surround view system according to claim 1, where:

the instructions are further executable to change the virtual camera position in a direction about a first ellipsoidally shaped body defined by a rotation of the first horizontal ellipse, and

where the viewing direction points towards a location on a second ellipsoidally shaped body defined by a rotation of the second horizontal ellipse.

10. A method comprising:

generating via an image processing device of a surround view system, a surround view based on image data corresponding to surroundings of an object;

projecting the surround view inversely onto a bowl-shaped projection surrounding the object;

generating a virtual user view via a virtual camera position and a viewing direction, where the virtual camera position is on a first horizontal ellipse located parallel to the ground at a first height above the ground, and the viewing direction directs from the virtual camera position to a point located on a second horizontal ellipse located parallel to the ground at a second height above the ground, where the second horizontal ellipse has a major axis and a minor axis, the major axis being longer than the minor axis, where the second height is lower than the first height, where the second horizontal ellipse is smaller than the first horizontal ellipse, and where each of the first horizontal ellipse and the second horizontal ellipse is smaller than the bowl-shaped projection at associated respective vertical locations of the bowl-shaped projection and each of the first horizontal ellipse and the second horizontal ellipse extends around the object; and

adjusting, via a control device, the viewing direction and the virtual camera position at least in a horizontal direction,

wherein the virtual camera position and the viewing direction are determined based on the first and second horizontal ellipses, the method further comprising;

maintaining the virtual camera position; and

changing the viewing direction by moving the point located on the second horizontal ellipse responsive to a command for a horizontal view change.

11. The method according to claim 10, further comprising:

maintaining the virtual camera position; and

changing the viewing direction until a predefined maximum opening angle of the viewing direction is obtained, wherein an opening angle of the viewing direction is defined by a projection of a direct line of the virtual camera position to a center of the second horizontal ellipse and by a direct line of the center of the second horizontal ellipse to the point located on the second horizontal ellipse.

12. The method according to claim 11, further comprising:

changing the virtual camera position on the first horizontal ellipse in a horizontal direction; and

determining the viewing direction of the virtual camera position on the first horizontal ellipse so that the predefined maximum opening angle is maintained.

13. The method according to claim 10, further comprising:

determining a perimeter of the second horizontal ellipse, where the perimeter of the second horizontal ellipse corresponds to a perimeter of the object.

14. The method according to claim 10, further comprising:

initially positioning the virtual camera behind the object in a starting object view, where the virtual camera position is located on a first axis of the first horizontal ellipse, and where the viewing direction points toward a location located on a first axis of the second horizontal ellipse.

15. The method according to claim 10, further comprising:

positioning the first horizontal ellipse and the second horizontal ellipse coaxially and parallel to each other, and where the point to which the viewing direction directs from the virtual camera position is located on a perimeter of the second horizontal ellipse.

16. The method according to claim 10, further comprising:

changing the virtual camera position vertically from a first height above the object to a second height above the object, where a perimeter of the second horizontal ellipse and a perimeter of the first horizontal ellipse are based on the second height above the object.

17. The method according to claim 10, further comprising:

changing the virtual camera position in a direction about a first ellipsoidally shaped body defined by a rotation of the first horizontal ellipse, where the viewing direction points towards a location on a second ellipsoidally shaped body defined by a rotation of the second horizontal ellipse.

18. A method performed by an image processing device, comprising:

generating a surround view based on image data, where the image data includes image data associated with surroundings of a vehicle;

projecting the surround view inversely onto a bowl-shaped projection surrounding the vehicle;

generating a virtual user view on a display via a virtual camera position and a viewing direction, where the virtual camera position is on a first horizontal ellipse parallel to the ground at a first height above the ground, and the viewing direction directs from the virtual camera position to a point located on a second horizontal ellipse parallel to the ground at a second height above the ground, where the second height is lower than the first height, where the second horizontal ellipse is smaller than the first horizontal ellipse, and where each of the first horizontal ellipse and the second horizontal ellipse is smaller than the bowl-shaped projection at associated respective vertical locations of the bowl-shaped projection and each of the first horizontal ellipse and the second horizontal ellipse extends around the object,

wherein the virtual camera position and the viewing direction are determined based on the first and second horizontal ellipses, the method further comprising;

maintaining the virtual camera position; and

changing the viewing direction by moving the point located on the second horizontal ellipse responsive to a command for a horizontal view change.