PANEL TRANSFORM

• US 20170024851A1
• Filed: 07/24/2015
• Published: 01/26/2017
• Est. Priority Date: 07/24/2015
• Status: Active Grant

• Current Assignee: Robert Bosch GmbH, Robert Bosch LLC (Robert Bosch GmbH)
• Original Assignee: Robert Bosch GmbH
• Inventors: Baran, Matthew T.
• Granted Patent: US 10,453,173 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 23/62 Control of parameters via user interfaces
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

System and method of performing various transformations between an omnidirectional image model and a rectilinear image model for use in vehicle imaging systems. In particular, a panel transform system and method for transforming an omnidirectional image from an omnidirectional camera positioned on a vehicle to a rectilinear image. The rectilinear image is then displayed to the driver for use while performing vehicle maneuvers. The panel transform system and method also provide a rectilinear image model based on the omnidirectional camera for use with existing rectilinear imaging processing systems. The rectilinear image model is determined based on a variable set of input parameters that are defined by both automatic and manual system inputs such as a steering wheel angle sensor and a user interface.

Claims

1. A method of converting an omnidirectional image from an omnidirectional camera located on a vehicle to a rectilinear image by a control unit, the method comprising:

accessing an omnidirectional camera model representative of an omnidirectional image on a unit sphere in an optical coordinate system, the omnidirectional camera model having a first plurality of pixels;

receiving a set of parameters defining a local plane tangent to the unit sphere, the local plane representative of a rectilinear camera model having a second plurality of pixels;

determining a plane extent of the local plane based on the set of parameters, the plane extent defining a boundary of the local plane;

determining a relationship between the first plurality of pixels and the second plurality of pixels;

receiving, at the control unit, the omnidirectional image from the omnidirectional camera; and

generating a rectilinear image from the omnidirectional image based on the relationship between the first plurality of pixels and the second plurality of pixels.

2. The method of claim 1, wherein receiving the set of parameters defining the local plane tangent to the unit sphere includes receiving information describing a vector originating at an origin of the optical coordinate system and pointing to a point of contact between the local plane and the unit sphere.

3. The method of claim 2, wherein the information describing the vector includes a plurality of angles.

4. The method of claim 2, wherein the set of parameters further includes a field of view and a resolution of the local plane.

5. The method of claim 3, wherein the vector defines a line of sight from the optical origin to a region of interest on the omnidirectional image.

6. The method of claim 1, wherein defining a boundary of the local plane further includes defining a complete perimeter of the local plane with respect to the optical coordinate system.

7. The method of claim 1, further comprising determining a sample spacing of the second plurality of pixels based on an angular sample spacing of the omnidirectional camera model and the set of parameters.

8. The method of claim 1, wherein the local plane tangent to the unit sphere defines a region of interest in the omnidirectional image.

9. The method of claim 8, further comprising mapping a location of each pixel of the first plurality of pixels within the region of interest defined by the set of parameters to a world location described by optical coordinates.

10. The method of claim 9, wherein mapping a location of each pixel of the first plurality of pixels further comprises calibrating the mapping of each location of the plurality of locations on the unit sphere with a predefined calibration pattern described by optical coordinates.

11. The method of claim 1, wherein determining the relationship between the first plurality of pixels and the second plurality of pixels includes

determining a local plane grid that defines coordinate points on the local plane with respect to the optical coordinate system;

and

determining a unit sphere grid that defines coordinate points on the unit sphere with respect to the optical coordinate system by projecting the coordinate points of the local plane grid onto the unit sphere.

12. The method of claim 9, wherein each coordinate point on the unit sphere grid defines each pixel location of the first plurality of pixels, and wherein each corresponding coordinate point on the local plane grid defines each corresponding pixel location of the second plurality of pixels.

13. The method of claim 10, wherein generating the rectilinear image includes copying information from each of the first plurality of pixels from the omnidirectional image to each corresponding pixel of the second plurality of pixels.

14. The method of claim 1, further comprising displaying the rectilinear image on a display in the vehicle.

15. The method of claim 1, wherein receiving a set of parameters defining a local plane tangent to a unit sphere includes receiving the set of parameters from an external controller.

16. The method of claim 1, wherein receiving a set of parameters defining a local plane tangent to a unit sphere includes determining the set of parameters by the control unit based on driver input.

17. The method of claim 1, wherein generating a rectilinear image is performed by projecting a portion of the omnidirectional image from the unit sphere onto the local plane based on the set of parameters.

18. The method of claim 1, further comprising

receiving a second set of parameters defining a second local plane tangent to the unit sphere, the second local plane representative of a second rectilinear camera model having a third plurality of pixels;

determining a relationship between the first plurality of pixels and the third plurality of pixels;

receiving, at the control unit, the omnidirectional image from the omnidirectional camera; and

generating a second rectilinear image from the omnidirectional image based on the relationship between the first plurality of pixels and the third plurality of pixels;

wherein the second rectilinear image has a different perspective view than the rectilinear image.

19. The method of claim 18, further comprising displaying the second rectilinear image on a display in the vehicle.

20. The method of claim 1, wherein determining a relationship between the first plurality of pixels and the second plurality of pixels is performed, in part, through interpolation using look-up tables.