OBSTACLE DETECTION IN VEHICLE USING A WIDE ANGLE CAMERA AND RADAR SENSOR FUSION

• US 20200134396A1
• Filed: 04/18/2019
• Published: 04/30/2020
• Est. Priority Date: 10/25/2018
• Status: Active Grant

• Current Assignee: Ambarella International LP (Ambarella Incorporated)
• Original Assignee: Ambarella Incorporated
• Inventors: Porta, Pier Paolo
• Granted Patent: US 10,860,899 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 1/23 with a predetermined field of view
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 15/86 Combinations of sonar systems with lidar systems; Combinations of sonar systems with systems not using wave reflection
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G05D 1/0231 using optical position detecting means position-fixing by using electromagnetic waves other than radio waves, e.g. optical position detecting means G01S5/16
• CPC Class Codes: G05D 1/0255 using acoustic signals, e.g. ultra-sonic singals sonar systems designed for anti-collision purposes G01S15/93
• CPC Class Codes: G05D 1/0257 using a radar radar systems designed for anti-collision purposes between land vehicles or between land vehicle and fixed obstacles G01S13/931
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06V 10/80 Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level multimodal speaker identification or verification G10L17/10
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

An apparatus includes a primary surround view camera, a supplementary camera, a detection and ranging sensor, and a surround view display. The primary surround view camera is generally placed at a front of a vehicle and provides an operator of the vehicle with a view of the road. The at least one detection and ranging sensor is generally mounted adjacent to the supplementary camera and configured to detect obstacles within a field of view of the supplementary camera. An output of the primary surround view camera is generally used to produce a two-dimensional view of an area around the vehicle and an output of the supplementary camera is (i) reduced to a portion of the field of view of the supplementary camera in which the detection and ranging sensor detected an obstacle and (ii) overlaid on the two-dimensional view of the area around the vehicle to inform the operator of the detected obstacle.

Claims

1. An apparatus comprising:

a primary surround view camera placed at a front of a vehicle and providing an operator of the vehicle with a view of the road;

a supplementary camera placed on the vehicle higher than the primary surround view camera;

at least one detection and ranging sensor mounted adjacent to the supplementary camera and configured to detect obstacles within a field of view of the supplementary camera; and

a surround view display, wherein an output of the primary surround view camera is used to produce a two-dimensional view of an area around the vehicle and an output of the supplementary camera is (i) reduced to a portion of the field of view of the supplementary camera in which the detection and ranging sensor detected an obstacle and (ii) overlaid on the two-dimensional view of the area around the vehicle to inform the operator of the detected obstacle.

2. The apparatus according to claim 1, wherein the supplementary camera has a field of view less than or substantially equal to 180 degrees.

3. The apparatus according to claim 1, wherein the supplementary camera has a fisheye lens.

4. The apparatus according to claim 1, wherein the at least one detection and ranging sensor utilizes at least one of radar, lidar, and ultrasound.

5. The apparatus according to claim 1, further comprising a processing circuit connected to the supplementary camera and the detection and ranging sensor, wherein said processing circuit is configured to perform a dewarping operation on an output of the supplementary camera to obtain the portion of the field of view of the supplementary camera in which the obstacle is detected.

6. The apparatus according to claim 5, wherein the dewarping operation performed by the processing circuit is directed by an output of the at least one detection and ranging sensor.

7. The apparatus according to claim 5, wherein the processing circuit performs a camera and sensor fusion operation using outputs of the supplementary camera and the at least one detection and ranging sensor.

8. The apparatus according to claim 1, wherein the vehicle comprises an autonomous vehicle.

9. A method of obstacle detection comprising:

mounting a primary surround view camera at a front of a vehicle, wherein the primary surround view camera is configured to provide an operator of the vehicle with a view of the road;

mounting a supplementary camera at a location higher on the vehicle than the location of the primary surround view camera;

mounting a detection and ranging sensor with the supplementary camera; and

detecting obstacles within a field of view of the supplementary camera using the detection and ranging sensor, and reducing an output of the supplementary camera to a portion of the field of view of the supplementary camera in which the detection and ranging sensor detected an obstacle.

10. The method according to claim 9, wherein the supplementary camera has a field of view less than or equal to 180 degrees.

11. The method according to claim 9, wherein the supplementary camera has a fisheye lens.

12. The method according to claim 9, further comprising performing a dewarping operation on an output of the supplementary camera to obtain the portion of the field of view of the supplementary camera in which the obstacle was detected.

13. The method according to claim 12, wherein the dewarping operation is performed by a processing circuit and is directed by an output of the detection and ranging sensor.

14. The method according to claim 13, wherein:

the processing circuit performs camera and sensor fusion using outputs of the supplementary camera and the detection and ranging sensor; and

the detection and ranging sensor utilizes at least one of radar, lidar, and ultrasound.

15. The method according to claim 9, further comprising displaying the portion of the field of view of the supplementary camera in which the detection and ranging sensor detected an obstacle to the operator of the vehicle.