All-round view monitoring system for a motor vehicle

• US 10,099,615 B2
• Filed: 09/28/2015
• Issued: 10/16/2018
• Est. Priority Date: 09/29/2014
• Status: Active Grant

• Current Assignee: VisLab S. r. l.
• Original Assignee: Ambarella Incorporated
• Inventors: Cerri, Pietro, Laghi, Maria Chiara, Grisleri, Paolo, Bertozzi, Massimo, Broggi, Alberto
• Primary Examiner(s): Perungavoor, Sathyanaraya V
• Assistant Examiner(s): Dhillon, Peet
• Application Number:
  US14/867,183
• Publication Number:
  US 20160094808A1
• Time in Patent Office: 1,114 Days
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/12 Mirror assemblies combined with other articles, e.g. clocks
• CPC Class Codes: B60R 1/27 providing all-round vision, e.g. using omnidirectional cameras
• CPC Class Codes: B60R 1/31 providing stereoscopic vision
• CPC Class Codes: B60R 2001/1253 with cameras, video cameras or video screens
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: B60R 2300/8093 for obstacle warning
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A view monitoring system for a vehicle, comprising four pairs of cameras, a processing unit, a visual display unit, an obstacle detection unit and a signaling unit. The four pairs of cameras may be positioned on a front, rear, and sides of the vehicle and detect images of the same number of areas near the vehicle. The processing unit may be configured to provide a single overall image of the surroundings of the vehicle in response to the images. The visual display unit may display the overall image. The obstacle detection unit may detect obstacles in the overall image. The signaling unit may signal the detected obstacles to the driver.

Claims

1. An apparatus comprising:

a plurality of stereo cameras positioned on a vehicle and configured to generate a plurality of images of an environment in an area surrounding said vehicle, wherein each of said stereo cameras comprises two cameras having respective optical axises separated by a corresponding distance;

a plurality of first circuits each connected to a respective one of said stereo cameras to receive said images, said first circuits (i) operating in parallel and (ii) configured to generate a plurality of intermediate point clouds in parallel in response to said images, wherein each of said intermediate point clouds corresponds to one of said stereo cameras;

a second circuit configured to generate a point cloud representative of said environment in three dimensions by merging said intermediate point clouds;

a third circuit configured to estimate a surface area of a ground in said area based on said point cloud; and

a fourth circuit configured to detect one or more parking spaces located in said area surrounding said vehicle based on said point cloud.

2. The apparatus according to claim 1, wherein said third circuit is further configured to detect one or more obstacles in said parking spaces based on said point cloud.

3. The apparatus according to claim 2, wherein said fourth circuit is further configured to set one or more parking maneuvers of said vehicle that maneuver said vehicle into a vacant one of said parking spaces in response to said obstacles.

4. The apparatus according to claim 3, further comprising a fifth circuit configured to automatically drive said vehicle into said vacant parking space based on said parking maneuvers set by said fourth circuit.

5. The apparatus according to claim 1, wherein said first circuits are further configured to process said images by executing at least one of a rectification, a distortion correction, and a disparity calculation.

6. The apparatus according to claim 1, further comprising a display configured to display a three-dimensional image of said environment based on said point cloud.

7. The apparatus according to claim 1, wherein said vehicle advances in said surface area of said ground.

8. The apparatus according to claim 2, further comprising a sound emitter configured to emit a sound inside said vehicle in response to detection of said obstacles in said parking spaces.

9. The apparatus according to claim 2, further comprising a fifth circuit configured to present a message to a display that warns a driver of said vehicle about a presence of said obstacles in said parking spaces.

10. The apparatus according to claim 2, further comprising a fifth circuit configured to prevent an opening of one or more doors of said vehicle in response to detection of one or more of said obstacles in proximity to said one or more doors.

11. The apparatus according to claim 1, further comprising a fourth fifth circuit configured to apply one or more augmented reality techniques to said point cloud.

12. A method for monitoring, comprising the steps of:

generating a plurality of images of an environment in an area surrounding a vehicle using a plurality of stereo cameras positioned on said vehicle, wherein each of said stereo cameras comprises two cameras having respective optical axises separated by a corresponding distance;

receiving said images at a plurality of circuits each being connected to a respective one of said stereo cameras;

generating a plurality of intermediate point clouds in parallel in response to said images using said circuits operating in parallel, wherein each of said intermediate point clouds corresponds to one of said stereo cameras;

generating a point cloud representative of said environment in three dimensions by merging said intermediate point clouds;

estimating a surface area of a ground in said area based on said point cloud; and

detecting one or more parking spaces located in said area surrounding said vehicle based on said point cloud.

13. The method according to claim 12, further comprising the step of:

detecting one or more obstacles in said parking spaces based on said point cloud.

14. The method according to claim 13, further comprising the step of:

setting one or more parking maneuvers of said vehicle that maneuver said vehicle into a vacant one of said parking spaces in response to said obstacles.

15. The method according to claim 14, further comprising the step of:

automatically driving said vehicle into said vacant parking space based on said parking maneuvers.

16. The method according to claim 12,

wherein said vehicle advances in said surface area of said ground.

17. The method according to claim 13, further comprising the step of:

emitting a sound inside said vehicle in response to detection of said obstacles in said parking spaces.

18. The method according to claim 13, further comprising the step of:

presenting a message to a display that warns a driver of said vehicle about a presence of said obstacles in said parking spaces.

19. The method according to claim 13, further comprising the step of:

preventing an opening of one or more doors of said vehicle in response to detection of one or more of said obstacles in proximity to said one or more doors.

20. The method according to claim 12, further comprising the step of:

applying one or more augmented reality techniques to said point cloud.