Vehicle parking assist system with vision-based parking space detection

• US 10,078,789 B2
• Filed: 07/14/2016
• Issued: 09/18/2018
• Est. Priority Date: 07/17/2015
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Gupta, Nikhil, Corral-Soto, Eduardo R.
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): Shin, Brian
• Application Number:
  US15/210,223
• Publication Number:
  US 20170017848A1
• Time in Patent Office: 796 Days
• Field of Search: None
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2422/00 Indexing codes relating to the special location or mounting of sensors
• CPC Class Codes: B60W 30/06 Automatic manoeuvring for parking
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06V 20/586 of parking space
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/0962 having an indicator mounted inside the vehicle, e.g. giving voice messages
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: G08G 1/168 Driving aids for parking, e.g. acoustic or visual feedback on parking space

Abstract

A parking assist system of a vehicle includes a camera that, when disposed at the vehicle, has a field of view exterior of the vehicle. An image processor is operable to process image data captured by the camera to detect parking space markers indicative of a parking space and to identify empty or available parking spaces. The image processor includes a parking space detection algorithm that detects parking space markers by (i) extracting low level features from captured image data, (ii) classifying pixels as being part of a parking space line or not part of a parking space line, (iii) performing spatial line fitting to find lines in the captured images and to apply parking space geometry constraints, and (iv) detecting and selecting rectangles in the captured images.

Claims

1. A parking assist system of a vehicle, said parking assist system comprising:

a camera configured to be disposed at a vehicle, wherein said camera, when disposed at the vehicle, has a field of view exterior of the vehicle;

a control comprising an image processor operable to process image data captured by said camera when said camera is disposed at the vehicle;

wherein said control, responsive to processing by said image processor of captured image data, detects parking space markers indicative of a parking space and identifies empty or available parking spaces;

wherein said control uses a parking space detection algorithm that detects parking space markers by (i) extracting low level features from captured image data, (ii) classifying pixels as being part of a parking space line or not part of a parking space line, (iii) performing spatial line fitting to find lines in the captured images and to apply parking space geometry constraints, and (iv) detecting and selecting rectangles in the captured images;

wherein said parking space detection algorithm employs a recursive temporal stabilization mechanism that predicts parking space markers when the parking space markers of a detected parking space disappear from the field of view of said camera; and

wherein said control, responsive to identification of an empty or available parking space, at least in part controls the vehicle to park the vehicle in the identified empty or available parking space.

2. The parking assist system of claim 1, wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements.

3. The parking assist system of claim 2, wherein said camera comprises a grayscale camera.

4. The parking assist system of claim 1, wherein the detected rectangles are processed to construct hypothesized parking rectangles, whereby said parking space detection algorithm selects an optimal rectangle using machine learning concepts.

5. The parking assist system of claim 1, wherein said camera comprises a front camera and wherein said image processor processes image data captured by said front camera to detect parking space markers indicative of a parking space forward of the vehicle and to identify empty or available parking spaces forward of the vehicle.

6. The parking assist system of claim 1, wherein said camera is part of a multi-camera vision system of the vehicle, and wherein said multi-camera vision system comprises a plurality of cameras disposed at the vehicle so as to have respective fields of view exterior the vehicle.

7. The parking assist system of claim 6, wherein said plurality of cameras comprises a front camera disposed at a forward portion of the vehicle and having a field of view forward of the vehicle, a driver-side camera disposed at a driver side of the vehicle and having a field of view sideward of the vehicle, a passenger-side camera disposed at a passenger side of the vehicle and having a field of view sideward of the vehicle, and a rear camera disposed at a rear portion of the vehicle and having a field of view rearward of the vehicle.

8. The parking assist system of claim 7, wherein said image processor processes image data captured by at least one of said front camera and said rear camera to detect parking space markers indicative of a parking space and to identify empty or available parking spaces.

9. The parking assist system of claim 1, wherein the recursive temporal stabilization mechanism produces a stable rectangle parking space and predicts rectangle parking spaces when the parking space markers of a detected parking space disappear from the field of view of said camera.

10. A parking assist system of a vehicle, said parking assist system comprising:

a plurality of cameras configured to be disposed at a vehicle so as to have respective fields of view exterior of the vehicle;

a control comprising an image processor operable to process image data captured by said cameras when said cameras are disposed at the vehicle;

wherein said control, responsive to processing by said image processor of captured image data, (i) detects parking space markers indicative of a parking space and (ii) identifies empty or available parking spaces; and

wherein said control uses a parking space detection algorithm that detects parking space markers by (i) extracting low level features from captured image data, (ii) classifying pixels as being part of a parking space line or not part of a parking space line, (iii) performing spatial line fitting to find lines in the captured images and to apply parking space geometry constraints, and (iv) detecting and selecting rectangles in the captured images;

wherein said parking space detection algorithm employs a recursive temporal stabilization mechanism that predicts parking space markers when the parking space markers of a detected parking space disappear from the field of view of said cameras; and

wherein said control, responsive to identification of an empty or available parking space, at least in part controls the vehicle to park the vehicle in the identified empty or available parking space.

11. The parking assist system of claim 10, wherein each of said cameras comprises a pixelated imaging array having a plurality of photosensing elements.

12. The parking assist system of claim 11, wherein each of said cameras comprises a grayscale camera.

13. The parking assist system of claim 10, wherein the detected rectangles are processed to construct hypothesized parking rectangles, whereby said parking space detection algorithm selects an optimal rectangle using machine learning concepts.

14. The parking assist system of claim 10, wherein said image processor processes image data captured by at least one of a front camera of said plurality of cameras and a rear camera of said plurality of cameras to detect parking space markers indicative of a parking space forward or rearward of the vehicle and to identify empty or available parking spaces forward or rearward of the vehicle.

15. The parking assist system of claim 10, wherein said cameras are part of a multi-camera vision system of the vehicle, and wherein image data captured by said cameras is used to provide a surround vision display for viewing by the driver of the vehicle.

16. The parking assist system of claim 10, wherein said plurality of cameras comprises a front camera disposed at a forward portion of the vehicle and having a field of view forward of the vehicle, a driver-side camera disposed at a driver side of the vehicle and having a field of view sideward of the vehicle, a passenger-side camera disposed at a passenger side of the vehicle and having a field of view sideward of the vehicle, and a rear camera disposed at a rear portion of the vehicle and having a field of view rearward of the vehicle.

17. The parking assist system of claim 10, wherein the recursive temporal stabilization mechanism produces a stable rectangle parking space and predicts rectangle parking spaces when the parking space markers of a detected parking space disappear from the field of view of said cameras.

18. A parking assist system of a vehicle, said parking assist system comprising:

a plurality of cameras configured to be disposed at a vehicle so as to have respective fields of view exterior of the vehicle;

wherein said plurality of cameras comprises at least a front camera disposed at a forward portion of the vehicle and having a field of view forward of the vehicle, and a rear camera disposed at a rear portion of the vehicle and having a field of view rearward of the vehicle;

an image processor operable to process image data captured by said front and rear cameras when said cameras are disposed at the vehicle;

wherein said image processor processes captured image data to (i) detect parking space markers indicative of a parking space and (ii) identify empty or available parking spaces;

wherein said image processor comprises a parking space detection algorithm that detects parking space markers by (i) extracting low level features from captured image data, (ii) classifying pixels as being part of a parking space line or not part of a parking space line, (iii) performing spatial line fitting to find lines in the captured images and to apply parking space geometry constraints, and (iv) detecting and selecting rectangles in the captured images;

wherein the detected rectangles are processed to construct hypothesized parking rectangles, whereby said parking space detection algorithm selects an optimal rectangle using machine learning concepts;

wherein said parking space detection algorithm employs a recursive temporal stabilization mechanism that produces a stable rectangle parking space and that predicts parking lines and rectangles when the parking space markers of a detected parking space disappear from the field of view of said cameras; and

a control that, responsive to identification of an empty or available parking space, at least in part controls the vehicle to park the vehicle in the identified empty or available parking space.

19. The parking assist system of claim 18, wherein each of said cameras comprises a pixelated imaging array having a plurality of photosensing elements, and wherein each of said cameras comprises a grayscale camera.

20. The parking assist system of claim 18, wherein the detected rectangles are processed to construct hypothesized parking rectangles, whereby said parking space detection algorithm selects an optimal rectangle using machine learning concepts.