Pedestrian detection when a vehicle is reversing

• US 10,150,414 B2
• Filed: 07/08/2016
• Issued: 12/11/2018
• Est. Priority Date: 07/08/2016
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Myers, Scott Vincent, Gurghian, Alexandru Mihai, Micks, Ashley Elizabeth, Karandikar, Ramchandra Ganesh
• Primary Examiner(s): Wu, Zhen Y
• Application Number:
  US15/205,701
• Publication Number:
  US 20180009378A1
• Time in Patent Office: 886 Days
• Field of Search: 340435
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60R 2300/307 virtually distinguishing relevant parts of a scene from the background of the scene
• CPC Class Codes: B60R 2300/8066 for monitoring rearward traffic
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

Techniques and implementations pertaining to detection of moving objects, such as pedestrians, when a vehicle moves in a rearward direction are described. A method may involve identifying a region of interest when a vehicle moves in a rearward direction. The method may involve detecting a moving object in the region of interest. The method may also involve determining whether a collision with the moving object by the vehicle moving in the rearward direction is likely. The method may further involve providing a human-perceivable signal responsive to a determination that the collision is likely.

Claims

1. A method, comprising:

identifying, by a processor of a vehicle, a region of interest when a vehicle moves in a rearward direction;

detecting, by the processor based on data received from one or more sensors on the vehicle, a moving object in the region of interest;

determining, by the processor, whether a collision with the moving object by the vehicle moving in the rearward direction is likely; and

providing, by the processor, a human-perceivable signal responsive to a determination that the collision is likely,

wherein the identifying of the region of interest comprises localizing the region of interest using one or more images captured by a rearward-facing camera with drive history data and one or more digital maps plus at least one of;

global positioning system (GPS) data,

data from one or more wirelessly-connected devices, and

data from one or more wirelessly-connected sensors, and

wherein the providing of the human-perceivable signal comprises varying a tone of an auditory signal to indicate whether the moving object is moving closer to or away from the vehicle.

2. The method of claim 1, wherein the identifying of the region of interest further comprises refining the localized region of interest using a light-detection-and-ranging (LIDAR) sensor, a radar sensor, an ultrasound sensor, or any combination thereof.

3. The method of claim 1, wherein the detecting of the moving object in the region of interest comprises analyzing one or more images captured by a rearward-facing camera using a machine learning algorithm.

4. The method of claim 3, wherein the machine learning algorithm comprises a deep neural network.

5. The method of claim 1, wherein the determining of whether a collision with the moving object by the vehicle moving in the rearward direction is likely comprises:

tracking a movement of the moving object;

tracking a distance between the moving object and the vehicle using the data received from the one or more sensors and information on one or more operational parameters of the vehicle;

determining a trajectory of the vehicle; and

determining whether the collision with the moving object by the vehicle is likely based on the movement of the moving object, the distance between the moving object and the vehicle, and the trajectory of the vehicle.

6. The method of claim 1, wherein the providing of the human-perceivable signal comprises:

providing the human-perceivable signal which comprises either or both of the auditory signal and a visual signal;

tracking a distance between the moving object and the vehicle using the data received from the one or more sensors and information on one or more operational parameters of the vehicle; and

varying one or more aspects of the human-perceivable signal responsive to a change in the distance between the moving object and the vehicle.

7. The method of claim 6, wherein the tracking of the distance between the moving object and the vehicle comprises tracking the distance between the moving object and the vehicle using information on one or more of a steering angle of the vehicle, a speed of the vehicle, and a current distance between the moving object and the vehicle using a rearward-facing camera, a light-detection-and-ranging (LIDAR) sensor, a radar sensor, an ultrasound sensor, or any combination thereof.

8. The method of claim 6, wherein the varying of the one or more aspects of the human-perceivable signal comprises:

varying at least one of a frequency and a volume of the human-perceivable signal in addition to the tone when the human-perceivable signal comprises the auditory signal; and

varying one or more of a color, a brightness, and a frequency of the human-perceivable signal when the human-perceivable signal comprises the visual signal.

9. The method of claim 1, further comprising:

displaying an elevated view of a surrounding of the vehicle to show the vehicle and the moving object; and

highlighting the moving object in the displaying of the elevated view.

10. An apparatus implementable in a vehicle, comprising:

a camera configured to capture one or more images in a rearward direction with respect to the vehicle; and

one or more processors operatively coupled to the camera, the one or more processors configured to perform operations comprising;

identifying, based at least in part on the one or more images, a region of interest when the vehicle moves in a rearward direction;

detecting a moving object in the region of interest;

determining whether a collision with the moving object by the vehicle moving in the rearward direction is likely; and

providing a human-perceivable signal responsive to a determination that the collision is likely,

wherein, in identifying the region of interest, the one or more processors are configured to localize the region of interest using the one or more images captured by the camera with drive history data and one or more digital maps plus at least one of;

global positioning system (GPS) data,

data from one or more wirelessly-connected devices, and

from one or more wirelessly-connected sensors.

11. The apparatus of claim 10, further comprising:

one or more of a light-detection-and-ranging (LIDAR) sensor, a radar sensor, and an ultrasound sensor,

wherein, in identifying the region of interest, the one or more processors are further configured to refine the localized region of interest using one or more of the LIDAR sensor, the radar sensor, and the ultrasound sensor.

12. The apparatus of claim 10, wherein, in detecting the moving object in the region of interest, the one or more processors are configured to analyze the one or more images captured by the camera using a machine learning algorithm.

13. The apparatus of claim 12, wherein the machine learning algorithm comprises a deep neural network.

14. The apparatus of claim 10, wherein, in determining whether a collision with the moving object by the vehicle moving in the rearward direction is likely, the one or more processors are configured to perform operations comprising:

tracking a movement of the moving object;

tracking a distance between the moving object and the vehicle;

determining a trajectory of the vehicle; and

determining whether the collision with the moving object by the vehicle is likely based on the movement of the moving object, the distance between the moving object and the vehicle, and the trajectory of the vehicle.

15. The apparatus of claim 10, wherein, in providing the human-perceivable signal, the one or more processors are configured to perform operations comprising:

providing the human-perceivable signal which comprises either or both of an auditory signal and a visual signal;

tracking a distance between the moving object and the vehicle; and

varying one or more aspects of the human-perceivable signal responsive to a change in the distance between the moving object and the vehicle.

16. The apparatus of claim 15, wherein, in tracking the distance between the moving object and the vehicle, the one or more processors are configured to track the distance between the moving object and the vehicle using information on one or more of a steering angle of the vehicle, a speed of the vehicle, and a current distance between the moving object and the vehicle using the camera, a light-detection-and-ranging (LIDAR) sensor, a radar sensor, an ultrasound sensor, or any combination thereof.

17. The apparatus of claim 15, wherein, in varying the one or more aspects of the human-perceivable signal, the one or more processors are configured to perform operations comprising:

varying one or more of a tone, a frequency, and a volume of the human-perceivable signal when the human-perceivable signal comprises the auditory signal; and

varying one or more of a color, a brightness, and a frequency of the human-perceivable signal when the human-perceivable signal comprises the visual signal.

18. The apparatus of claim 10, further comprising:

a display device,

wherein the one or more processors are further configured to perform operations comprising;

displaying, on the display device, an elevated view of a surrounding of the vehicle to show the vehicle and the moving object; and

highlighting, on the display device, the moving object in the displaying of the elevated view.