Detecting physical threats approaching a vehicle

• US 10,139,827 B2
• Filed: 06/28/2016
• Issued: 11/27/2018
• Est. Priority Date: 06/28/2016
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Charette, Francois, Myers, Scott Vincent, Scaria, Lisa, Gurghian, Alexandru Mihai
• Primary Examiner(s): Weber, Tamara L
• Application Number:
  US15/195,827
• Publication Number:
  US 20170371339A1
• Time in Patent Office: 882 Days
• Field of Search: None
• CPC Class Codes: B60R 21/12 which protect the occupants against personal attack from the inside or the outside of the vehicle
• CPC Class Codes: B60R 25/10 actuating a signalling device
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 10/30 including control of auxiliary equipment, e.g. air-conditioning compressors or oil pumps
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2420/54 Audio sensitive means, e.g. ultrasound
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2720/24 Direction of travel
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 50/0098 Details of control systems ensuring comfort, safety or stability not otherwise provided for
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B60W 60/0016 of the vehicle or its occupants
• CPC Class Codes: G08B 13/19613 Recognition of a predetermined image pattern or behaviour pattern indicating theft or intrusion
• CPC Class Codes: G08B 13/19647 Systems specially adapted for intrusion detection in or around a vehicle

Abstract

The present invention extends to methods, systems, and computer program products for detecting physical threats approaching a vehicle. External sensors on a vehicle capture the environment around the vehicle. Approaching targets detected by the external sensors can be fed into a neural network to recognize and/or classify approaching targets as potential threats. Tracking mechanisms (e.g., Kalman filters, Particle filters, etc.) can leverage temporal information to determine if a threat is approaching a vehicle. When an approaching threat is detected, a vehicle can activate one or more counter measures to deter the threat. When a vehicle includes autonomous driving capabilities, counter measures can include automatically attempting to drive away from an approaching threat.

Claims

1. A method for use at a vehicle, the method comprising:

determining that the path of a first object is likely to cause the first object and a corresponding second object to travel near the vehicle based on filtered sensor data from one or more sensors externally mounted on the vehicle;

providing the filtered sensor data as input to a neural network;

receiving a threat classification from the neural network classifying the first object and the corresponding second object collectively as a non-vehicular threat to an occupant of the vehicle;

indicating the threat classification in the vehicle cabin; and

automatically maneuvering the vehicle to get away from the first object and the second object.

2. The method of claim 1, wherein receiving a threat classification comprises receiving an indication that the first object is a person and the corresponding second object is a weapon possessed by the person.

3. The method of claim 1, wherein receiving a threat classification comprises receiving an indication that a weapon is approaching the vehicle.

4. The method of claim 1, wherein receiving a threat classification comprises receiving an indication that the first object is a person and the corresponding second object is a mask worn by the person.

5. The method of claim 1, further comprising activating counter measures to deter the non-vehicular threat.

6. The method of claim 1, wherein receiving a threat classification comprises receiving an indication that the first object is a dog and the corresponding second object is the dog'"'"'s teeth.

16. The method of claim 1, further comprising automatically activating counter measures at the vehicle including one or more of:

recording the first object, locking the doors of the vehicle, calling emergency services, issuing an audible warning in the vehicle cabin, honking a horn of the vehicle, or flashing lights of the vehicle.

7. A method for use at a vehicle, the method comprising:

determining that the path of an object is likely to cause the object to travel near to the vehicle, including;

using one or more sensors mounted to the vehicle to monitor an area in proximity to the vehicle for approaching objects; and

filtering data from the one or more sensors on a heterogeneous computing platform at the vehicle to determine that the object has a speed and direction indicative of the object approaching space occupied by the vehicle;

providing the filtered data for the approaching object as input to a neural network;

receiving an indication from the neural network that the approaching object and a second object associated with the approaching object collectively represent a non-vehicular threat to a vehicle occupant based at least in part on identification of the second object; and

activating counter measures at the vehicle to address the non-vehicular threat, including automatically attempting to maneuver the vehicle to take the vehicle occupant away from both the approaching object and the second object.

8. The method of claim 7, wherein activating counter measures at the vehicle to address the non-vehicular threat comprises automatically making a phone call to emergency services.

9. The method of claim 7, wherein receiving an indication from the neural network that the approaching object and a second object associated with the approaching object collectively represents a non-vehicular threat comprises receiving a threat probability from the neural network.

10. The method of claim 7, wherein using one or more sensors mounted to the vehicle to monitor an area in proximity to the vehicle for approaching objects comprises using at least two of:

a camera, a LIDAR sensor, and a sound based object detector to monitor an area in proximity to the vehicle for approaching objects.

11. The method of claim 10, wherein filtering data from the one or more sensors on a heterogeneous computing platform comprises filtering data from the plurality of sensors on at least two of:

a Central Processing Unit (CPU), a Graphical Processing Unit (GPU), and a Field Programmable Gate Array (FPGA) in the vehicle.

12. The method of claim 7, wherein filtering data from the one or more sensors comprises filtering the data using one or more of:

a Kalman filter and a Particle Filter.

13. The method of claim 7, wherein filtering data from the one or more sensors comprises determining an optical flow of the non-vehicular threat.

14. The method of claim 7, wherein receiving an indication from the neural network that the approaching object and a second object associated with the approaching object collectively represent a non-vehicular threat to the vehicle occupant comprises receiving an indication that the approaching object is a person and the associated second object is one of:

a weapon in possession of the person or a mask in possession of the person.

15. The method of claim 7, wherein activating counter measures at the vehicle comprises activating counter measures including one or more of:

recording the approaching object, locking the doors of the vehicle, issuing an audible warning in the vehicle cabin, honking a horn of the vehicle, or flashing lights of the vehicle.

17. A vehicle, the vehicle comprising:

one or more externally mounted sensors for monitoring an area in proximity to the vehicle;

one or more processors;

system memory coupled to one or more processors, the system memory storing instructions that are executable by the one or more processors;

the one or more processors configured to execute the instructions stored in the system memory to;

determine that the path of an object is likely to cause the object and a second object to travel near to the vehicle, including;

use the one or more externally mounted sensors to monitor the area in proximity to the vehicle for approaching objects; and

filter data from the one or more sensors to determine that the object has a speed and direction indicative of the object approaching space occupied by the vehicle;

provide the filtered data as input to a neural network;

receive a threat probability from the neural network indicating a probability that the object and the second object collectively represent a non-vehicular threat to a vehicle occupant; and

control one or more vehicle components, wherein the one or more vehicle components maneuver the vehicle to get away from both the object and the second object.

18. The vehicle of claim 17, wherein the one or more externally mounted sensors include one or more of:

a camera, a LIDAR sensor, a RADAR sensor, and an ultrasonic sensor.

19. The vehicle of claim 17, wherein the one or more processors configured to execute the instructions to receive an indication from the neural network that the approaching object represents a non-vehicular threat to the vehicle occupants comprises the one or more processors configured to execute the instructions to receive an indication that the approaching object is a person and the associated second object is one of:

a weapon in possession of the person or a mask in possession of the person.

20. The vehicle of claim 17, wherein the one or more processors configured to execute the instructions to activate counter measures at the vehicle comprises the one or more processors configured to execute the instructions to activate counter measures including one or more of:

recording the object, locking the doors of the vehicle, issuing an audible warning in the vehicle cabin, honking a horn of the vehicle, or flashing lights of the vehicle.