SYSTEMS AND METHODS FOR SURVEILLANCE OF A VEHICLE USING CAMERA IMAGES

• US 20180272992A1
• Filed: 03/24/2017
• Published: 09/27/2018
• Est. Priority Date: 03/24/2017
• Status: Active Grant

• Current Assignee: Denso International America Incorporated (DENSO Corporation), Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Denso International America Incorporated (DENSO Corporation), Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Gage, Sergei I., Gloomis, Jay Robert, Rios, Isabel
• Granted Patent: US 10,421,436 B2
• Time in Patent Office: Days
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/307 virtually distinguishing relevant parts of a scene from the background of the scene
• CPC Class Codes: B60R 2300/8073 for vehicle security, e.g. parked vehicle surveillance, burglar detection
• CPC Class Codes: B60R 25/102 a signal being sent to a remote location, e.g. a radio signal being transmitted to a police station, a security company or the owner
• CPC Class Codes: B60R 25/305 using a camera
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 10/95 structured as a network, e.g. client-server architectures
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle

Abstract

System, methods, and other embodiments described herein relate to surveillance of a vehicle. In one embodiment, a method includes, in response to detecting a surveillance event associated with the vehicle, collecting information from one or more sensors of the vehicle about the surveillance event. The method includes classifying the information according to a security profile to identify a threat type and a threat level of the surveillance event. The method includes controlling the vehicle to provide a response to the surveillance event according to the threat type and the threat level.

Claims

1. A surveillance system for providing security for a vehicle, comprising:

one or more processors;

a memory communicably coupled to the one or more processors and storing;

a monitoring module including instructions that when executed by the one or more processors cause the one or more processors to, in response to detecting a surveillance event associated with the vehicle, collect information from one or more sensors of the vehicle about the surveillance event; and

a response module including instructions that when executed by the one or more processors cause the one or more processors to classify the information according to a security profile to identify a threat type and a threat level of the surveillance event,

wherein the response module includes instructions to control the vehicle to provide a response to the surveillance event according to the threat type and the threat level.

2. The surveillance system of claim 1, wherein the monitoring module further includes instructions to detect the surveillance event by monitoring the one or more sensors to detect an activating input associated with the surveillance event including monitoring a stream of data from the one or more sensors, and wherein the activating input is defined according to the security profile to include one or more signals from the one or more sensors that are indicative of the surveillance event.

3. The surveillance system of claim 1, wherein the monitoring module further includes instructions to collect the information by acquiring images from one or more cameras of the vehicle including activating the one or more cameras to provide the images of a surrounding environment of the vehicle, and wherein the monitoring module further includes instructions to analyze the images to identify an object in the images associated with the surveillance event and to track the object while the object moves across separate fields-of-view for the one or more cameras.

4. The surveillance system of claim 1, wherein the response module further includes instructions to classify the information to identify the threat type by (i) identifying a type of one or more objects that are proximate to the vehicle from images obtained from one or more cameras of the vehicle, and (ii) identifying characteristics of the one or more objects,

wherein the response module further includes instructions to classify the information to identify the threat level by assessing characteristics of the surveillance event according to the security profile, and wherein the characteristics include at least (i) a proximity of the one or more objects to the vehicle, (ii) the threat type, and (iii) a duration for which the one or more objects is proximate to the vehicle.

5. The surveillance system of claim 1, wherein the monitoring module further includes instructions to detect the surveillance event by receiving a wireless signal from a nearby security system indicating an occurrence of the surveillance event, and wherein the monitoring module further includes instructions to collect the information by wirelessly querying nearby sources for the information.

6. The surveillance system of claim 1, wherein the security profile indicates different activating inputs from the one or more sensors that correlate with different types of surveillance events, and wherein the security profile specifies correlations between threat levels and threat types for which different responses are to be provided by the vehicle,

wherein the threat type indicates a type of an object associated with the surveillance event, an orientation of the object, and a current speed of the object, and

wherein the type of the object specifies at least whether the object is a vehicle.

7. The surveillance system of claim 1, wherein the response module further includes instructions to control the vehicle to provide the response by determining the response from the security profile as a function of the threat type and threat level,

wherein the response module further includes instructions to control the vehicle to provide the response by transmitting video to a remote device to query the remote device about whether to generate an alarm for the surveillance event when the threat type and the threat level indicate a marginal threat.

8. The surveillance system of claim 1, wherein the response module further includes instructions to control the vehicle to provide the response by generating an alarm when the threat type and the threat level indicate a severe threat, and wherein the response module further includes instructions to control the vehicle to provide the response by logging the information for the surveillance event when the threat type and the threat level indicate a low threat.

9. A non-transitory computer-readable medium storing instructions that when executed by one or more processors cause the one or more processors to:

in response to detecting a surveillance event associated with a vehicle, collect information from one or more sensors of the vehicle about the surveillance event,

classify the information according to a security profile to identify a threat type and a threat level of the surveillance event, and

control the vehicle to provide a response to the surveillance event according to the threat type and the threat level.

10. The non-transitory computer-readable medium of claim 9, wherein the instructions to detect the surveillance event include instructions to monitor the one or more sensors to detect an activating input associated with the surveillance event including monitoring a stream of data from the one or more sensors, and wherein the activating input is defined according to the security profile to include one or more signals from the one or more sensors that are indicative of the surveillance event.

11. The non-transitory computer-readable medium of claim 9, wherein the instructions to collect the information include instructions to acquire images from one or more cameras of the vehicle including activating the one or more cameras to provide the images of a surrounding environment of the vehicle, and wherein the instructions to analyze the images include instructions to identify an object in the images associated with the surveillance event and to track the object while the object moves across separate fields-of-view for the one or more cameras.

12. The non-transitory computer-readable medium of claim 9, wherein the instructions to classify the information include instructions to identify the threat type by (i) identifying a type of one or more objects that are proximate to the vehicle from images obtained from one or more cameras of the vehicle, and (ii) identifying characteristics of the one or more objects,

wherein the instructions to classify the information include instructions to identify the threat level by assessing characteristics of the surveillance event according to the security profile, and wherein the characteristics include at least (i) a proximity of the one or more objects to the vehicle, (ii) the threat type, and (iii) a duration for which the one or more objects is proximate to the vehicle.

13. The non-transitory computer-readable medium of claim 9, wherein the instructions to detect the surveillance event include instructions to receive a wireless signal from a nearby security system indicating an occurrence of the surveillance event, and wherein the instructions to collect the information include instructions to wirelessly querying nearby sources for the information.

14. A method of providing security for a vehicle, comprising:

in response to detecting a surveillance event associated with the vehicle, collecting information from one or more sensors of the vehicle about the surveillance event;

classifying the information according to a security profile to identify a threat type and a threat level of the surveillance event; and

controlling the vehicle to provide a response to the surveillance event according to the threat type and the threat level.

15. The method of claim 14, wherein detecting the surveillance event includes monitoring the one or more sensors to detect an activating input associated with the surveillance event by monitoring a stream of data from the one or more sensors, and wherein the activating input is defined according to the security profile to include one or more signals from the one or more sensors that are indicative of the surveillance event.

16. The method of claim 14, wherein collecting the information includes acquiring images from one or more cameras of the vehicle by activating the one or more cameras to provide the images of a surrounding environment of the vehicle, and wherein acquiring the images includes analyzing the images to identify an object in the images associated with the surveillance event and tracking the object while the object moves across separate fields-of-view for the one or more cameras.

17. The method of claim 14, wherein classifying the information to identify the threat type includes (i) identifying a type of one or more objects that are proximate to the vehicle from images obtained from one or more cameras of the vehicle, and (ii) identifying characteristics of the one or more objects, wherein classifying the information to identify the threat level includes assessing characteristics of the surveillance event according to the security profile, and wherein the characteristics include at least (i) a proximity of the one or more objects to the vehicle, (ii) the threat type, and (iii) a duration for which the one or more objects is proximate to the vehicle.

18. The method of claim 14, wherein detecting the surveillance event includes receiving a wireless signal from a nearby security system indicating an occurrence of the surveillance event, and wherein collecting the information includes wirelessly querying nearby sources for the information.

19. The method of claim 15, wherein the security profile indicates different activating inputs from the one or more sensors that correlate with different types of surveillance events, and wherein the security profile specifies correlations between threat levels and threat types for which different responses are to be provided by the vehicle,

wherein the threat type indicates a type of an object associated with the surveillance event, an orientation of the object, and a current speed of the object,

wherein the type of the object specifies at least whether the object is a vehicle.

20. The method of claim 14, wherein controlling the vehicle to provide the response includes determining the response from the security profile as a function of the threat type and threat level,

wherein the response includes transmitting video to a remote device to query the remote device about whether to generate an alarm for the surveillance event when the threat type and the threat level indicate a marginal threat,

wherein the response includes generating an alarm when the threat type and the threat level indicate a severe threat, and wherein the response includes logging the information for the surveillance event when the threat type and the threat level indicate a low threat.