Image analysis technologies for identifying abnormal vehicle conditions

• US 10,275,670 B1
• Filed: 03/07/2018
• Issued: 04/30/2019
• Est. Priority Date: 03/07/2018
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Li, Yuntao, Zhang, Dingchao, Myers, Jeffrey S.
• Primary Examiner(s): Feild, Joseph H
• Assistant Examiner(s): Akhter, Sharmin
• Application Number:
  US15/914,745
• Time in Patent Office: 419 Days
• Field of Search: 340457, 3405731-576
• CPC Class Codes: B60R 25/102 a signal being sent to a remote location, e.g. a radio signal being transmitted to a police station, a security company or the owner
• CPC Class Codes: B60R 25/25 using biometry
• CPC Class Codes: B60R 25/305 using a camera
• CPC Class Codes: G06V 20/593 Recognising seat occupancy
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G08B 21/22 responsive to presence or absence of persons

Abstract

Systems and methods for using image analysis techniques to assess abnormal vehicle operating conditions are disclosed. According to aspects, a computing device may access and analyze image data depicting an individual(s) within a vehicle. Based on the depicted individuals(s) and optionally on other data, the computing device may determine that an abnormal condition exists. In response, the computing device may generate a notification and transmit the notification to an electronic device of an individual associated with the vehicle.

Claims

1. A system for assessing abnormal vehicle occupancy conditions within a vehicle, comprising:

a transceiver;

at least one image sensor configured to capture image data;

a non-transitory memory storing a set of computer-executable instructions; and

a processor communicatively coupled to the transceiver, the at least one image sensor, and the memory, and configured to execute the computer-executable instructions to cause the processor to;

access the image data from the at least one image sensor, the image data comprising a plurality of image frames depicting an interior of the vehicle,

determine, from the plurality of image frames, at least a portion of image frames that depict a plurality of faces of a plurality of individuals located within the vehicle,

analyze at least the portion of image frames to determine a plurality of positions within the vehicle at which the plurality of individuals are respectively located, wherein the plurality of positions indicates a first individual of the plurality of individuals located in a driver'"'"'s seat of the vehicle and a remainder of the plurality of individuals located in a back seat of the vehicle,

determine, based on the plurality of positions within the vehicle at which the plurality of individuals are respectively located, that an abnormal condition exists, and

in response to determining that the abnormal condition exists;

generate a notification indicating the abnormal condition, and

transmit, via the transceiver, the notification to an electronic device.

2. The system of claim 1, wherein the processor is further configured to:

analyze at least the portion of image frames to determine that an individual of the plurality of individuals located within the vehicle is unauthorized.

3. The system of claim 2, wherein to analyze at least the portion of image frames to determine that the individual of the plurality of individuals located within the vehicle is unauthorized, the processor is configured to:

determine, using a facial recognition technique on at least the portion of image frames that depict the face of the individual, a facial recognition result, and

determine that the facial recognition result is not included in a listing of authorized individuals for the vehicle.

4. The system of claim 2, wherein to determine that the facial recognition result is not included in the listing of authorized individuals for the vehicle, the processor is configured to:

transmit the facial recognition result to a backend server via the transceiver, the backend server configured to check the facial recognition result against the listing of authorized individuals for the vehicle, and

receive, from the backend server via the transceiver, an indication that the facial recognition result is not included in the listing of authorized individuals for the vehicle.

5. The system of claim 2, wherein the notification further indicates that the individual located within the vehicle is unauthorized, and wherein the electronic device is associated with an owner of the vehicle.

6. The system of claim 1, wherein the processor is further configured to:

determine that an amount of the plurality of individuals located within the vehicle exceeds a normal amount of individuals associated with the vehicle.

7. The system of claim 6, wherein to determine that the amount of the plurality of individuals located within the vehicle exceeds the normal amount of individuals associated with the vehicle, the processor is configured to:

access a policy associated with the vehicle, the policy indicating the normal amount of individuals associated with the vehicle, and

determine that the amount of the plurality of individuals located within the vehicle exceeds the normal amount of individuals associated with the vehicle.

8. The system of claim 1, wherein the processor is further configured to:

analyze at least the portion of image frames to estimate an age of an individual of the plurality of individuals, wherein the age of the individual is less than a threshold age.

9. The system of claim 8, wherein the processor is further configured to:

in response to estimating the age of the individual;

access a set of sensor data associated with an environment of the vehicle, and

determine that at least one parameter associated with the set of sensor data meets or exceeds at least one condition.

10. The system of claim 9, wherein the set of sensor data comprises an outside temperature and an operating condition of the vehicle, and wherein to determine that the at least one parameter associated with the set of sensor data meets or exceeds at least one condition, the processor is configured to:

determine that (i) the outside temperature exceeds a threshold temperature, and (ii) the operating condition of the vehicle is off.

11. A computer-implemented method of assessing abnormal vehicle occupancy conditions within a vehicle, the method comprising:

accessing image data from at least one image sensor, the image data comprising a plurality of image frames depicting an interior of the vehicle;

determining, by a processor from the plurality of image frames, at least a portion of image frames that depict a plurality of faces of a plurality of individuals located within the vehicle;

analyzing, by the processor, at least the portion of image frames to determine a plurality of positions within the vehicle at which the plurality of individuals are respectively located;

wherein the plurality of positions indicates a first individual of the plurality of individuals located in a driver'"'"'s seat of the vehicle and a remainder of the plurality of individuals located in a back seat of the vehicle,

determining, by the processor based on the plurality of positions within the vehicle at which the plurality of individuals are respectively located, that an abnormal condition exists; and

in response to determining that the abnormal condition exists;

generating a notification indicating the abnormal condition, and

transmitting, via a network connection, the notification to an electronic device.

12. The computer-implemented method of claim 11, further comprising:

analyzing at least the portion of image frames to determine that an individual of the plurality of individuals located within the vehicle is unauthorized.

13. The computer-implemented method of claim 12, analyzing at least the portion of image frames to determine that the individual of the plurality of individuals located within the vehicle is unauthorized comprises:

determining, using a facial recognition technique on at least the portion of image frames that depict the face of the individual, a facial recognition result; and

determining that the facial recognition result is not included in a listing of authorized individuals for the vehicle.

14. The computer-implemented method of claim 12, wherein determining that the facial recognition result is not included in the listing of authorized individuals for the vehicle comprises:

transmitting the facial recognition result to a backend server, the backend server configured to check the facial recognition result against the listing of authorized individuals for the vehicle; and

receiving, from the backend server, an indication that the facial recognition result is not included in the listing of authorized individuals for the vehicle.

15. The computer-implemented method of claim 12, wherein the notification further indicates that the individual located within the vehicle is unauthorized, and wherein the electronic device is associated with an owner of the vehicle.

16. The computer-implemented method of claim 11, further comprising:

determining that an amount of the plurality of individuals located within the vehicle exceeds a normal amount of individuals associated with the vehicle.

17. The computer-implemented method of claim 16, wherein determining that the amount of the plurality of individuals located within the vehicle exceeds the normal amount of individuals associated with the vehicle comprises:

accessing a policy associated with the vehicle, the policy indicating the normal amount of individuals associated with the vehicle; and

determining that the amount of the plurality of individuals located within the vehicle exceeds the normal amount of individuals associated with the vehicle.

18. The computer-implemented method of claim 11, further comprising:

analyzing at least the portion of image frames to estimate an age of an individual of the plurality of individuals, wherein the age of the individual is less than a threshold age.

19. The computer-implemented method of claim 18, further comprising:

in response to estimating the age of the individual;

accessing a set of sensor data associated with an environment of the vehicle, and

determining that at least one parameter associated with the set of sensor data meets or exceeds at least one condition.

20. The computer-implemented method of claim 19, wherein the set of sensor data comprises an outside temperature and an operating condition of the vehicle, and wherein determining that the at least one parameter associated with the set of sensor data meets or exceeds at least one condition comprises:

determining that (i) the outside temperature exceeds a threshold temperature, and (ii) the operating condition of the vehicle is off.