Image analysis technologies for identifying abnormal vehicle conditions

• US 10,783,386 B1
• Filed: 02/12/2019
• Issued: 09/22/2020
• Est. Priority Date: 03/07/2018
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Li, Yuntao, Zhang, Dingchao, Myers, Jeffrey S.
• Primary Examiner(s): Feild, Joseph H
• Assistant Examiner(s): Akhter, Sharmin
• Application Number:
  US16/273,642
• Time in Patent Office: 588 Days
• Field of Search: 340457, 3405731-576
• CPC Class Codes: B60R 25/102 a signal being sent to a remote location, e.g. a radio signal being transmitted to a police station, a security company or the owner
• CPC Class Codes: B60R 25/25 using biometry
• CPC Class Codes: B60R 25/305 using a camera
• CPC Class Codes: G06V 20/593 Recognising seat occupancy
• CPC Class Codes: G06V 40/16 Human faces, e.g. facial parts, sketches or expressions
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G08B 21/22 responsive to presence or absence of persons

Abstract

Systems and methods for using image analysis techniques to assess abnormal vehicle operating conditions are disclosed. According to aspects, a computing device may access and analyze image data depicting an individual(s) within a vehicle. Based on the depicted individuals(s) and optionally on other data, the computing device may determine that an abnormal condition exists. In response, the computing device may generate a notification and transmit the notification to an electronic device of an individual associated with the vehicle.

Claims

1. A system for assessing abnormal vehicle occupancy conditions within a vehicle, comprising:

a transceiver;

at least one image sensor configured to capture image data;

a non-transitory memory storing a set of computer-executable instructions; and

a processor communicatively coupled to the transceiver, the at least one image sensor, and the memory, and configured to execute the computer-executable instructions to cause the processor to;

access the image data from the at least one image sensor, the image data depicting a plurality of individuals located within the vehicle, including a first individual located in a driver seat of the vehicle and a set of additional individuals,

analyze the image data to determine that the first individual located in the driver seat is authorized,

in response to determining that the first individual located in the driver seat is authorized, analyze at least a portion of the image data depicting at least the set of additional individuals to determine that an abnormal condition exists, the abnormal condition associated with at least the set of additional individuals located within the vehicle, and

in response to determining that the abnormal condition exists;

generate a notification indicating the abnormal condition, and

transmit, via the transceiver, the notification to an electronic device.

2. The system of claim 1, wherein the image data comprises a plurality of image frames, and wherein the processor is further configured to:

determine, from the plurality of image frames, at least a portion of image frames that depict a face of the first individual located in the driver seat.

3. The system of claim 2, wherein to analyze the image data to determine that the first individual located in the driver seat is authorized, the processor is configured to:

determine, using a facial recognition technique on at least the portion of image frames that depict the face of the first individual, a facial recognition result, and

determine that the facial recognition result is included in a listing of authorized individuals for the vehicle.

4. The system of claim 3, wherein to determine that the facial recognition result is included in the listing of authorized individuals for the vehicle, the processor is configured to:

transmit the facial recognition result to a backend server via the transceiver, the backend server configured to check the facial recognition result against the listing of authorized individuals for the vehicle, and

receive, from the backend server via the transceiver, an indication that the facial recognition result is included in the listing of authorized individuals for the vehicle.

5. The system of claim 1, wherein to analyze at least the portion of the image data to determine that the abnormal condition exists, the processor is configured to:

determine that an amount of the set of additional individuals located within the vehicle exceeds a normal amount of individuals associated with the vehicle.

6. The system of claim 5, wherein to determine that the amount of the set of additional individuals located within the vehicle exceeds the normal amount of individuals associated with the vehicle, the processor is configured to:

access a policy associated with the vehicle, the policy indicating the normal amount of individuals associated with the vehicle, and

determine that the amount of the set of additional individuals located within the vehicle exceeds the normal amount of individuals associated with the vehicle.

7. The system of claim 1, wherein to analyze at least the portion of the image data to determine that the abnormal condition exists, the processor is configured to:

analyze at least the portion of the image data to estimate an age of an additional individual of the set of additional individuals, wherein the age of the additional individual is less than a threshold age.

8. A computer-implemented method of assessing abnormal vehicle occupancy conditions within a vehicle, the method comprising:

accessing image data from at least one image sensor, the image data depicting a plurality of individuals located within the vehicle, including a first individual located in a driver seat of the vehicle and a set of additional individuals;

analyzing, by a processor, the image data to determine that the first individual located in the driver seat is authorized;

in response to determining that the first individual located in the driver seat is authorized, analyzing, by the processor, at least a portion of the image data depicting at least the set of additional individuals to determine that an abnormal condition exists, the abnormal condition associated with at least the set of additional individuals located within the vehicle; and

in response to determining that the abnormal condition exists;

generating a notification indicating the abnormal condition, and

transmitting, via a network connection, the notification to an electronic device.

9. The computer-implemented method of claim 8, wherein the image data comprises a plurality of image frames, and wherein the processor is further configured to method further comprises:

determining, from the plurality of image frames, at least a portion of image frames that depict a face of the first individual located in the driver seat.

10. The computer-implemented method of claim 9, wherein analyzing the image data to determine that the first individual located in the driver seat is authorized comprises:

determining, using a facial recognition technique on at least the portion of image frames that depict the face of the first individual, a facial recognition result, and

determining that the facial recognition result is included in a listing of authorized individuals for the vehicle.

11. The computer-implemented method of claim 10, wherein determining that the facial recognition result is included in the listing of authorized individuals for the vehicle comprises:

transmitting the facial recognition result to a backend server via the transceiver, the backend server configured to check the facial recognition result against the listing of authorized individuals for the vehicle, and

receiving, from the backend server via the transceiver, an indication that the facial recognition result is included in the listing of authorized individuals for the vehicle.

12. The computer-implemented method of claim 8, wherein analyzing at least the portion of the image data to determine that the abnormal condition exists comprises:

determining that an amount of the set of additional individuals located within the vehicle exceeds a normal amount of individuals associated with the vehicle.

13. The computer-implemented method of claim 12, wherein determining that the amount of the set of additional individuals located within the vehicle exceeds the normal amount of individuals associated with the vehicle comprises:

accessing a policy associated with the vehicle, the policy indicating the normal amount of individuals associated with the vehicle, and

determining that the amount of the set of additional individuals located within the vehicle exceeds the normal amount of individuals associated with the vehicle.

14. The computer-implemented method of claim 8, wherein analyzing at least the portion of the image data to determine that the abnormal condition exists comprises:

analyzing at least the portion of the image data to estimate an age of an additional individual of the set of additional individuals, wherein the age of the additional individual is less than a threshold age.

15. A non-transitory computer-readable medium, comprising:

a plurality of instructions which, when executed by one or more processors, cause the one or more processors to;

access image data from at least one image sensor, the image data depicting a plurality of individuals located within the vehicle, including a first individual located in a driver seat of the vehicle and a set of additional individuals,

analyze the image data to determine that the first individual located in the driver seat is authorized,

in response to determining that the first individual located in the driver seat is authorized, analyze at least a portion of the image data depicting at least the set of additional individuals to determine that an abnormal condition exists, the abnormal condition associated with at least the set of additional individuals located within the vehicle, and

in response to determining that the abnormal condition exists;

generate a notification indicating the abnormal condition, and

transmit the notification to an electronic device.

16. The non-transitory computer-readable medium of claim 15, wherein the image data comprises a plurality of image frames, and wherein the plurality of instructions further cause the one or more processors to:

determine, from the plurality of image frames, at least a portion of image frames that depict a face of the first individual located in the driver seat.

17. The non-transitory computer-readable medium of claim 16, wherein one or more instructions, of the plurality of instructions, to analyze the image data to determine that the first individual located in the driver seat is authorized cause the one or more processors to:

determine, using a facial recognition technique on at least the portion of image frames that depict the face of the first individual, a facial recognition result, and

determine that the facial recognition result is included in a listing of authorized individuals for the vehicle.

18. The non-transitory computer-readable medium of claim 17, wherein one or more instructions, of the plurality of instructions, to determine that the facial recognition result is included in the listing of authorized individuals for the vehicle cause the one or more processors to:

transmit the facial recognition result to a backend server via the transceiver, the backend server configured to check the facial recognition result against the listing of authorized individuals for the vehicle, and

receive, from the backend server via the transceiver, an indication that the facial recognition result is included in the listing of authorized individuals for the vehicle.

19. The non-transitory computer-readable medium of claim 15, wherein one or more instructions, of the plurality of instructions, to analyze at least the portion of the image data to determine that the abnormal condition exists cause the one or more processors to:

determine that an amount of the set of additional individuals located within the vehicle exceeds a normal amount of individuals associated with the vehicle.

20. The non-transitory computer-readable medium of claim 19, wherein one or more instructions, of the plurality of instructions, to determine that the amount of the set of additional individuals located within the vehicle exceeds the normal amount of individuals associated with the vehicle cause the one or more processors to:

access a policy associated with the vehicle, the policy indicating the normal amount of individuals associated with the vehicle, and

determine that the amount of the set of additional individuals located within the vehicle exceeds the normal amount of individuals associated with the vehicle.