Technology for detecting onboard sensor tampering

• US 10,755,495 B1
• Filed: 09/25/2017
• Issued: 08/25/2020
• Est. Priority Date: 09/25/2017
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Chan, Aaron Scott, Sanchez, Kenneth J., Taylor, Gina, Thalluri, Saivikas, Yan, Jie
• Primary Examiner(s): Ziaeianmehdizadeh, Navid
• Application Number:
  US15/714,919
• Time in Patent Office: 1,065 Days
• Field of Search: None
• CPC Class Codes: G06F 11/327 Alarm or error message display
• CPC Class Codes: G07C 5/008 communicating information to a remotely located station transmission systems for measured values G08C
• CPC Class Codes: G07C 5/04 using counting means or digital clocks
• CPC Class Codes: G07C 5/0816 Indicating performance data, e.g. occurrence of a malfunction
• CPC Class Codes: G07C 5/0841 Registering performance data recording measured values G01D; information storage G11B
• CPC Class Codes: G07C 5/085 using electronic data carriers

Abstract

Systems and methods detecting onboard sensor tampering are disclosed. According to embodiments, data captured by interior sensors within a vehicle may be analyzed to determine an indication that the activity of the vehicle operator either cannot be sufficiently detected or cannot be sufficiently identified using the captured data (e.g., that the captured data may be compromised). A date and time associated with the indication may be recorded, and a vehicle operator associated with the indication may be identified. A possible cause for the compromised data may be diagnosed, and notification may be generated indicating that the activity of the vehicle operator either cannot be sufficiently detected or cannot be sufficiently identified, and/or the possible cause. Additionally, a recommendation for restoring sensor functionality may be generated for the vehicle operator based the possible cause.

Claims

1. A computer-implemented method in an electronic device for identifying interior automobile sensor tampering, the method comprising:

capturing data by one or more interior sensors within an automobile, wherein the one or more interior sensors within the automobile include one or more of;

an image sensor, a temperature sensor, an electromagnetic radiation sensor, a sound sensor, an acceleration sensor, or a velocity sensor;

analyzing, by a processor configured to detect and identify an activity of an automobile operator, the data captured by the one or more interior sensors to determine an indication that the activity of the automobile operator either cannot be detected or cannot be identified using the data captured by the one or more interior sensors;

recording a date and time associated with the indication that the activity of the automobile operator either cannot be detected or cannot be identified;

identifying, by the processor, the automobile operator associated with the indication that the activity of the automobile operator either cannot be detected or cannot be identified;

diagnosing, by the processor, a possible cause of the indication that the activity of the automobile operator either cannot be detected or cannot be identified, wherein diagnosing includes determining, based at least on a mechanical spring loaded mechanism that is activated when a plug is unplugged, that one or more of a main source of power or a backup source of power for one or more of the interior sensors has been disabled;

generating one or more of a notification that the activity of the automobile operator cannot be detected or identified, or a notification indicating the possible cause;

generating an indication for the automobile operator based on the possible cause;

transmitting, to a server, a log of one or more of the indication that the activity of the automobile operator either cannot be detected or cannot be identified, the date and time, the identity of the automobile operator, or the possible cause.

2. The computer-implemented method of claim 1, wherein analyzing the data captured by the one or more interior sensors to determine an indication that the activity of the automobile operator either cannot be detected or cannot be identified using the data captured by the one or more interior sensors comprises:

analyzing, by the processor, the data captured by the one or more interior sensors to detect an activity of the automobile operator;

determining, by the processor, one or more possible identifications of the detected activity of the automobile operator;

determining, based on analysis of the detected activity, a low statistical confidence value or a high statistical error value for each of the one or more possible identifications of the detected activity, wherein

a low statistical confidence value is a statistical confidence value below a threshold statistical confidence value, and

a high statistical error value is a statistical error value above a threshold statistical error value.

3. The computer-implemented method of claim 1, wherein analyzing the data captured by the one or more interior sensors to determine an indication that the activity of the automobile operator either cannot be detected or cannot be identified using the data captured by the one or more interior sensors comprises:

receiving, by the processor, an indication that the automobile is in motion;

analyzing, by the processor, the data captured by the one or more interior sensors; and

determining that no operator activity can be detected during a period of time while the automobile is in motion.

4. The computer-implemented method of claim 3, wherein the data captured by the one or more interior sensors includes image data, and wherein determining that no operator activity can be detected during a period of time while the automobile is in motion comprises:

analyzing, by the processor, the image data captured by the one or more interior sensors to determine that no movement is indicated by a series of images captured during a period of time while the automobile is in motion.

5. The computer-implemented method of claim 1, wherein the data captured by the one or more interior sensors includes temperature data, and

wherein analyzing the data captured by the one or more interior sensors to determine an indication that the activity of the automobile operator either cannot be detected or cannot be identified using the data captured by the one or more interior sensors comprises:

determining that the data captured by one or more of the interior sensors indicates a temperature greater than a threshold temperature, or

determining that the data captured by the one or more interior sensors indicates a temperature difference between two of the interior sensors greater than a threshold temperature difference.

6. The computer-implemented method of claim 1, wherein the data captured by the one or more interior sensors includes electromagnetic radiation data, and

wherein analyzing the data captured by the one or more interior sensors to determine an indication that the activity of the automobile operator either cannot be detected or cannot be identified using the data captured by the one or more interior sensors comprises:

determining that the data captured by one or more of the interior sensors indicates a level of electromagnetic radiation higher than a certain high threshold;

determining that the data captured by one or more of the interior sensors indicates a level of electromagnetic radiation lower than a certain low threshold;

or

determining that the data captured by the one or more interior sensors indicates an electromagnetic radiation level difference between two of the interior sensors greater than a threshold electromagnetic radiation level difference.

7. The computer-implemented method of claim 1, wherein the data captured by the one or more interior sensors includes image data, and

wherein analyzing the data captured by the one or more interior sensors to determine an indication that the activity of the automobile operator either cannot be detected or cannot be identified using the data captured by the one or more interior sensors comprises:

analyzing, by the processor, the images captured by the one or more interior image sensors to determine that one or more captured images indicate an image resolution below a threshold image resolution value.

8. The computer-implemented method of claim 1, wherein diagnosing, by the processor, the possible cause of the indication that the activity of the automobile operator either cannot be detected or cannot be identified comprises:

accessing a database including indications that the activity of the automobile operator either cannot be detected or cannot be identified using the data captured by the one or more interior sensors, and at least one corresponding possible cause for each indication; and

identifying the possible cause corresponding to the indication determined by the processor.

9. The computer-implemented method of claim 1, wherein the data captured by the one or more interior sensors includes data captured by one or more interior image sensors or data captured by one or more interior microphone, and

wherein identifying, by the processor, an automobile operator associated with the indication that the activity of the automobile operator either cannot be detected or cannot be identified comprises one or more of:

analyzing, by the processor, the data detected by one or more of the interior image sensors to identify the automobile operator at the time of the indication that the activity of the automobile operator either cannot be detected or cannot be identified,

identifying the automobile operator associated with a most recent login credential provided to an electronic device associated with the automobile at the time of the indication that the activity of the automobile operator either cannot be detected or cannot be identified, or

identifying the automobile operator associated with a most recent voice recorded by the interior microphone at the time of the indication that the activity of the automobile operator either cannot be detected or cannot be identified.

10. An electronic device configured to identify interior automobile sensor tampering, comprising:

one or more interior sensors within an automobile, including one or more of;

an image sensor, a temperature sensor, an electromagnetic radiation sensor, a sound sensor, an acceleration sensor, or a velocity sensor;

a transceiver configured to communicate data via at least one network connection;

a memory configured to store non-transitory computer executable instructions;

a processor configured to interface with the transceiver and the memory, and configured to execute the non-transitory computer executable instructions to cause the processor to;

receive data captured by the one or more interior sensors within an automobile;

analyze the data captured by the one or more interior sensors to determine an indication that the activity of the automobile operator either cannot be detected or cannot be identified using the data captured by the one or more interior sensors;

record a date and time associated with the indication that the activity of the automobile operator either cannot be detected or cannot be identified;

identify the automobile operator associated with the indication that the activity of the automobile operator either cannot be detected or cannot be identified;

diagnose a possible cause of the indication that the activity of the automobile operator either cannot be detected or cannot be identified, wherein diagnosing includes determining, based at least on a mechanical spring loaded mechanism that is activated when a plug is unplugged, that one or more of a main source of power or a backup source of power for one or more of the interior sensors has been disabled;

generate one or more of a notification that the activity of the automobile operator either cannot be detected or cannot be identified, or a notification indicating the possible cause;

generate an indication for the automobile operator based on the possible cause;

transmit, to a server, a log of one or more of the indication that the activity of the automobile operator either cannot be detected or cannot be identified, the date and time, the identity of the automobile operator, or the possible cause.

11. The electronic device of claim 10, wherein the computer executable instructions causing the processor to analyze the data captured by the one or more interior sensors to determine an indication that the activity of the automobile operator either cannot be detected or cannot be identified using the data captured by the one or more interior sensors, cause the processor to:

analyze the data captured by the one or more interior sensors to detect an activity of the automobile operator;

determine one or more possible identifications of the detected activity of the automobile operator;

determine, based on analysis of the detected activity, a low statistical confidence value or a high statistical error value for each of the one or more possible identifications of the detected activity, wherein

a low statistical confidence value is a statistical confidence value below a threshold statistical confidence value, and

a high statistical error value is a statistical error value above a threshold statistical error value.

12. The electronic device of claim 10, wherein the computer executable instructions causing the processor to analyze the data captured by the one or more interior sensors to determine an indication that the activity of the automobile operator either cannot be detected or cannot be identified using the data captured by the one or more interior sensors, further cause the processor to:

receive an indication that the automobile is in motion;

analyze the data captured by the one or more interior sensors; and

determine that no operator activity can be detected during a period of time while the automobile is in motion.

13. The electronic device of claim 12, wherein the data captured by the one or more interior sensors includes data captured by one or more interior image sensors, and

wherein the computer executable instructions causing the processor to determine that no operator activity can be detected during a period of time while the automobile is in motion, further cause the processor to:

analyze the data captured by the one or more interior image sensors to determine that no movement is indicated by a series of images captured during a period of time while the automobile is in motion.

14. The electronic device of claim 10, wherein the data captured by the one or more interior sensors includes data captured by one or more interior temperature sensors, and

wherein the computer executable instructions causing the processor to analyze the data captured by the one or more interior sensors to determine an indication that the activity of the automobile operator either cannot be detected or cannot be identified using the data captured by the one or more interior sensors, further cause the processor to:

determine that the data captured by one or more of the interior sensors indicates a temperature greater than a threshold temperature, or

determine that the data captured by the one or more interior sensors indicates a temperature difference between two of the interior sensors greater than a threshold temperature difference.

15. The electronic device of claim 10, wherein the data captured by the one or more interior sensors includes data captured by one or more interior electromagnetic radiation sensors, and

wherein the computer executable instructions causing the processor to analyze the data captured by the one or more interior sensors to determine an indication that the activity of the automobile operator either cannot be detected or cannot be identified using the data captured by the one or more interior sensors, further cause the processor to

determine that the data captured by one or more of the interior sensors indicates a level of electromagnetic radiation higher than a certain high threshold;

determine that the data captured by one or more of the interior sensors indicates a level of electromagnetic radiation lower than a certain low threshold;

or

determine that the data captured by the one or more interior sensors indicates an electromagnetic radiation level difference between two of the interior sensors greater than a threshold electromagnetic radiation level difference.

16. The electronic device of claim 10, wherein the data captured by the one or more interior sensors includes data captured by one or more interior image sensors, and

wherein the computer executable instructions causing the processor to analyze the data captured by the one or more interior sensors to determine an indication that the activity of the automobile operator either cannot be detected or cannot be identified using the data captured by the one or more interior sensors, further cause the processor to:

analyze image data captured by the one or more interior image sensors to determine that the image data indicates an image resolution below a threshold image resolution value.

17. The electronic device of claim 10, wherein the computer executable instructions causing the processor to diagnose the possible cause for the indication that the activity of the automobile operator either cannot be detected or cannot be identified using the data captured by the one or more interior sensors, further cause the processor to:

access a database including indications that the activity of the automobile operator either cannot be detected or cannot be identified using the data captured by the one or more interior sensors and at least one corresponding possible cause for each indication; and

identify the possible cause corresponding to the indication determined by the processor.

18. The electronic device of claim 10, wherein the data captured by the one or more interior sensors includes data captured by one or more interior image sensors or data captured by one or more interior microphone, and

wherein the computer executable instructions causing the processor to identify the automobile operator associated with the indication that the activity of the automobile operator either cannot be detected or cannot be identified using the data captured by the one or more interior sensors, further cause the processor to:

analyze the data detected by one or more of the interior image sensors to identify the automobile operator at the time of the indication that the activity of the automobile operator cannot be detected or cannot be identified,

identify the automobile operator associated with the most recent login credentials provided to an electronic device associated with the automobile at the time of the indication that the activity of the automobile operator cannot be identified, or

identify the automobile operator associated with a most recent voice recorded by the interior microphone at the time of the indication that the activity of the automobile operator either cannot be detected or cannot be identified.