Processing system having a machine learning engine for providing a customized driving assistance output

• US 10,388,157 B1
• Filed: 03/13/2018
• Issued: 08/20/2019
• Est. Priority Date: 03/13/2018
• Status: Active Grant

• Current Assignee: Allstate Insurance Company (Allstate Corporation)
• Original Assignee: Allstate Insurance Company (Allstate Corporation)
• Inventors: Hayes, Howard, Chintakindi, Sunil
• Primary Examiner(s): Lieu, Julie B
• Application Number:
  US15/919,313
• Time in Patent Office: 525 Days
• Field of Search: None
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G08G 1/0112 from the vehicle, e.g. floating car data [FCD]
• CPC Class Codes: G08G 1/0129 for creating historical data or processing based on historical data
• CPC Class Codes: G08G 1/0145 for active traffic flow control
• CPC Class Codes: G08G 1/09626 where the origin of the information is within the own vehicle, e.g. a local storage device, digital map
• CPC Class Codes: G08G 1/163 involving continuous checking
• CPC Class Codes: G08G 1/164 Centralised systems, e.g. external to vehicles
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

Systems and apparatuses for generating customized driving assistance outputs are provided. The system may collect initial sensor data from a rotating mobile device. The system may analyze the initial sensor data to determine whether to collect additional sensor data. Based on analysis of the initial sensor data and the additional sensor data, the system may determine a customized driving assistance output. The system may determine one or more notification outputs and one or more instruction outputs for the driver based on the customized driving assistance output. The system may update an aggregate driving output based on whether or not the instruction output was completed, as determined from subsequent sensor data.

Claims

1. A method comprising:

transmitting, by a sensor data analysis and device control system including a processor and to a mobile device, an instruction to position an orientation of the mobile device within a first vehicle;

receiving, by the sensor data analysis and device control system and from the mobile device, initial sensor data comprising sensor data around and within the first vehicle collected during or after the positioning;

analyzing, in real-time and by the sensor data analysis and device control system, the initial sensor data to identify a potential hazard;

receiving, after the identifying the potential hazard, by the sensor data analysis and device control system, and from the mobile device, additional sensor data associated with the potential hazard;

determining, by the sensor data analysis and device control system and based on the initial sensor data and the additional sensor data, a customized driving assistance output associated with the potential hazard;

generating, by the sensor data analysis and device control system and based on the customized driving assistance output, a notification output comprising an indication of the potential hazard; and

generating, by the sensor data analysis and device control system and after the generating the notification output, an instruction output comprising an instruction to modify a position or operation of the first vehicle responsive to the potential hazard; and

transmitting the notification output and the instruction output to the mobile device.

2. The method of claim 1, wherein the instruction to position the mobile device within the first vehicle comprises an instruction to initiate rotation of the mobile device within the first vehicle.

3. The method of claim 1, wherein the initial sensor data comprises at least one of:

video sensor data, audio sensor data, vibration sensor data, and proximity sensor data.

4. The method of claim 1, further comprising:

receiving, by the sensor data analysis and device control system, data related to at least one of;

a known route of the first vehicle, a risk map for the known route, weather information, and insurance information;

analyzing, by the sensor data analysis and device control system, the data to identify an orientation for the mobile device; and

transmitting, by the sensor data analysis and device control system and to the mobile device, an instruction to modify a position of the mobile device to the identified orientation.

5. The method of claim 4, wherein the identifying the potential hazard comprises identifying, during the receiving the initial sensor data, the potential hazard.

6. The method of claim 1, wherein the potential hazard comprises at least one of:

a pothole, an accident, a second vehicle, and an aggregate driving output of a driver of the second vehicle that is below a predetermined threshold value.

7. The method of claim 6, further comprising:

determining the aggregate driving output of the driver of the second vehicle by extracting data from a database comprising a plurality of aggregate driving outputs; and

transmitting, by the sensor data analysis and device control system, to the mobile device, and based on the aggregate driving output of the driver of the second vehicle, an instruction to capture additional images of the driver of the second vehicle.

8. The method of claim 7, wherein the plurality of aggregate driving outputs comprises an aggregate driving output of a driver of the first vehicle.

9. The method of claim 8, further comprising:

receiving, by the sensor data analysis and device control system, a reaction of the driver of the first vehicle to the instruction output; and

updating, by the sensor data analysis and device control system and responsive to receiving the reaction, the aggregate driving output of the driver of the first vehicle.

10. The method of claim 1, wherein the generating the notification output comprises:

generating, by the sensor data analysis and device control system and based on historical data, a risk map;

generating, by the sensor data analysis and device control system, an indication of the potential hazard on the risk map; and

transmitting, by the sensor data analysis and device control system and to the mobile device, the risk map for display on the mobile device.

11. An apparatus comprising:

a memory; and

a processor coupled to the memory and programmed with computer-executable instructions for performing steps comprising;

transmitting, by a sensor data analysis and device control system including a processor and to a mobile device, an instruction to position an orientation of the mobile device within a first vehicle;

receiving, by the sensor data analysis and device control system and from the mobile device, initial sensor data comprising sensor data around and within the first vehicle collected during or after the positioning;

analyzing, in real-time and by the sensor data analysis and device control system, the initial sensor data to identify a potential hazard;

receiving, after the identifying the potential hazard, by the sensor data analysis and device control system, and from the mobile device, additional sensor data associated with the potential hazard;

determining, by the sensor data analysis and device control system and based on the initial sensor data and the additional sensor data, a customized driving assistance output associated with the potential hazard;

generating, by the sensor data analysis and device control system and based on the customized driving assistance output, a notification output comprising an indication of the potential hazard; and

generating, by the sensor data analysis and device control system and after the generating the notification output, an instruction output comprising an instruction to modify a position or operation of the first vehicle responsive to the potential hazard; and

transmitting the notification output and the instruction output to the mobile device.

12. The apparatus of claim 11, wherein the generating the notification output comprises:

generating, by the sensor data analysis and device control system, an output to be displayed on one of a plurality of windows comprising the first vehicle, wherein the one of the plurality of windows is closer to the potential hazard than other windows comprising the plurality of windows; and

transmitting, to an operating system of the first vehicle, an instruction to display the notification output on the one of the plurality of windows.

13. The apparatus of claim 11, wherein the instruction output comprises one of:

an instruction for a driver of the first vehicle to take an action responsive to the potential hazard or an instruction for an operating system of the first vehicle to cause the first vehicle to perform the action responsive to the potential hazard.

14. The apparatus of claim 13, wherein the instruction output comprises an instruction to change lanes.

15. The apparatus of claim 11, wherein the mobile device is mounted in a cradle and wherein the instruction to position the mobile device comprises instructing the mobile device to initiate rotation of the cradle.

16. The apparatus of claim 11, wherein the instruction to position the mobile device comprises an instruction to initiate at least one of:

horizontal rotation of the mobile device, vertical rotation of the mobile device, or a combination of horizontal and vertical rotations of the mobile device.

17. The apparatus of claim 11, wherein the mobile device comprises the sensor data analysis and device control system.

18. A non-transitory computer readable medium storing computer executable instructions, which when executed by a processor, cause a computing device to perform steps comprising:

transmitting, by a sensor data analysis and device control system including a processor and to a mobile device, an instruction to position an orientation of the mobile device within a first vehicle;

receiving, by the sensor data analysis and device control system and from the mobile device, initial sensor data comprising sensor data around and within the first vehicle collected during or after the positioning;

analyzing, in real-time and by the sensor data analysis and device control system, the initial sensor data to identify a potential hazard;

receiving, after the identifying the potential hazard, by the sensor data analysis and device control system, and from the mobile device, additional sensor data associated with the potential hazard;

determining, by the sensor data analysis and device control system and based on the initial sensor data and the additional sensor data, a customized driving assistance output associated with the potential hazard;

generating, by the sensor data analysis and device control system and based on the customized driving assistance output, a notification output comprising an indication of the potential hazard; and

generating, by the sensor data analysis and device control system and after the generating the notification output, an instruction output comprising an instruction to modify a position or operation of the first vehicle responsive to the potential hazard; and

transmitting the notification output and the instruction output to the mobile device.

19. The non-transitory computer readable medium of claim 18, wherein the computing device is further configured to perform:

comparing, by the sensor data analysis and device control system, the customized driving assistance output to a predetermined driving assistance threshold; and

determining, by the sensor data analysis and device control system and based on the comparing, to generate the notification output.

20. The non-transitory computer readable medium of claim 19, wherein the computing device is further configured to perform:

analyzing, in real-time and by the sensor data analysis and device control system and after the generating the instruction output, the initial sensor data to identify a second potential hazard;

receiving, by the sensor data analysis and device control system, after the determining the second potential hazard, and from the mobile device, second additional sensor data associated with the second potential hazard;

determining, by the sensor data analysis and device control system and based on the initial sensor data and the second additional sensor data, a second customized driving assistance output associated with the second potential hazard;

determining, by the sensor data analysis and device control system, that the second customized driving assistance output is below the predetermined driving assistance threshold; and

determining, by the sensor data analysis and device control system and based on the determination that the customized driving assistance output is below the predetermined driving assistance threshold, that a second notification output should not be generated.