Systems and methods regarding image distification and prediction models

• US 10,289,938 B1
• Filed: 05/16/2017
• Issued: 05/14/2019
• Est. Priority Date: 05/16/2017
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Flowers, Elizabeth, Dua, Puneit, Balota, Eric, Phillips, Shanna L.
• Primary Examiner(s): Coleman, Stephen P
• Application Number:
  US15/596,155
• Time in Patent Office: 728 Days
• Field of Search: None
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects

Abstract

Systems and methods are described for generating an image-based prediction model, where a computing device may obtain a set 3D images from a 3D image data source. Each of the 3D images can have 3D point cloud data and a Distification technique can be applied to the 3D point cloud data of each 3D image to generate output feature vector(s). The output feature vector(s) may then be used to train and generate the image-based prediction model.

Claims

1. A multi-dimensioning computing device configured to generate enhanced image-based prediction models based on two-dimensional (2D) image matrices determined from source three-dimensional (3D) images, the multi-dimensioning computing device comprising:

an image processor;

a memory communicatively coupled to the image processor;

a distification component implemented on the memory and executing on the image processor to;

obtain a set of one or more three dimensional (3D) images from a 3D image data source, wherein each 3D image in the set is associated with 3D point cloud data, and wherein the 3D image data source is an imaging device onboard a vehicle;

initiate a distification enhancement using the 3D point cloud data of each 3D image, the distification enhancement including (1) generating a 2D image matrix from each of the one or more 3D images, and (2) generating one or more output feature vectors based on each 2D image matrix and respective 3D image; and

generate an enhanced prediction model by training the enhanced prediction model using the one or more output feature vectors,

wherein the memory of the multi-dimensioning computing device is updated with the enhanced prediction model to configure the multi-dimensioning computing device to output enhanced predictions from new 3D images based on the enhanced prediction model.

2. The multi-dimensioning computing device of claim 1, wherein training the enhanced prediction model further comprises using one or more batches of the output feature vectors, wherein the one or more batches of the output feature vectors correspond to one or more subsets of 3D images from the set of the one or more 3D images.

3. The multi-dimensioning computing device of claim 1, wherein the 3D image data source is a remote computing device.

4. The multi-dimensioning computing device of claim 1, wherein the enhanced prediction model is trained using a neural network.

5. The multi-dimensioning computing device of claim 4, wherein the neural network is a convolutional neural network.

6. The multi-dimensioning computing device of claim 1, wherein the set of one or more 3D images relate to one or more driver behaviors.

7. The multi-dimensioning computing device of claim 6, wherein the one or more driver behaviors include any one or more of the following:

left hand calling, right hand calling, left hand texting, right hand texting, eating, drinking, adjusting the radio, or reaching for the backseat.

8. The multi-dimensioning computing device of claim 6, further configured to receive a second set of one or more 3D images as input into the enhanced prediction model to determine, for each 3D image of the second set, a driver behavior classification and corresponding probability value, wherein the second set of one or more 3D images relates to the one or more driver behaviors, and wherein the probability value indicates the probability that the 3D image is associated with the driver behavior classification.

9. The multi-dimensioning computing device of claim 8, wherein, for each 3D image of the second set, the 3D image is assigned an image classification equal to the driver behavior classification that has a greatest corresponding probability value.

10. The multi-dimensioning computing device of claim 8, wherein, for at least one of the 3D images in the second set, the driver behavior classification and the probability value are transmitted to a different computing device.

11. A multi-dimensioning method of generating enhanced image-based prediction models based on two-dimensional (2D) image matrices determined from source three-dimensional (3D) images, the multi-dimensioning method comprising:

obtaining, with a distification component implemented on a memory and executing on an image processor, a set of one or more three dimensional (3D) images from a 3D image data source, wherein each 3D image in the set is associated with 3D point cloud data, and wherein the 3D image data source is an imaging device onboard a vehicle;

initiating a distification enhancement using the 3D point cloud data of each 3D image, the distification enhancement including (1) generating a 2D image matrix from each of the one or more 3D images, and (2) generating one or more output feature vectors based on each 2D image matrix and respective 3D image; and

generating an enhanced prediction model by training the enhanced prediction model using the one or more output feature vectors,

wherein the memory of the multi-dimensioning computing device is updated with the enhanced prediction model to configure the multi-dimensioning computing device to output enhanced predictions from new 3D images based on the enhanced prediction model.

12. The multi-dimensioning method of claim 11, wherein training the enhanced prediction model further comprises using one or more batches of the output feature vectors, wherein the one or more batches of the output feature vectors correspond to one or more subsets of 3D images from the set of the one or more 3D images.

13. The multi-dimensioning method of claim 11, wherein the 3D image data source is a remote computing device.

14. The multi-dimensioning method of claim 11, wherein the enhanced prediction model is trained using a neural network.

15. The multi-dimensioning method of claim 14, wherein the neural network is a convolutional neural network.

16. The multi-dimensioning method of claim 11, wherein the set of one or more 3D images relate to one or more driver behaviors.

17. The multi-dimensioning method of claim 16, wherein the one or more driver behaviors include any one or more of the following:

left hand calling, right hand calling, left hand texting, right hand texting, eating, drinking, adjusting the radio, or reaching for the backseat.

18. The multi-dimensioning method of claim 16, further comprising receiving a second set of one or more 3D images as input into the enhanced prediction model to determine, for each 3D image of the second set, a driver behavior classification and corresponding probability value, wherein the second set of one or more 3D images relates to the one or more driver behaviors, and wherein the probability value indicates the probability that the 3D image is associated with the driver behavior classification.

19. The multi-dimensioning method of claim 18, wherein, for each 3D image of the second set, the 3D image is assigned an image classification equal to the driver behavior classification that has a greatest corresponding probability value.

20. The multi-dimensioning method of claim 18, wherein, for at least one of the 3D images in the second set, the driver behavior classification and the probability value are transmitted to a different computing device.