Machine learning approach for detecting mobile phone usage by a driver

• US 9,721,173 B2
• Filed: 04/04/2014
• Issued: 08/01/2017
• Est. Priority Date: 04/04/2014
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Inventors: Paul, Peter, Xu, Beilei, Loce, Robert P., Wade, Thomas F.
• Primary Examiner(s): TSAI, TSUNG YIN
• Application Number:
  US14/245,212
• Publication Number:
  US 20150286884A1
• Time in Patent Office: 1,215 Days
• Field of Search: None
• CPC Class Codes: G06F 18/2132 based on discrimination criteria, e.g. discriminant analysis
• CPC Class Codes: G06V 10/464 using a plurality of salient features, e.g. bag-of-words [BoW] representations
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness

Abstract

A system and method for detecting electronic device use by a driver of a vehicle including acquiring an image including a vehicle from an associated image capture device positioned to view oncoming traffic, locating a windshield region of the vehicle in the captured image, processing pixels of the windshield region of the image for computing a feature vector describing the windshield region of the vehicle, applying the feature vector to a classifier for classifying the image into respective classes including at least classes for candidate electronic device use and candidate electronic device non-use, and outputting the classification.

Claims

1. A method for detecting electronic device use by a driver of a vehicle, the method comprising:

acquiring a monochrome NIR image including a vehicle from an associated image capture device positioned to view oncoming traffic;

locating a region of the vehicle in the monochrome NIR image;

processing pixels of the located region of the monochrome NIR image for computing a feature vector describing a windshield region of the vehicle;

applying the feature vector to a classifier for classifying the monochrome NIR image into respective classes including at least classes for candidate electronic device use and candidate electronic device non-use; and

,

outputting the classification;

wherein processing the image includes generating a global descriptor describing the entire image using a process selected from a group consisting of;

a Successive Mean Quantization Transform (SMQT);

a Scale-Invariant Feature Transform (SIFT);

a Histogram of Gradients (HOG);

a Bag-of-Visual-Words Representation;

a Fisher Vector(FV) Representation; and

,

a combination of the above;

wherein the SMQT process includes determining a feature vector for each pixel in the image by analyzing adjacent pixels in the region of the pixel, wherein for each pixel, the pixel is designated as a center pixel in a region of adjacent pixels, an average value for the pixels in the region is determined, the average value is set as a threshold value, the value of each pixel in the region is compared to the threshold, each pixel in the region is assigned a binary value based on the comparison, and a binary number is generated for the region.

2. The method of claim 1 wherein processing the pixels of the located region include extracting the windshield region from the monochrome NIR image, and wherein classifying the monochrome NIR image is performed without searching the windshield region for specific image content.

3. The method of claim 2, further comprising defining a passenger side and a driver side within the windshield region, and extracting the driver side of the windshield region from the windshield region.

4. The method of claim 1 further comprising, before acquiring the monochrome NIR image, training a classifier using at least a first sample image including a driver holding an electronic device and at least a second sample image including a driver not holding an electronic device.

5. The method of claim 1 further comprising, in response to the vehicle being classified as a candidate electronic device user, performing further processing on the monochrome NIR image for determining whether the vehicle is a violator or non-violator.

6. The method of claim 5, wherein the further processing on the monochrome MIR image includes detecting a unique vehicle identifier associated with the vehicle in the monochrome NIR image, and comparing the unique vehicle identifier to a list of authorized vehicle identifiers to determine whether the vehicle is a violater or non-violater.

7. The method of claim 1, wherein the outputting further includes:

providing notice of a candidate violator to an associated user device for further review.

8. A system configured to perform image analysis for detecting electronic device use by a driver of a vehicle comprising:

a non-multispectral image capture device operably connected to a data processing device that captures a monochrome NIR image of a target vehicle; and

a processor-usable medium embodying computer code, said processor-usable medium being coupled to said data processing device, said computer program code comprising instructions executable by sad processor and configured for;

locating a windshield region of the vehicle in the captured image;

processing pixels of the windshield region of the image for computing a feature vector describing the windshield region of the vehicle, the feature vector defining a set of generic features that describe the entire windshield region;

applying the feature vector to a classifier for classifying the image into respective classes including at least classes for candidate electronic device use and candidate electronic device non-use; and

, outputting the classification;

wherein processing the pixels of the windshield region includes generating a global descriptor describing the entire image using a process selected from a group consisting of;

a Successive Mean Quantization Transform (SMQT);

a Scale-Invariant Feature Transform (SIFT);

a Histogram of Gradients (HOG);

a Bag-of-Visual-Words Representation;

a Fisher Vector (FV) Representation; and

,

a combination of the above;

wherein the SMQT process includes determining a feature vector for each pixel in the image by analyzing adjacent pixels in the region of the pixel, wherein for each pixel, the pixel is designated as a center pixel in a region of adjacent pixels, an average value for the pixels in the region is determined, the average value is set as a threshold value, the value of each pixel in the region is compared to the threshold, each pixel in the region is assigned a binary value based on the comparison, and a binary number is generated for the region.

9. A system as set forth in claim 8, wherein said computer program code comprising instructions executable by said processor is further configured for, after locating a windshield region of the vehicle in the captured image, extracting the windshield region from the image, and wherein classifying the image is performed without searching the windshield region for specific image content.

10. The method of claim 9, wherein said computer program code comprising instructions executable by said processor is further configured for defining a passenger side and a driver side within the windshield region, and extracting the passenger side of the windshield region from the windshield region.

11. The method of claim 8 wherein said computer program code comprising instructions executable by said processor is further configured for, prior to acquiring the image with the image capture device, training a classifier using at least a first sample image including a driver holding an electronic device and at least a second sample image including a driver not holding an electronic device.

12. The method of claim 8, wherein said computer program code comprising instructions executable by said processor is further configured for, in response to the vehicle being classified as a candidate electronic device user, performing further processing on the image for determining whether the vehicle is a violator or non-violator.

13. The method of claim 12, wherein the further processing on the image includes detecting a unique vehicle identifier associated with the vehicle in the associated image, and comparing the unique vehicle identifier to a list of authorized vehicle identifiers to determine whether the vehicle is a violater or non-violater.

14. The method of claim 8, wherein the outputting further includes providing notice of a candidate violator to an associated user device for further review.

15. A non-transitory computer-usable medium for performing image analysis for detecting electronic device use by a driver of a vehicle said computer-usable medium embodying a computer program code, said computer program code comprising computer executable instructions configured for:

acquiring a monochrome NIR image including a vehicle from an associated image capture device positioned to view oncoming traffic;

locating a windshield region of the vehicle in the captured image;

processing pixels of the windshield region of the image for computing a feature vector describing the windshield region of the vehicle, the feature vector defining a set of generic features that describe the entire windshield region without searching for specific image content within the windshield region;

applying the feature vector to a classifier for classifying the image into respective classes including at least classes for candidate electronic device use and candidate electronic device non-use; and

,

outputting the classification;

wherein processing the pixels of the windshield region includes generating a global descriptor describing the entire image using a process selected from a group consisting of;

a Successive Mean Quantization Transform (SMQT);

a Scale-Invariant Feature Transform (SIFT);

a Histogram of Gradients (HOG);

a Bag-of-Visual-Words Representation;

a Fisher Vector (FV) Representation; and

,

a combination of the above;

wherein the SMQT process includes determining a feature vector for each pixel in the image by analyzing adjacent pixels in the pixel, wherein for each pixel, the pixel is designated as a center region of adjacent pixels, an average value for the pixels in the region is determined, the average value is set as a threshold value, the value of each pixel in the region is compared to the threshold, each pixel in the region is assigned a binary value based on the comparison, and a binary number is generated for the region.

16. The non-transitory computer-usable medium of claim 15, the instructions further comprising, after locating a windshield region of the vehicle in the captured image, extracting the windshield region from the image.

17. The non-transitory computer-usable medium of claim 16, the instructions further comprising defining a passenger side and a driver side within the windshield region, and extracting the passenger side of the windshield region from the windshield region.