Method for detecting driver cell phone usage from side-view images

• US 9,842,266 B2
• Filed: 02/05/2015
• Issued: 12/12/2017
• Est. Priority Date: 04/04/2014
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Inventors: Bulan, Orhan, Artan, Yusuf O., Loce, Robert P., Paul, Peter
• Primary Examiner(s): CHEN, XUEMEI G
• Application Number:
  US14/614,746
• Publication Number:
  US 20150286885A1
• Time in Patent Office: 1,041 Days
• Field of Search: 1 1, 3406861, 3409322, 340438, 340576, 348 77, 348148, 351210, 382103, 382104, 382118, 4554141, 455418, 704235, 705 4, 715863
• CPC Class Codes: G06V 10/50 by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
• CPC Class Codes: G06V 20/597 Recognising the driver's state or behaviour, e.g. attention or drowsiness

Abstract

A system and method for detecting electronic device use by a driver of a vehicle including acquiring an image including a vehicle from an associated image capture device positioned to view oncoming traffic, locating a windshield region of the vehicle in the captured image, processing pixels of the windshield region of the image for computing a feature vector describing the windshield region of the vehicle, applying the feature vector to a classifier for classifying the image into respective classes including at least classes for candidate electronic device use and candidate electronic device non-use, and outputting the classification.

Claims

1. A method for detecting electronic device use by a driver of a vehicle, the method comprising:

acquiring a first image including a vehicle from an associated non-multispectral image capture device positioned to view oncoming traffic, the first image including a windshield of the vehicle;

locating a first region of the vehicle in the first image;

processing pixels of the first region of the first image for computing a first feature vector describing a windshield region of the vehicle;

applying the first feature vector to a classifier for classifying the first image into respective classes including at least classes for candidate electronic device use and candidate electronic device non-use;

acquiring a second image including the vehicle from an associated non-multispectral image capture device positioned to view oncoming traffic, the second image including a front passenger side window or a driver side window of the vehicle;

locating a second region of the vehicle in the second image;

processing pixels of the second region of the second image for computing a second feature vector describing a side window region of the vehicle;

applying the second feature vector to a classifier for classifying the second image into respective classes including at least classes for candidate electronic device use and candidate electronic device non-use;

analyzing the first image to detect whether a passenger is present in the vehicle;

calculating a confidence score for each of the first and second images;

wherein the calculating the confidence score for the second image includes discounting the confidence score of the second image when a passenger is detected in the first image; and

,

outputting the classification of the first image and the classification of the second image;

wherein processing the pixels in the images is performed by a process selected from a group consisting of;

a Successive Mean Quantization Transform (SMQT);

a Scale-Invariant Feature Transform (SIFT);

a Histogram of Gradients (HOG);

a Bag-of-Visual-Words Representation;

a Fisher Vector (FV) Representation; and

,

a combination of the above.

2. The method of claim 1, wherein processing the pixels of the located first region includes extracting the windshield region from the first image.

3. The method of claim 2, further comprising defining a passenger side and a driver side within the windshield region, and extracting the driver side of the windshield region from the windshield region.

4. The method of claim 2, further comprising defining a passenger side and a driver side within the windshield region, and extracting the passenger side of the windshield region from the windshield region.

5. The method of claim 1, wherein the processing the pixels of the located second region includes extracting a passenger side window region from the image.

6. The method of claim 1 further comprising, before acquiring the images, training a classifier using at least a first sample image including a driver holding an electronic device and at least a second sample image including a driver not holding an electronic device, the first and second sample images including at least a corresponding first region or second region of a sample vehicle.

7. The method of claim 1 further comprising, in response to the vehicle being classified as a candidate electronic device user, performing further processing on at least one of the first or second images for determining whether the vehicle is a violator or non-violator.

8. The method of claim 1, wherein the classification of the second image is discarded when a passenger is detected in the first image.

9. A system configured to perform image analysis for detecting electronic device use by a driver of a vehicle comprising:

at least one non-multispectral image capture device operably connected to a data processing device that captures a first image of a vehicle including a windshield region and a second image of the vehicle including a side window region; and

a processor-usable medium embodying computer code, said processor-usable medium being coupled to said data processing device, said computer code comprising instructions executable by said data processing device and configured for;

locating a windshield region of the vehicle in the first image;

processing pixels of the windshield region of the first image for computing a first feature vector describing the windshield region of the vehicle;

applying the first feature vector to a classifier for classifying the first image into respective classes including at least classes for candidate electronic device use and candidate electronic device non-use;

locating a side window region of the vehicle in the second image;

processing pixels of the side window region of the second image for computing a second feature vector describing the side window region of the vehicle;

applying the second feature vector to a classifier for classifying the second image into respective classes including at least classes for candidate electronic device use and candidate electronic device non-use;

analyzing the first image to detect whether a passenger is present in the vehicle;

calculating a confidence score for each of the first and second images;

wherein the calculating the confidence score for the second image includes discounting the confidence score of the second image when a passenger is detected in the first image; and

,

outputting the classifications of the first and second images;

wherein processing the pixels in the images is performed by a process selected from a group consisting of;

a Successive Mean Quantization Transform (SMQT);

a Scale-Invariant Feature Transform (SIFT);

a Histogram of Gradients (HOG);

a Bag-of-Visual-Words Representation;

a Fisher Vector (FV) Representation; and

,

a combination of the above.

10. The system as set forth in claim 9, wherein said computer code comprising instructions executable by said data processing device is further configured for, after locating a windshield region of the vehicle in the first image, extracting the windshield region from the first image.

11. The system as set forth in claim 10, wherein said computer code comprising instructions executable by said data processing device is further configured for defining a passenger side and a driver side within the windshield region, and extracting the driver side of the windshield region from the windshield region.

12. The system as set forth in claim 10, wherein said computer code comprising instructions executable by said data processor device is further configured for defining a passenger side and a driver side within the windshield region, and extracting the passenger side of the windshield region from the windshield region.

13. The system as set forth in claim 9, wherein said computer code comprising instructions executable by said data processing device is further configured for processing the pixels of the located side window region including extracting a passenger side window region from the second image.

14. The system as set forth in claim 9, wherein said computer code comprising instructions executable by said data processing device is further configured for, prior to acquiring the images with the at least one image capture device, training a classifier using at least a first sample image including a driver holding an electronic device and at least a second sample image including a driver not holding an electronic device, the first and second sample images including at least a respective first region or second region of a sample vehicle.

15. The system as set forth in claim 9, wherein said computer code comprising instructions executable by said data processing device is further configured for, in response to the vehicle being classified as a candidate electronic device user, performing further processing on the images for determining whether the vehicle is a violator or non-violator.

16. The system as set forth in claim 9, wherein said computer code comprising instructions executable by said data processing device is further configured to discard the classification of the second image when a passenger is detected in the first image.