Systems and methods for detecting cell phone usage by a vehicle operator

• US 9,165,201 B2
• Filed: 09/15/2011
• Issued: 10/20/2015
• Est. Priority Date: 09/15/2011
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Zhang, Yeqing, Xu, Beilei, Mestha, Lalit K., Pennington, Graham
• Primary Examiner(s): Lefkowitz, Sumati
• Assistant Examiner(s): ROWLAND, JHERE M
• Application Number:
  US13/233,780
• Publication Number:
  US 20130070957A1
• Time in Patent Office: 1,496 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

An embodiment generally relates to systems and methods for determining cell phone usage automatically by individuals operating vehicles. A processing module can process multi-spectral images or videos of individuals and detect different regions in the image such as face regions, hand regions, and cell phone regions. Further, the processing module can analyze the regions based on locations and numbers of skin pixels and cell phone pixels to determine if the individual is holding his or her cell phone near his or her face. Based on the analysis, it can be determined whether the individual is operating the cell phone. Further, the analysis can yield a confidence level associated with the cell phone usage.

Claims

1. A method of processing data, the method comprising:

processing, by a processor, data associated with an image or a video of an individual operating a vehicle, wherein the processing comprises;

analyzing the data to identify skin pixels and cell phone pixels within the image or the video based on a set of material characteristics;

performing a facial detection technique on the data to identify a facial region comprising a portion of the skin pixels;

identifying a region of interest (ROI) in a proximity to the facial region;

analyzing an additional portion of the skin pixels and the cell phone pixels within the ROI; and

determining whether the individual is using a cell phone based on the analysis of the additional portion of the skin pixels and the cell phone pixels;

wherein;

the image is a multi-spectral image captured using a plurality of wavelengths;

the material characteristics comprise;

material reflectance of skin at the plurality of wavelengths, and

material reflectance of the cell phone at the plurality of wavelengths.

2. The method of claim 1, wherein the data is analyzed according to a principal component analysis (PCA) technique.

3. The method of claim 1, wherein determining whether the individual is using the cell phone comprises:

determining that a number of pixels in the additional portion of the skin pixels within the region of interest (ROI) exceeds a first threshold number of pixels;

determining that a number of the cell phone pixels within the ROI exceeds a second threshold number of pixels; and

deeming that the individual is using the cell phone based on the determinations.

4. The method of claim 1, wherein determining whether the individual is using the cell phone comprises:

determining that a number of pixels in the additional portion of the skin pixels within the region of interest (ROI) exceeds a first threshold number of pixels;

determining that a number of the cell phone pixels within the ROI does not exceed a second threshold number of pixels; and

deeming that the individual is not using the cell phone based on the determinations.

5. The method of claim 1, wherein the processing further comprises:

analyzing additional images of the multi-spectral image; and

determining a confidence level of the usage of the cell phone based on the analysis of the additional images.

6. The method of claim 1, wherein the region of interest (ROI) is adjacent to the facial region.

7. The method of claim 1, wherein the additional portion of the skin pixels comprises hand pixels.

8. The method of claim 1, wherein:

the skin pixels in the ROI comprises a skin pixel blob;

the cell phone pixels in the ROI comprises a cell phone pixel blob;

the determining comprises;

determining that the individual is operating the cell phone based on the estimating; and

the processing further comprises issuing an alert based on the determining that the individual is operating the cell phone.

9. A system for processing data, the system comprising:

a capture device configured to capture data comprising an image or a video of an individual operating a vehicle; and

a processor coupled to the capture device and configured to perform actions comprising;

processing the data, wherein the processing comprises;

analyzing the data to identify skin pixels and cell phone pixels within the image or the video based on a set of material characteristics;

performing a facial detection technique on the data to identify a facial region comprising a portion of the skin pixels;

identifying a region of interest (ROI) in a proximity to the facial region;

analyzing an additional portion of the skin pixels and the cell phone pixels within the ROI; and

determining whether the individual is using a cell phone based on the analysis of the additional portion of the skin pixels and the cell phone pixels;

wherein;

the image is a multi-spectral image captured using a plurality of wavelengths;

the material characteristics comprise;

material reflectance of skin at the plurality of wavelengths, and

material reflectance of the cell phone at the plurality of wavelengths.

10. The system of claim 9, wherein the data is analyzed according to a principal component analysis (PCA) technique.

11. The system of claim 9, wherein determining whether the individual is using the cell phone comprises:

determining that a number of pixels in the additional portion of the skin pixels within the region of interest (ROI) exceeds a first threshold number of pixels;

determining that a number of the cell phone pixels within the ROI exceeds a second threshold number of pixels; and

deeming that the individual is using the cell phone based on the determinations.

12. The system of claim 9, wherein determining whether the individual is using the cell phone comprises:

determining that a number of pixels in the additional portion of the skin pixels within the region of interest (ROI) exceeds a first threshold number of pixels;

determining that a number of the cell phone pixels within the ROI does not exceed a second threshold number of pixels; and

deeming that the individual is not using the cell phone based on the determinations.

13. The system of claim 9, wherein the processing further comprises:

analyzing additional images of the multi-spectral image; and

determining a confidence level of the usage of the cell phone based on the analysis of the additional images.

14. The system of claim 9, wherein the region of interest (ROI) is adjacent to the facial region.

15. The system of claim 9, wherein the additional portion of the skin pixels comprises hand pixels.

16. The system of claim 9, wherein:

the skin pixels in the ROI comprises a skin pixel blob;

the cell phone pixels in the ROI comprises a cell phone pixel blob;

the determining comprises;

estimating a relative position of the skin pixel blob and the cell phone pixel blob in relation to the face region; and

determining that the individual is operating the cell phone based on the estimating; and

the processing further comprises issuing an alert based on the determining that the individual is operating the cell phone.