System and method for detecting seat belt violations from front view vehicle images

• US 9,552,524 B2
• Filed: 09/15/2014
• Issued: 01/24/2017
• Est. Priority Date: 09/15/2014
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Artan, Yusuf, Bulan, Orhan, Loce, Robert P., Paul, Peter
• Primary Examiner(s): Bella, Matthew
• Assistant Examiner(s): TORRES, JOSE
• Application Number:
  US14/486,211
• Publication Number:
  US 20160078306A1
• Time in Patent Office: 862 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: B60N 2/002 Seats provided with an occupancy detection means mounted therein or thereon detection means responsive to presence or absence of children B60N2/266
• CPC Class Codes: B60N 2210/24 Cameras
• CPC Class Codes: B60R 2022/4816 for sensing locking of buckle
• CPC Class Codes: B60R 2022/485 for sensing belt anchor position, belt orientation, or the like
• CPC Class Codes: B60R 22/48 Control systems, alarms, or interlock systems, for the correct application of the belt or harness
• CPC Class Codes: G06F 18/214 Generating training patterns; Bootstrap methods, e.g. bagging or boosting
• CPC Class Codes: G06F 18/2411 based on the proximity to a decision surface, e.g. support vector machines
• CPC Class Codes: G06V 10/751 Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 20/59 inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
• CPC Class Codes: G06V 20/593 Recognising seat occupancy

Abstract

A system and method for detecting a seat belt violation in a vehicle. The method may include receiving an image including a region of interest where a seat belt is visible if the seat belt is buckled. One or more first features may be extracted from the image. The one or more first features may be compared to one or more second features extracted from a plurality of sample images. It may be determined whether the seat belt is buckled in the image based upon the comparison.

Claims

1. A method for detecting a seat belt violation in a vehicle, comprising:

receiving an image of a front side of a vehicle;

detecting a windshield of the vehicle in the image;

cropping the windshield out of the image and discarding a remainder of the image;

locating a passenger through the windshield;

identifying a region of interest through the windshield that includes a portion of the passenger where a seat belt is visible if the seat belt is buckled, wherein the region of interest is defined relative to a plurality of landmark points that are at set positions with respect to the windshield of the vehicle, wherein a first of the landmark points is on a rear view mirror of the vehicle, wherein a second of the landmark points is at a bottom center of the windshield, and wherein a third of the landmark points is on a dashboard of the vehicle in front of a steering wheel;

extracting one or more first features from the region of interest of the image;

comparing the one or more first features to one or more second features extracted from a plurality of sample images, wherein the one or more second features are used to train a classifier; and

determining, using the classifier, whether the seat belt is buckled in the image based upon the comparison.

2. The method of claim 1, wherein the seat belt is buckled in at least one of the sample images, and the seat belt is unbuckled, and thus not visible, in at least one of the sample images.

3. The method of claim 1, further comprising locating a passenger in a front seat of the vehicle.

4. The method of claim 3, further comprising defining the region of interest relative to a location of the passenger.

5. The method of claim 1, further comprising detecting a windshield in the image.

6. The method of claim 5, wherein the one or more features are extracted from the region of interest in the image, and wherein the region of interest is smaller than the windshield.

7. The method of claim 1, wherein the one or more first features comprises a Fisher Vector, a bag-of-words, a histogram of oriented gradients, Harris Corners, local binary patterns, speeded-up robust features, scale-invariant feature transforms, or a combination thereof.

8. The method of claim 1, wherein a relationship between an area identifying a passenger'"'"'s face and the region of interest is defined using the equations:

r_ROI(i)=r_BB(i)+R(i) 

 

(1)

c_ROI(i)=c_BB(i)+C(i) 

 

(2)

where r represents row indices of the image, c represents column indices of the image, i represents corners of a box in the image, R represents row offsets, C represents column offsets, BB represents the area identifying the passenger'"'"'s face, and ROI represents the region of interest.

9. The method of claim 1, wherein the region of interest is at least partially defined relative to the landmark points using the equations:

r_ROI(i)=f_i(LM) 

 

(1)

c_ROI(i)=g_i(LM) 

 

(2)

where r represents row indices of the image, c represents column indices of the image, i represents corners of a box in the image, f_i represents a function that outputs a linear combination of row indices of the landmark points, g_i represents a function that outputs a linear combination of column indices of the landmark points, and LM represents the landmark points.

10. The method of claim 1, wherein extracting the one or more first features from the region of interest of the image comprises:

identifying a pixel from the region of interest of the image;

designating the pixel as a center pixel in a region;

determining an average value for the region, wherein the average value is between a predetermined minimum value and a predetermined maximum value; and

comparing each pixel in the region to the average value of the region.

11. The method of claim 10, further comprising:

assigning each pixel in the region a binary value based on the comparison;

generating an aggregate binary value for the region based upon the binary value assigned to each pixel in the region;

converting the aggregate binary value to a decimal number, wherein the decimal number is a descriptor of the center pixel; and

generating a feature vector using the decimal number for each of the pixels in the image.

12. The method of claim 1, wherein the one or more first features comprise Fisher Vector features corresponding to the region of interest, and further comprising inputting the Fisher Vector features into a linear support vector machine classifier.

13. A method for detecting a seat belt violation in a vehicle, comprising:

receiving a plurality of first images, each including a region of interest where a seat belt is visible if the seat belt is buckled, wherein the seat belt is buckled in at least one of the first images, and the seat belt is unbuckled, and thus not visible, in at least one of the first images;

extracting one or more first features from each of the first images;

training a classifier to determine whether the seat belt is buckled using the one or more first extracted features;

receiving a second image including the region of interest, wherein the region of interest in the second image is defined relative to a plurality of landmark points that are at set positions with respect to a windshield of the vehicle, wherein a first of the landmark points is on a rear view mirror of the vehicle, wherein a second of the landmark points is at a bottom center of the windshield, and wherein a third of the landmark points is on a dashboard of the vehicle in front of a steering wheel;

extracting one or more second features from the second image;

comparing the one or more second extracted features to the one or more first extracted features; and

determining whether the seat belt is buckled in the second image based upon the comparison.

14. The method of claim 13, further comprising locating the windshield in the image, wherein the one or more second features are extracted from the region of interest in the second image, and wherein the region of interest is smaller than the windshield.

15. The method of claim 14, further comprising defining the region of interest relative to a location of a passenger'"'"'s face visible through the windshield.

16. The method of claim 14, further comprising creating one or more global feature representations using the one or more second features.

17. The method of claim 14, wherein the classifier comprises a support vector machine, a decision tree, a neural network, or a combination thereof.

18. A system for detecting a seat belt violation in a vehicle, comprising:

a training device configured to;

receive a plurality of first images, each including a region of interest where a seat belt is visible if the seat belt is buckled;

extract one or more first features from each of the first images; and

determine whether the seat belt is buckled using the one or more first extracted features; and

a determination device configured to;

receive a second image including the region of interest, wherein the region of interest in the second image is defined relative to a plurality of landmark points that are at set positions with respect to a windshield of the vehicle, wherein a first of the landmark points is on a rear view mirror of the vehicle, wherein a second of the landmark points is at a bottom center of the windshield, and wherein a third of the landmark points is on a dashboard of the vehicle in front of a steering wheel;

extract one or more second features from the second image;

compare the one or more second extracted features to the one or more first extracted features; and

determine whether the seat belt is buckled in the second image based upon the comparison.

19. The system of claim 18, further comprising:

an image capture device configured to capture first images, the second image, or both; and

an illuminator connected to the image capture device.

20. The system of claim 18, wherein the seat belt is buckled in at least one of the first images, and the seat belt is unbuckled, and thus not visible, in at least one of the first images.