System and method for seat occupancy detection from ceiling mounted camera using robust adaptive threshold criteria

• US 9,378,421 B2
• Filed: 03/31/2015
• Issued: 06/28/2016
• Est. Priority Date: 09/29/2014
• Status: Expired due to Fees

• Current Assignee: Palo Alto Research Center, Inc. (Xerox Holdings Corp.), Xerox Corporation (Xerox Holdings Corp.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Loce, Robert P., Sultani, Waqas, Wu, Hao, Xu, Beilei
• Primary Examiner(s): Liew, Alex
• Application Number:
  US14/673,943
• Publication Number:
  US 20160092733A1
• Time in Patent Office: 455 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/593 Recognising seat occupancy
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

A method for detecting sitting behavior includes acquiring a sequence of frames capturing a scene-of-interest at an overhead view. The method includes detecting at least one empty seat within the scene-of-interest and associating the seat as being unoccupied and the frame as a reference frame. The method includes extracting reference features describing a region of the unoccupied seat in the reference frame and quantifying the reference features to form a reference feature vector. The method includes extracting features describing the region in a given frame and quantifying the features to form a current feature vector. The method includes measuring a change in a feature vector over time using the reference feature vector and the current feature vector. The method includes and determining a status of the seat in the given frame as being one of occupied and unoccupied based on the change in the feature vector.

Claims

1. A method for detecting sitting behavior, the method comprising:

acquiring a sequence of frames capturing a scene-of-interest at an overhead view;

detecting at least one empty seat within the scene-of-interest and associating the seat as being unoccupied and the frame as a reference frame;

extracting reference features describing a region of the unoccupied seat in the reference frame and quantifying the reference features to form a reference feature vector;

extracting features describing the region in a given frame and quantifying the features to form a current feature vector;

measuring a change in a feature vector over time using the reference feature vector and the current feature vector; and

,

determining a status of the seat in the given frame as being one of occupied and unoccupied based on the change in the feature vector.

2. The method of claim 1, wherein the extracting the reference features and features includes measuring color attributes and the quantifying the reference features and the features includes computing a color histogram describing the region and generating a feature vector using the histogram.

3. The method of claim 1, wherein the measuring the change includes:

determining a trajectory of deviation reflecting the feature vector over time;

in response to the trajectory meeting and exceeding an adaptive threshold, associating a frame substantially at a point in the sequence of frames where the trajectory met and exceeded the adaptive threshold as a candidate frame where sitting behavior is observed.

4. The method of claim 1, wherein the measuring the change in the feature vector includes computing a KL divergence between the reference and given frames using the reference feature vector and current feature vector.

5. The method of claim 1 further comprising:

determining a trajectory of deviation reflecting the feature vector over time;

computing a first derivative of the trajectory over time;

determining a second derivative of the trajectory over time; and

,

in response to the second derivative being substantially at a global minimum, associating the given frame as a first candidate frame where sitting behavior is observed.

6. The method of claim 1, further comprising:

in response to the change in the feature vector not meeting a predetermined threshold, associating a status of the seat in the given frame as being unoccupied; and

,

in response to the change in the feature vector meeting or exceeding the predetermined threshold, associating the status of the seat in the given frame as being occupied.

7. The method of claim 6 further comprising:

acquiring a second sequence of frames capturing the scene-of-interest from a field of view different from that in the first sequence of frames;

determining a region of interest (“

ROI”

) where sitting behavior is expected within the scene;

extracting activity features from the ROI in each frame of the second sequence and generating an activity feature vector using the activity features;

applying the activity feature vector of the each frame to a previously trained classifier to obtain a score for the each frame;

mapping scores for the second sequence onto a score map;

identifying a maximum score on the score map;

associating a second candidate frame corresponding to the global maximum as showing characteristics of potential sitting behavior.

8. The method of claim 7 further comprising:

determining a difference between the first and second candidate frames;

in response to the difference falling within a predetermined range, updating a status of the seat in the given frame as being occupied.

9. The method of claim 1, wherein the detecting the at least one unoccupied seat is performed by an object detection method selected from a group consisting:

deformable part model approach;

color-based approach;

texture-based approach;

seat detection;

segmentation;

manual orientation; and

a combination of the above.

10. A system for detecting sitting behavior, the system comprising a sitting detection determination unit including a memory and a processor in communication with the processor configured to:

acquire a sequence of frames capturing a scene-of-interest at an overhead view;

detect at least one empty seat within the scene-of-interest and associating the seat as being unoccupied and the frame as a reference frame;

extract reference features describing a region of the unoccupied seat in the reference frame and quantifying the reference features to form a reference feature vector;

extract features describing the region in a given frame and quantifying the features to form a current feature vector;

measure a change in a feature vector over time using the reference feature vector and the current feature vector; and

,

determine a status of the seat in the given frame as being one of occupied and unoccupied based on the change in the feature vector.

11. The system of claim 10, wherein the processor is configured to extract the reference features and features by measuring color attributes, and the processor is configured to quantify the reference features and the features by computing a color histogram describing the region and generating a feature vector using the histogram.

12. The system of claim 10, where the processor is further configured to:

determine a trajectory of deviation reflecting the feature vector over time;

in response to the trajectory meeting and exceeding an adaptive threshold, associate a frame substantially at a point in the sequence of frames where the trajectory met and exceeded the adaptive threshold as a candidate frame where sitting behavior is observed.

13. The system of claim 10, where the processor is further configured to:

compute a KL divergence between the reference and given frames using the reference feature vector and current feature vector.

14. The system of claim 10, where the processor is further configured to:

determine a trajectory of deviation reflecting the feature vector over time;

compute a first derivative of the trajectory over time;

determine a second derivative of the trajectory over time; and

,

in response to the second derivative being substantially at a global minimum, associating the given frame as a first candidate frame where sitting behavior is observed.

15. The system of claim 10, where the processor is further configured to:

in response to the change in the feature vector not meeting a predetermined threshold, associate a status of the seat in the given frame as being unoccupied; and

,

in response to the change in the feature vector meeting or exceeding the predetermined threshold, associate the status of the seat in the given frame as being occupied.

16. The system of claim 15, where the processor is further configured to:

acquire a second sequence of frames capturing the scene-of-interest from a field of view different from that in the first sequence of frames;

determine a region of interest (“

ROI”

) where sitting behavior is expected within the scene;

extract activity features from the ROI in each frame of the second sequence and generating an activity feature vector using the activity features;

apply the activity feature vector of the each frame to a previously trained classifier to obtain a score for the each frame;

map scores for the second sequence of frames onto a score map;

identify a maximum score on the score map;

associate a second candidate frame corresponding to the global maximum as showing characteristics of potential sitting behavior.

17. The system of claim 16, where the processor is further configured to:

determine a difference between the first and second candidate frames;

in response to the difference falling within a predetermined range, update a status of the seat in the given frame as being occupied.

18. The system of claim 10, wherein the detecting the at least one unoccupied seat is performed by an object detection method selected from a group consisting:

deformable part model approach;

color-based approach;

texture-based approach;

seat detection;

segmentation;

manual orientation; and

a combination of the above.

19. A method for detecting sitting behavior, the method comprising:

acquiring a sequence of frames capturing the scene-of-interest;

determining a region of interest (“

ROI”

) where sitting behavior is expected within the scene-of-interest;

extracting activity features from the ROI in each frame of the sequence and generating an activity feature vector using the activity features;

applying the activity feature vector of the each frame to a previously trained classifier to obtain a score for the each frame;

mapping scores for the sequence onto a score map;

identifying a maximum score on the score map;

associating a candidate frame corresponding to a global maximum as showing characteristics of potential sitting behavior.