Online learning method for people detection and counting for retail stores

• US 9,639,747 B2
• Filed: 03/15/2013
• Issued: 05/02/2017
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Pelco (Schneider Electric SE)
• Original Assignee: Pelco (Schneider Electric SE)
• Inventors: Zhu, Hongwei, Aghdasi, Farzin, Millar, Greg M., Mitchell, Stephen J.
• Primary Examiner(s): Kholdebarin, Iman K
• Application Number:
  US13/839,410
• Publication Number:
  US 20140270358A1
• Time in Patent Office: 1,509 Days
• Field of Search: None
• CPC Class Codes: G06F 18/217 Validation; Performance evaluation; Active pattern learning techniques
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06V 10/776 Validation; Performance evaluation
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

People detection can provide valuable metrics that can be used by businesses, such as retail stores. Such information can be used to influence any number of business decisions such a employment hiring and product orders. The business value of this data hinges upon its accuracy. Thus, a method according to the principles of the current invention outputs metrics regarding people in a video frame within a stream of video frames through use of an object classifier configured to detect people. The method further comprises automatically updating the object classifier using data in at least a subset of the video frames in the stream of video frames.

Claims

1. A method of detecting people in a stream of images, the method comprising:

outputting metrics regarding people in a first subset of video frames within a stream of video frames through use of an object classifier configured to detect people as a function of image gradients calculated for edge information of objects, histogram of oriented gradient (HOG) features extracted from the image gradients calculated for the edge information, and automatically tunable coefficients;

the edge information including edge data of a head-shoulder area of the people;

identifying training samples by detecting a head-shoulder area in a second subset of video frames within the stream of video frames using at least one of (i) template matching and (ii) motion blobs and color blobs extracted from motion pixels and skin color pixels, the second subset of video frames including fewer frames than the first subset of video frames; and

automatically updating the object classifier using the training samples identified.

2. The method of claim 1 wherein automatically updating the object classifier includes updating on a periodic basis.

3. The method of claim 1 wherein automatically updating the object classifier is done in an unsupervised manner.

4. The method of claim 1 further comprising positioning a camera at an angle sufficient to allow the camera to capture the stream of video frames used to identify distinctions between features of people and background.

5. The method of claim 1 further comprising calculating the metrics at a camera capturing the stream of video frames.

6. The method of claim 1 further comprising calculating the metrics external from a camera capturing the stream of video frames.

7. The method of claim 1 further comprising:

processing the metrics to produce information; and

providing the information to a customer on a one time basis, periodic basis, or non-periodic basis.

8. The method of claim 1 wherein automatically updating the object classifier further comprises determining a level of confidence about the metrics.

9. The method of claim 1 wherein the training samples indicate a presence or an absence of people.

10. The method of claim 1 wherein the classifier detects people as a function of histogram of oriented gradient (HOG) features and tunable coefficients and wherein updating the classifier comprises automatically tuning the coefficients.

11. A system for detecting people in a stream of images, the system comprising:

an output module implemented by a processor, the output module configured to output metrics regarding people in a first subset of video frames within a stream of video frames through use of an object classifier configured to detect people as a function of image gradients calculated for edge information of objects, histogram of oriented gradient (HOG) features extracted from the image gradients calculated for the edge information, and automatically tunable coefficients;

the edge information including edge data of a head-shoulder area of the people; and

an update module implemented by the processor and configured to;

identify training samples by detecting a head-shoulder area in a second subset of video frames within the stream of video frames using at least one of (i) template matching and (ii) motion blobs and colors blobs extracted from motion pixels and skin color pixels, the second subset of video frames including fewer frames than the first subset of video frames; and

automatically update the object classifier using the training samples identified.

12. The system of claim 11 wherein the update module is further configured to update the object classifier on a periodic basis.

13. The system of claim 11 wherein the update module is configured to update the object classifier in an unsupervised manner.

14. The system of claim 11 further comprising a camera positioned at an angle sufficient to allow the camera to capture the stream of video frames used to identify distinctions between features of people and background.

15. The system of claim 11 further comprising a camera configured to capture the stream of video frames and calculate the metrics.

16. The system of claim 11 wherein the metrics are calculated external from a camera capturing the stream of video frames.

17. The system of claim 11 further comprising a processing module configured to process the metrics to produce information, the information to be provided to a customer on a one time basis, periodic basis, or non-periodic basis.

18. The system of claim 11 wherein automatically updating the object classifier further comprises determining a level of confidence about the metrics.

19. The system of claim 11 wherein the training samples indicate a presence or an absence of people.

20. The system of claim 11 wherein the classifier detects people as a function of histogram of oriented gradient (HOG) features and tunable coefficients and wherein updating the classifier comprises automatically tuning the coefficients.

21. A non-transitory computer readable medium having stored thereon a sequence of instructions which, when loaded and executed by a processor coupled to an apparatus, causes the apparatus to:

output metrics regarding people in a first subset of video frames within a stream of video frames through use of an object classifier configured to detect people as a function of image gradients calculated for edge information of objects, histogram of oriented gradient (HOG) features extracted from the image gradients calculated for the edge information, and automatically tunable coefficients;

the edge information including edge data of a head-shoulder area of the people;

identify training samples by detecting a head-shoulder area in a second subset of video frames within the stream of video frames using at least one of (i) template matching and (ii) motion blobs and color blobs extracted from motion pixels and skin color pixels, the second subset of video frames including fewer frames than the first subset of video frames; and

automatically update the object classifier using the training samples identified.