Method and apparatus for detecting people by a surveillance system

• US 9,367,733 B2
• Filed: 11/21/2012
• Issued: 06/14/2016
• Est. Priority Date: 11/21/2012
• Status: Active Grant

• Current Assignee: Pelco (Schneider Electric SE)
• Original Assignee: Pelco (Schneider Electric SE)
• Inventors: Zhu, Hongwei, Wang, Lei, Aghdasi, Farzin, Millar, Greg M.
• Primary Examiner(s): Harandi, Siamak
• Assistant Examiner(s): Zohrabyan, Narek
• Application Number:
  US13/683,977
• Publication Number:
  US 20140139660A1
• Time in Patent Office: 1,301 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

Surveillance systems may be found in both private and public spaces. In private spaces, they can be designed to help provide and monitor secure premises. Similarly, public spaces may also use surveillance systems to determine an allocation of public resources. A camera surveillance system according to an embodiment of the invention uses advanced image processing techniques to determine whether an object moving across a scene is a person. The camera surveillance system achieves an accurate and efficient classification by selectively processing a set of features associated with the object, such as features that define an omega shape. By selectively processing the set of features associated with the object, the methods and systems described herein reduce the computational complexity of standard image processing/object detection techniques.

Claims

1. A method for detecting people by a surveillance system, the method comprising:

identifying a plurality of foreground contour shapes of objects within a scene and a respective track for each of the plurality of foreground contour shapes from at least two frames in a video file;

determining a plurality of features of the foreground contour shapes within the scene and the tracks of the foreground contour shapes, the features including a set of salient moving directions, and directional aspect ratios;

assigning a respective intra-frame local detector to each location in the scene visited by the foreground contour shapes; and

classifying at least a portion of each of the foreground contour shapes by using a feature of the foreground contour shape and the respective track of each foreground contour shape and applying the assigned respective intra-frame detector to the foreground contour shapes of the visited scene locations to determine whether each foreground contour shape matches a person reference model.

2. The method of claim 1, further comprising

extracting features from the foreground contour shapes within the scene, the features including at least one of the following:

an object shape, object size, object height, object width, aspect ratio, directional aspect ratio of salient moving direction, head-shoulder feature, aspect ratio of object shape and histogram for chain codes of object shape.

3. The method of claim 1, further comprising:

identifying respective chain codes defining a perimeter of the foreground contour shapes;

determining at least object shape and object size from the respective chain codes; and

classifying the object shape and object size from the chain codes to determine whether each foreground contour shape matches the person reference model.

4. The method of claim 3, further comprising:

determining if the calculated object size is large wherein a large object comprises at least 30% of frame size of the video file.

5. The method of claim 1, further comprising:

extracting salient moving directions from the respective tracks of the foreground contour shapes in the video file;

extracting respective directional aspect ratios from the tracks of the foreground contour shapes in the video file;

and determining whether each foreground contour shape matches the person reference model based on the salient moving directions and the directional aspect ratios.

6. The method of claim 5, further comprising:

determining a respective expected movement from the respective tracks of the foreground contour shapes, extracted salient moving directions and directional aspect ratios; and

propagating features of the tracks of the foreground contour shapes.

7. The method of claim 1 wherein determining whether the foreground contour shapes match the person reference model includes detecting a head and shoulder shape.

8. The method of claim 7 wherein detecting a head and shoulder shape includes applying an omega classifier.

9. A non-transitory computer readable medium, the computer readable medium having program instructions stored thereon, the program instructions being executable by a processor, and, when executed by a processor, cause the processor to:

identify a plurality of foreground contour shapes of objects within a scene and a respective track for each of the plurality of foreground contour shapes from at least two frames in a video file;

determine a plurality of features of the foreground contour shapes within the scene and the tracks of the foreground contour shapes, the features including a set of salient moving directions, and directional aspect ratios;

assign a respective intra-frame local detector to each location in the scene visited by the foreground contour shapes; and

classify at least a portion of each of the foreground contour shapes by using a feature of the foreground contour shape and the respective track of each foreground contour shape and applying the assigned respective intra-frame detector to the foreground contour shapes of the visited scene locations to determine whether each foreground contour shape matches a person reference model.

10. The computer readable medium of claim 9, further including program instructions that when executed by the processor, cause the processor to:

extract features from the foreground contour shapes within the scene, the features including at least one of the following;

an object shape, object size, object height, object width, aspect ratio, directional aspect ratio of salient moving direction, head-shoulder feature, aspect ratio of object shape and histogram for chain codes of object shape.

11. The computer readable medium of claim 9, further including program instructions that when executed by the processor, cause the processor to:

identify respective chain codes defining a perimeter of the foreground contour shapes;

determine at least object shape and object size from the respective chain codes; and

classify the object shape and object size from the chain codes to determine whether each foreground contour shape matches the person reference model.

12. A surveillance system, comprising:

a camera configured to capture image data from a scene;

a memory with computer code instructions stored thereon, the memory with one or more processors, and the computer code instructions being configured to cause the system to implement a scene analyzer in communication with the camera and configured to receive image data of the scene, the scene analyzer further configured to;

identify a plurality of foreground contour shapes of objects within a scene and a respective track for each of the plurality of foreground contour shapes from at least two frames in a video file;

determine a plurality of features of the foreground contour shapes within the scene and the tracks of the foreground contour shapes, the features including a set of salient moving directions, and directional aspect ratios;

assign a respective intra-frame local detector to each location in the scene visited by the foreground contour shapes; and

classify at least a portion of each of the foreground contour shapes by using a feature of the foreground contour shape and the respective track of each foreground contour shape and applying the assigned respective intra-frame detector to the foreground contour shapes of the visited scene locations to determine whether each foreground contour shape matches a person reference model.

13. The surveillance system of claim 12, wherein the camera is a fixed surveillance camera.

14. The surveillance system of claim 12, wherein the camera is configured to support at least two simultaneous video streams.

15. The surveillance system of claim 14, wherein the at least two simultaneous video streams are customizable.

16. The surveillance system of claim 14, wherein the scene analyzer is configured to:

customize a high bit rate stream according to intra-frame local detector instructions.