Video object classification with object size calibration

• US 9,697,425 B2
• Filed: 12/05/2014
• Issued: 07/04/2017
• Est. Priority Date: 03/03/2008
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Avigilon Analytics Corporation (Motorola Solutions, Inc.)
• Inventors: Saptharishi, Mahesh, Lisin, Dimitri A., Lipchin, Aleksey, Reyzin, Igor
• Primary Examiner(s): Tran, Nhan T
• Application Number:
  US14/562,572
• Publication Number:
  US 20150093035A1
• Time in Patent Office: 942 Days
• Field of Search: 348143, 348169, 348153-155, 382224, 382226
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 23/611 where the recognised objects include parts of the human body
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/631 Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters
• CPC Class Codes: H04N 23/66 Remote control of cameras or camera parts, e.g. by remote control devices
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A camera system comprises an image capturing device and an object classification module connected to the image capturing device. The object classification module is operable to determine whether or not an object in an image is a member of an object class. The object classification module includes multiple decision steps configured in a cascade configuration, wherein at least one of the multiple decision steps is operable to (a) accept an object as a member of the object class, (b) reject an object as a member of the object class, and (c) call on a next step to determine whether or not an object is a member of the object class.

Claims

1. A camera system comprising:

video analytics for processing image data representing an image of a field of view projected on an image plane of an image capturing device, the video analytics comprising;

an object classification module including an object classifier that is operable to classify objects captured in the field of view based on the image data, wherein the object classifier is operable to classify objects as members or non-members of an object class, wherein the object classifier includes a stage corresponding to a feature/transformation combination, and wherein the feature/transformation combination corresponding to a feature extracted from the image data and a discriminant function for mapping the feature to a scalar value, the feature being selected from a set of possible features and the discriminant function being selected from a set of possible discriminant functions, the feature and the discriminant function being selected so as to maximize object classification performance of the object classifier; and

a calibration module connected to the object classification module for estimating representative sizes of members of the object class, the representative sizes corresponding to different regions of the image plane, wherein the calibration module is operable to automatically update the representative sizes in response to classifications performed by the object classifier during live operation, and the calibration module is operable to supply information representing the updated representative sizes to the object classifier to improve its object classification performance.

2. A camera system according to claim 1, wherein when the object classifier classifies an object as a member of an object class, a confidence parameter is associated with that classification, and wherein the calibration module updates the representative sizes when the confidence parameter indicates a high confidence that the classification is correct.

3. A camera system according to claim 1, further comprising:

a user station having a display and an input device for providing user feedback information in response to classifications made by the object classifier, wherein the user feedback information is used to improve estimation of the representative sizes.

4. A camera system according to claim 1, wherein the calibration information from the calibration module is used to improve object detection and classification accuracy of the camera system.

5. A camera system according to claim 1, further comprising:

a speed estimation module connected to the calibration module and operable to estimate speeds of classified objects located in different regions of the field of view.

6. A camera system according to claim 1, in which the object classifier is operable to classify objects without requiring initial manual calibration.

7. A camera system according to claim 1, wherein the set of possible features includes one or more of an aspect ratio, an edge orientation histogram, color information, and normalized saturation.

8. A camera system according to claim 7, wherein the edge orientation histogram is produced by using a steerable filter.

9. A camera system according to claim 1, wherein the set of possible discriminant functions includes one or both of a radial basis function and a sigmoid function.

10. A camera system according to claim 1, wherein the object classifier is operable to classify an object as a human or non-human.

11. A camera system according to claim 1, wherein the object classifier is operable to classify an object as a vehicle or non-vehicle.

12. A method of automatically calibrating a camera system, the method comprising:

receiving image data representing an image of a scene, the image corresponding to an image plane on which the scene is projected;

detecting a first object in the image, an image of the first object being detected at a location of the image plane, the image of the first object having a size corresponding to the location;

extracting a feature from the image data, the feature being selected from a set of possible features and corresponding to a feature/transformation combination;

providing a discriminant function for mapping the feature to a scalar value, the discriminant function being selected from a set of possible discriminant functions, the feature and the discriminant function being selected so as to maximize object classification performance of an object classifier;

classifying, with the object classifier, the first object as a first member of an object class;

calculating a parameter of a size function for the image plane based on the size of the first object; and

updating the parameter of the size function in response to detection and classification of a second member of the object class.

13. A method according to claim 12, wherein the first member of the object class and the second member of the object class both correspond to the first object.

14. A method according to claim 12, wherein the second member of the object class corresponds to a second object different from the first object.

15. A method according to claim 12, wherein classification of the second member of the object class generates a confidence parameter that represents a measure of confidence that the second object is a member of the object class, and wherein the updating step is performed when the confidence parameter indicates a high confidence that the classification is correct.

16. A method according to claim 12, wherein the representative sizes are determined by a parameterized function of position.

17. A method according to claim 16, wherein the parameterized function of position is a second-order polynomial in X and Y coordinates.

18. A method according to claim 12, wherein the second member is detected and classified during live operation of the camera system.

19. A method according to claim 12, wherein:

the image data represents multiple images of the scene, the first object is detected in the multiple images, multiple images of the first object are detected in different locations of the image plane, the multiple images of the first object having different sizes corresponding to the different locations;

the first object is classified in the multiple images as the first member of the object class; and

the parameter of the size function is calculated based on the different sizes of the first object, the size function being used to estimate representative sizes of the object class for the image plane.

20. A camera system comprising:

video analytics for processing image data representing an image of a field of view projected on an image plane of an image capturing device, the video analytics comprising;

an object classification module including an object classifier that is operable to classify objects captured in the field of view based on the image data, wherein the object classifier is operable to classify objects as members or non-members of an object class, wherein the object classifier includes a stage corresponding to a feature/transformation combination, and wherein the feature/transformation combination corresponding to a feature extracted from the image data and a discriminant function for mapping the feature to a scalar value, the feature being selected from a set of possible features and the discriminant function being selected from a set of possible discriminant functions, the feature and the discriminant function being selected so as to maximize object classification performance of the object classifier; and

a calibration module connected to the object classification module for estimating representative sizes of members of the object class, the representative sizes corresponding to different regions of the image plane, wherein the calibration module is operable to automatically update the representative sizes in response to classifications performed by the object classifier during in-field operation, a confidence parameter is associated with classifications of objects by the classification module, and the confidence parameter is determined at least in part based on the representative size of the associated object.