Video object classification

• US 9,659,238 B2
• Filed: 04/15/2014
• Issued: 05/23/2017
• Est. Priority Date: 08/28/2008
• Status: Active Grant

• Current Assignee: Kyndryl Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Brown, Lisa Marie, Feris, Rogerio Schmidt, Zhai, Yun, Hampapur, Arun, Chen, Longbin
• Primary Examiner(s): Newman, Michael A.
• Application Number:
  US14/253,266
• Publication Number:
  US 20150098661A1
• Time in Patent Office: 1,134 Days
• Field of Search: 382224, 382103, 382107, 382159, 382170, 382181, 382228
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06V 10/235 based on user input or interaction
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: H04W 4/02 Services making use of location information

Abstract

A system comprises an input component, a feature extractor, an object classifier, an adaptation component and a calibration tool. The input component is configured to receive one or more images, and the feature extractor is configured to extract features for one or more objects in the one or more images, the extracted features comprising at least one view-independent feature. The object classifier is configured to classify the one or more objects based at least in part on the extracted features and one or more object classification parameters, and the adaptation component is configured to adjust the classification of at least one of the objects based on one or more contextual parameters. The calibration tool is configured to adjust one or more of the object classification parameters based on likelihoods for characteristics associated with one or more object classes.

Claims

1. A system comprising:

an input component configured to receive one or more images;

a feature extractor configured to extract features for one or more objects in the one or more images, the extracted features comprising at least one view-independent feature, the one or more images comprising a video and the at least one view-independent feature comprising a level of deformation of at least one of the objects in the video, the level of deformation being determined utilizing differences of histograms of oriented gradients;

an object classifier configured to classify the one or more objects based at least in part on the extracted features and one or more object classification parameters;

an adaptation component configured to adjust the classification of at least one of the objects based on one or more contextual parameters; and

a calibration tool configured to adjust one or more of the object classification parameters based on likelihoods for characteristics associated with one or more object classes, the likelihoods comprising a location likelihood and a size likelihood, the location likelihood indicating that probability of a given object class at a given location in an image and the size likelihood indicating the probability of a given object size for a given object class and a given object location in the image;

wherein the input component, feature extractor, object classifier, adaptation component and calibration tool are implemented using at least one processing device comprising a processor coupled to a memory.

2. The system of claim 1, wherein the extracted features further comprise one or more view-dependent features comprising respective sizes of one or more of the objects, respective velocities of one or more of the objects, and respective locations of one or more of the objects.

3. The system of claim 1, wherein the one or more object classification parameters comprises object types.

4. The system of claim 3, wherein the object types comprise persons and vehicles.

5. The system of claim 3, wherein the object types comprise corresponding probability distributions for respective ones of the extracted features.

6. The system of claim 1, wherein the one or more contextual parameters comprises a camera view.

7. The system of claim 1, wherein the one or more contextual parameters comprises a time of day.

8. The system of claim 1, wherein the adaptation component is configured to modify histograms of the extracted features based on object samples having classification confidence values above a specified threshold.

9. The system of claim 8, wherein the adaptation component is configured to weight object samples based on the age of the object samples.

10. The system of claim 1, wherein the calibration tool is configured to define a normalized size of one or more object classes in accordance with one or more object models.

11. The system of claim 1, further comprising an interactive interface.

12. The system of claim 11, wherein the interactive interface is configured to permit a user to define the likelihoods for characteristics associated with the one or more object classes.

13. The system of claim 12, wherein the interactive interface is configured to permit the user to define one or more regions of interest in at least one of the images, wherein the likelihoods for characteristics associated with the one or more object classes are specific to a given region of interest.

14. A method comprising:

receiving one or more images;

extracting features for one or more objects in the one or more images, the extracted features comprising at least one view-independent feature, the one or more images comprising a video and the at least one view-independent feature comprising a level of deformation of at least one of the objects in the video, the level of deformation being determined utilizing differences of histograms of oriented gradients;

classifying the one or more objects based at least in part on the extracted features and one or more object classification parameters;

adjusting the classification of at least one of the objects based on one or more contextual parameters; and

adjusting one or more of the object classification parameters based on likelihoods for characteristics associated with one or more object classes, the likelihoods comprising a location likelihood and a size likelihood, the location likelihood indicating that probability of a given object class at a given location in an image and the size likelihood indicating the probability of a given object size for a given object class and a given object location in the image;

wherein the method is implemented using at least one processing device comprising a processor coupled to a memory.

16. The method of claim 14, wherein the extracted features further comprise one or more view-dependent features comprising respective sizes of one or more of the objects, respective velocities of one or more of the objects, and respective locations of one or more of the objects.

17. The method of claim 14, wherein adjusting the classification of at least one of the objects based on one or more contextual parameters comprises modifying histograms of the extracted features based on object samples having classification confidence values above a specified threshold, wherein the object samples are weighted based on age.

18. The method of claim 14, further comprising providing an interactive interface configured:

to permit a user to define the likelihoods for characteristics associated with the one or more object classes; and

to permit the user to define one or more regions of interest in at least one of the images, wherein the likelihoods for characteristics associated with the one or more object classes are specific to a given region of interest.

15. A computer program product comprising a non-transitory computer-readable storage medium including one or more programs which, when executed by a computer, implement the steps of:

receiving one or more images;

extracting features for one or more objects in the one or more images, the extracted features comprising at least one view-independent feature, the one or more images comprising a video and the at least one view-independent feature comprising a level of deformation of at least one of the objects in the video, the level of deformation being determined utilizing differences of histograms of oriented gradients;

classifying the one or more objects based at least in part on the extracted features and one or more object classification parameters;

adjusting the classification of at least one of the objects based on one or more contextual parameters; and

adjusting one or more of the object classification parameters based on likelihoods for characteristics associated with one or more object classes, the likelihoods comprising a location likelihood and a size likelihood, the location likelihood indicating that probability of a given object class at a given location in an image and the size likelihood indicating the probability of a given object size for a given object class and a given object location in the image.

19. The computer program product of claim 15, wherein adjusting the classification of at least one of the objects based on one or more contextual parameters comprises modifying histograms of the extracted features based on object samples having classification confidence values above a specified threshold, wherein the object samples are weighted based on age.

20. The computer program product of claim 15, further comprising providing an interactive interface configured:

to permit a user to define the likelihoods for characteristics associated with the one or more object classes; and

to permit the user to define one or more regions of interest in at least one of the images, wherein the likelihoods for characteristics associated with the one or more object classes are specific to a given region of interest.