Identification apparatus and method for identifying properties of an object detected by a video surveillance camera

• US 20080031493A1
• Filed: 05/22/2007
• Published: 02/07/2008
• Est. Priority Date: 05/22/2006
• Status: Active Grant

• Current Assignee: Axis AB (Canon Inc.)
• Original Assignee: Axis AB (Canon Inc.)
• Inventors: Brogren, Martin, Skans, Markus, Elvin, Daniel, Almbladh, Johan
• Granted Patent: US 8,478,040 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/758 Involving statistics of pixels or of feature values, e.g. histogram matching
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

The present invention deals with an identification apparatus (100) in a video surveillance system for identifying properties of an object captured in a video sequence by a video surveillance camera. The identification apparatus comprises: an object identification unit (102) for identifying a specific object in a video sequence; a color histogram generator (104) for generating a color histogram in at least two dimensions of a color space based on color and intensity information of the specific object identified in the video sequence, the color and intensity information originating from a plurality of images of the video sequence; and an object properties identificator (106) for identifying properties of the object based on the generated color histogram. The identified properties can then be used in a tracking device (200) of the video surveillance system for tracking an object between different video sequences, which may be captured by two different video surveillance cameras. The present invention also deals with a corresponding method for identifying properties of an object captured in a video sequence and a method for tracking the object in a video surveillance system.

Claims

1. An identification apparatus in a video surveillance system, the identification apparatus comprising:

an object identification unit for identifying a specific object in a video sequence;

a color histogram generator for generating a color histogram in at least two dimensions of a color space based on color and intensity information of the specific object identified in the video sequence, the color and intensity information originating from a plurality of images of the video sequence; and

an object properties identificator for identifying properties of the object based on the generated color histogram.

2. Identification apparatus according to claim 1, wherein the object properties identificator further comprises:

a histogram modeler for modeling the color histogram with a number of distributions, such as normal distributions.

3. Identification apparatus according to claim 2, wherein the object properties identificator further comprises:

a parameter calculator for calculating parameters of the number of distributions for identifying properties of the object; and

an object properties calculator for calculating properties of the object based on the calculated parameters.

4. Identification apparatus according to claim 1, wherein the object properties identificator further comprises:

a histogram transformer for transforming the color histogram from a Cartesian coordinate system into a polar coordinate system.

5. Identification apparatus according to claim 3, wherein the object properties calculator is arranged for calculating the properties of the object by weighting the parameters of the number of distributions depending on each parameter'"'"'s significance in recognition of the object.

6. Identification apparatus according to claim 4, wherein the histogram transformer is further arranged for transforming the color histogram from the polar coordinate system to a second Cartesian coordinate system, and arranging the origin of the second Cartesian coordinate system at the centre of mass of the object in the color histogram.

7. Identification apparatus according to claim 1, wherein the identification apparatus further comprises a normalization unit for normalizing the color and intensity information originating from each of the plurality of images with respect to the number of pixels the object is described by in each image.

8. Identification apparatus according to claim 1, wherein the color space is a color space in which intensity information and color information can be separated, such as YCbCr, HSV or YUV color space.

9. Identification apparatus according to claim 1, wherein the object properties identificator is arranged for identifying, based on the generated color histogram, parameters describing the object in the video sequence, and wherein the object properties identificator comprises an object properties calculator for calculating, based on the identified parameters, properties of the object.

10. Identification apparatus according to claim 1, wherein the object properties identificator is arranged for identifying a center of mass of the color histogram and a color histogram distribution;

and the object properties calculator is arranged for calculating parameters for identifying properties of the object based on the identified center of mass and the identified color histogram distribution.

11. A video surveillance camera comprising an identification apparatus according to claim 1.

12. A video surveillance system comprising:

a number of video surveillance cameras connected to a common communication network;

at least one identification apparatus connected to the common communication network for generating a color histogram of a specific object identified in a video sequence and for identifying properties of the specific object based on the color histogram;

a tracking device for tracking an object in a video surveillance system, the tracking device being connected to the common communication network, said device comprising;

a receiver for receiving information from the at least one identification apparatus, the information comprising information regarding identified properties of a candidate object identified in a first video sequence captured by a first video surveillance camera and information regarding identified properties of an object to track identified in a second video sequence captured by a second video surveillance camera;

a matching unit for matching the identified properties of the candidate object with the identified properties of the object to track; and

a calculation unit for calculating, based on the matching step, the likelihood of the candidate object being identical to the object to track.

13. Video surveillance system according to claim 12, wherein the first video surveillance camera and the second video surveillance camera are different video surveillance cameras in the video surveillance system.

14. Video surveillance system according to claim 12, wherein the first video surveillance camera and the second video surveillance camera is the same video surveillance camera in the video surveillance system.

15. Video surveillance system according to claim 12, wherein the tracking device further comprises:

a signal generator for generating a signal indicating the likelihood of the candidate object being identical to the object to track.

16. Video surveillance system according claim 12, wherein the information received from the at least one identification apparatus regarding the identified properties of the candidate object comprises parameters of the color histogram of the candidate object and the information received from the at least one identification apparatus regarding the identified properties of the object to track comprises parameters of the color histogram of the object to track, and wherein the matching unit is arranged for matching the parameters of the color histogram of the candidate object with the parameters of the color histogram of the object to track.

17. Video surveillance system according to claim 12, wherein the tracking device is situated at a central node of the system, connected to the number of video surveillance cameras via the common communication network.

18. A method in a video surveillance system for identifying properties of an object in a video sequence captured by a video surveillance camera, the method comprising the steps of:

identifying a specific object in the video sequence;

generating a color histogram in at least two dimensions of a color space, based on color and intensity information of the specific object identified in the video sequence, the color and intensity information originating from a plurality of images of the video sequence;

identifying properties of the object based on the generated color histogram.

19. Method according to claim 18, wherein the step of identifying properties of the object based on the generated color histogram comprises the steps of:

modeling the color histogram with a number of distributions, such as normal distributions, and

identifying properties of the object based on the generated and modeled color histogram;

20. Method according to claim 19, wherein the step of identifying properties of the object based on the generated and modeled color histogram further comprises:

calculating parameters of the number of distributions for identifying properties of the object; and

calculating properties of the object based on the calculated parameters.

21. Method according to claim 19, wherein the color histogram is modeled with a number of distributions using a distribution fitting algorithm, such as an Expectation Maximization algorithm.

22. Method according to claim 18, wherein the method further comprises the step of:

Transforming the color histogram from a Cartesian coordinate system into a polar coordinate system.

23. Method according to claim 20, wherein the step of calculating properties of the object based on the calculated parameters comprises:

Calculating properties of the object by weighting the parameters of the number of distributions depending on each parameter'"'"'s correlation with the properties of the object.

24. Method according to claim 22, wherein the step of calculating parameters of the distributions comprises:

Transforming the color histogram from the polar coordinate system to a second Cartesian coordinate system, and

arranging the origin of the second Cartesian coordinate system at the centre of mass of the object in the color histogram.

25. Method according to claim 24, wherein the step of calculating parameters further comprises the step of calculating the position of the number of distributions in the second Cartesian coordinate system, and wherein the position of the number of distributions in the second Cartesian coordinate system is used as a parameter for identifying properties of the object.

26. Method according to claim 18, wherein in the color histogram each color part of an object is represented by a blob having a certain extension in the color histogram, and wherein the object is described as a cluster of such blobs.

27. Method according to claim 20, wherein in a color histogram each color part of an object is represented by a blob having a certain extension in the color histogram, and wherein the object is described as a cluster of such blobs, and wherein the parameters of the number of distributions used for identifying properties of the object comprise:

centre of mass (ρ

_cw, α

_cw) of the cluster of blobs, reference angle (φ

) of the cluster of blobs, relative frequencies (π

_k) of each distribution, position of the distributions (ρ

1_k,ρ

2_k) in the co-ordinate system,

where k=1,2, . . . ,K is the distribution number and K is the total number of distributions in the model.

28. Method according to claim 18, wherein the histogram is generated in a two-dimensional color plane.

29. Method according to claim 18, wherein the color histogram is generated based on color and intensity information in substantially all images of the video sequence.

30. Method according to claim 18, wherein the step of identifying properties of the object based on the color histogram comprises:

identifying, based on the color histogram, parameters describing the object in the video sequence;

calculating, based on the identified parameters, properties of the object.

31. Method according to claim 30, wherein the step of identifying parameters describing the object in the video sequence comprises:

identifying a center of mass of the color histogram and a color histogram distribution; and

calculating parameters for identifying properties of the object based on the identified centre of mass and the identified color histogram distribution.

32. Method according to claim 31, wherein the center of mass is identified as an expectancy value and the extension as a covariance matrix.

33. A method for tracking an object in a video surveillance system, the method comprising the steps of:

identifying an object to track in a first video sequence captured by a first video surveillance camera;

identifying properties of the object to track in accordance with the method of claim 18;

identifying a candidate object in a second video sequence captured by a second video surveillance camera;

identifying properties of the candidate object in accordance with the method of claim 18;

matching the identified properties of the candidate object with the identified properties of the object to track;

calculating, based on the matching step, the likelihood of the candidate object being identical to the object to track.

34. Method according to claim 33, wherein the step of matching further comprises the steps of:

detecting a translation between the cluster of blobs generated for the object to track and the cluster of blobs generated for the candidate object;

detecting a rotation between the cluster of blobs generated for the object to track and the cluster of blobs generated for the candidate object;

detecting an internal translation between the blobs in the color histogram generated for the object to track and the blobs in the color histogram generated for the candidate object; and

comparing the detected translation, rotation and internal translation.

35. Method according to claim 34, wherein the step of comparing the detected translation, rotation and internal translation further comprises:

comparing the detected translation, rotation and internal translation by comparing parameters indicating the translation, rotation and internal translation; and

weighting the compared parameters depending on their significance for calculating the likelihood of the candidate object being identical to the object to track.

36. Method according to claim 33, the method further comprising the step of:

generating a signal indicating the likelihood of the candidate object being identical to the object to track.

37. Method according to claim 33, wherein the step of matching the identified properties of the candidate object with the identified properties of the object to track comprises:

comparing the number of blobs in the color histogram of the object to track with the number of blobs in the color histogram of the candidate object.

38. Method according to claim 37, wherein in the step of calculating the likelihood of the candidate object being identical to the object to track, the likelihood is lowered with a value depending on the difference as a percentage between the number of blobs in the color histogram of the object to track and the number of blobs in the color histogram of the candidate object.

39. Method according to claim 33, wherein the step of matching the identified properties of the candidate object with the identified properties of the object to track comprises:

matching parameters of the color histogram describing the candidate object with parameters of the color histogram describing the object to track.

40. Method according to claim 39, wherein the step of matching further comprises:

weighting the difference between matched parameters for the candidate object and the object to track depending on significance of the parameter in identifying properties of the object.