3D human pose and shape modeling

• US 9,165,190 B2
• Filed: 03/15/2013
• Issued: 10/20/2015
• Est. Priority Date: 09/12/2012
• Status: Active Grant

• Current Assignee: Motorola Solutions, Inc.
• Original Assignee: Avigilon Fortress Corporation
• Inventors: Zhang, Zhong, Yin, Weihong, Venetianer, Peter
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US13/838,511
• Publication Number:
  US 20140072170A1
• Time in Patent Office: 949 Days
• Field of Search: 382/100, 382/103, 382106-107, 382117-118, 382154-155, 382/162, 382/168, 382/173, 382/181, 382189-199, 382/209, 382/219, 382/232, 382/254, 382/274, 382/276, 382/291, 382/305, 382/312, 348/143, 348/142, 348/154, 348/159
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion
• CPC Class Codes: G06V 20/54 of traffic, e.g. cars on the road, trains or boats
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

Methods, devices and systems for performing video content analysis to detect humans or other objects of interest a video image is disclosed. The detection of humans may be used to count a number of humans, to determine a location of each human and/or perform crowd analyses of monitored areas.

Claims

1. A method of detecting human objects in a video, comprising:

determining pixels of a video image are foreground pixels, the group of foreground pixels constituting a foreground blob set of one or more foreground blobs;

for each of N locations within the video image, where N is an integer, comparing a corresponding predetermined shape with the foreground blob set to obtain a corresponding probability of a human at the location, thereby obtaining N probabilities corresponding to the N locations;

using the N probabilities, determining X humans are represented by the foreground blob set, where X is whole number;

using the determination of the representation of X humans, determining a crowd density within a first area of the real world;

comparing the crowd density to a threshold; and

providing at least one of a report, an alarm, and an event detection when the crowd density exceeds the threshold,

wherein a size of the corresponding predetermined shape for each of the N locations is determined in response to calibration of a video system,

wherein the video system is used to obtain the video image.

2. The method of claim 1, further comprising using the probability map to determine a location of each of the X humans.

3. The method of claim 2, wherein the determined location of each of the X humans is a location within an image plane corresponding to the video image.

4. The method of claim 2, wherein the determined location of each of the X humans is a location with respect to a physical ground plane corresponding to the real world.

5. The method of claim 1,

wherein determining foreground pixels of the video image comprises comparison of a first frame of a video image without foreground objects with comparison of a second frame of the video image containing the foreground objects.

6. The method of claim 1, wherein the predetermined shape is the same for each of the N locations.

7. The method of claim 1, wherein the predetermined shape for at least some of the N locations has a different size.

8. The method of claim 1,

wherein the calibration of the video system comprises determining an image size of a portion of the video image corresponding to an average human size at each of the N locations, and

wherein the size of the predetermined shape for each of the N locations is determined in response to the corresponding image size.

9. The method of claim 1,

wherein, for each of the N locations, the corresponding predetermined shape comprises an estimate of a foreground image part to be occupied in the video image when a human exists at the corresponding location.

10. The method of claim 9, wherein the estimate of the foreground image part for each of the N locations is calculated based on a projection of a model of a human in the real world onto an image plane of the video image.

11. The method of claim 1,

wherein the video image comprises a plurality of image frames, each image frame comprising a two dimensional image having the N locations, each of the N locations identified by a corresponding x, y coordinate pair within the two dimensional image.

12. The method of claim 11,

wherein each of the N locations is associated with a corresponding predetermined shape with respect to an image plane corresponding to the video image.

13. The method of claim 1, further comprising, for each of the N locations, calculating a recall ratio of the corresponding predetermined shape and the foreground blob to determine an associated probability.

14. The method of claim 13, wherein for each of the N locations, calculating the recall ratio comprises determining a ratio of (a) an area comprising an overlap of an area occupied by the predetermined shape and the foreground blob and (b) an area of the foreground blob.

15. The method of claim 1, further comprising:

creating a probability map with the N probabilities;

determining local maximums of probabilities of the probability map.

16. The method of claim 15, further comprising:

selecting a first location of the N locations corresponding to a local maximum of the probability map;

obtaining a first predetermined shape corresponding to the first location; and

calculating a first ratio of (a) an area comprising an overlap of an area occupied by the first predetermined shape and the foreground blob and (b) an area of the foreground blob.

17. The method claim 16, wherein the first ratio is used to determine that X humans are represented by the foreground blob.

18. The method of claim 16, further comprising:

calculating an overlap of an area occupied by the second predetermined shape and an area occupied by the first predetermined shape.

19. The method of claim 15, further comprising calculating a precision value and a recall value for each of m locations of the N locations, m being an integer, each of the m locations corresponding to a local maximum of the probability map.

20. The method of claim 19, wherein each of the m locations are selected sequentially 1 to m, a selection of an (m−

1)th location excluding selection of an mth location that falls within a first predetermined distance of the (m−

1)th location.

21. The method of claim 20, wherein each of the m locations are selected sequentially 1 to m wherein the selection of a next location of the m locations comprises selecting a location that is closest to a bottom edge of the video image for those locations corresponding to a local maximum that have not been excluded.

22. A method of detecting human objects in a video, comprising:

determining pixels of a video image of a real world scene are foreground pixels, the group of foreground pixels constituting a foreground blob set of one or more foreground blobs;

for each of N locations within the video image, where N is an integer, comparing a corresponding predetermined shape with the foreground blob set to determine X humans are represented by the foreground blob set, where X is whole number and a location of each of the X humans is determined as a location within a horizontal plane of the real world;

using the determination of the representation of X humans, determining a crowd density within a first area of the real world;

comparing the crowd density to a threshold; and

providing at least one of a report, an alarm, and an event detection when the crowd density exceeds the threshold,

wherein a size of the corresponding predetermined shape for each of the N locations is determined in response to calibration of a video system,

wherein the video system is used to obtain the video image.

23. The method of claim 22, further comprising detecting the existence of a crowd by reviewing at least some of the locations of the X humans.

24. The method of claim 22, further comprising determining an existence of a crowd when it is determined that Y of the X humans are located within a an area of the horizontal plane of the real world.

25. The method of claim 24, wherein the first area comprises a predetermined geometric shape having a predetermined area size within the real world.

26. The method of claim 24, wherein the first area comprises an area defined by a circle.

27. The method of claim 22, further comprising:

determining a first crowd density within the first area corresponding to a first frame of the video image;

determining a second crowd density within the first area corresponding to a second frame of the video image;

determining a crowd gathering event in response to the first crowd density and the second crowd density.

28. The method of claim 22, further comprising:

determining a first crowd density within the first area corresponding to a first frame of the video image;

determining a second crowd density within the first area corresponding to a second frame of the video image;

determining a crowd dispersing event in response to the first crowd density and the second crowd density.