Method of counting objects in a monitored environment and apparatus for the same

• US 20050078852A1
• Filed: 10/10/2003
• Published: 04/14/2005
• Est. Priority Date: 10/10/2003
• Status: Active Grant

• Current Assignee: Johnson Controls Tyco IP Holdings LLP (Johnson Controls International plc)
• Original Assignee: Sensormatic Electronics Corporation (Johnson Controls International plc)
• Inventors: Buehler, Christopher J.
• Granted Patent: US 7,346,187 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06V 20/53 Recognition of crowd images, e.g. recognition of crowd congestion

Abstract

Methods and system for determining, a number of objects, without tracking each of the objects, in first and second fields-of-view. First and second video frames are received from first and second image sources. The image sources have the first and second fields-of-view, and the fields-of-view are known to overlap at least in part. The number of objects is determined based on the first and second video frames and the known overlap.

Claims

1. A computerized method of video analysis comprising:

a) receiving a first video frame, the first video frame being generated by a first image source having a first field of view;

b) receiving a second video frame, the second video frame being generated by a second image source having a second field of view, the second field of view having a known overlap with at least part of the first field-of-view; and

c) determining a number of objects, without tracking each of the objects, in the first and second fields-of-view based on the first video frame and the second video frame and based on the known overlap.

2. The computerized method of video analysis of claim 1 wherein 1) the first video frame comprises a first plurality of image regions, 2) the second video frame comprises a second plurality of image regions, 3) each image region corresponds to a distinct field-of-view region, and 4) at least one set of overlapping image regions is known to correspond to one distinct field-of-view region, the computerized method further comprising c-1) creating a data structure identifying the image regions that correspond to each distinct field-of-view region based on the known overlap.

3. The computerized method of video analysis of claim 2 further comprising c-2) determining a number of objects included in a distinct field-of-view region based on the number of objects included in the image regions that correspond to the distinct field-of-view region.

4. The computerized method of video analysis of claim 3 wherein the number of objects included in a distinct field-of-view region is determined to be equal to the number of objects included in the image region, corresponding to the distinct field-of view region, that that includes the most objects.

5. The computerized method of video analysis of claim 2 wherein the number of objects in the first and second fields-of-view is determined to be equal to a total number of objects included in each distinct field-of-view region.

6. A computerized method of video analysis comprising:

a) receiving a plurality of first video frames, the first video frames being generated over a period time by a first image source having a first field of view;

b) receiving a plurality of second video frames, the second video frames being generated over a period of time by a second image source having a second field of view, the second field of view having a known overlap with at least part of the first field-of-view; and

c) determining a number of objects, without tracking each of the objects, in the first and second fields-of-view at an instant in time based on a first video frame and a second video frame generated at about the instant in time and based on the known overlap.

7. The computerized method of video analysis of claim 6 wherein 1) the first video frames comprise a first plurality of image regions, 2) the second video frames comprise a second plurality of image regions, 3) each image region corresponds to a distinct field-of-view region, and 4) at least one set of overlapping image regions is known to correspond to one distinct field-of-view region, the computerized method further comprising c-1) maintaining a data structure identifying the image regions that correspond to each distinct field-of-view region.

8. The computerized method of video analysis of claim 7 further comprising c-2) maintaining, for a distinct field-of-view region in the data structure, an average of the number of objects included in the distinct field-of-view region in frames in which such distinct field-of-view region has included an object.

9. The computerized method of video analysis of claim 8 further comprising:

c-3) determining that the maintained average for such distinct field-of-view region is greater than a threshold; and

c-4) subdividing the image regions corresponding to such distinct field-of-view region.

10. The computerized method of video analysis of claim 9 further comprising:

c-5) removing such distinct field-of-view region from the data structure; and

c-6) adding at least one of new distinct field-of-view region to the data structure corresponding to at least one subdivided image region.

11. The computerized method of video analysis of claim 9 wherein the threshold is greater than 1.0 and less than about 2.0.

12. The computerized method of video analysis of claim 7 further comprising c-2) determining a number of objects included in a distinct field-of-view region based on the number of objects included in the image regions that correspond to the distinct field-of-view region.

13. The computerized method of video analysis of claim 12 wherein the number of objects included in a distinct field-of-view region is determined to be equal to the number of objects included in the image region, corresponding to the distinct field-of view region, that that includes the most objects.

14. The computerized method of video analysis of claim 7 wherein the number of objects in the first and second fields-of-view is determined to be equal to a total number of objects included in each distinct field-of-view region.

15. The computerized method of claim 7 further comprising:

c-2) receiving updated overlap information; and

c-3) altering data in the data structure identifying image regions that correspond to a distinct field-of-view region based on the updated overlap information.

16. The computerized method of claim 6 further comprising d) averaging the number of objects included in the first and second fields-of-view over at least two instants in time.

17. A computer method of video analysis comprising:

a) receiving a first video frame, the first video frame being generated by a first image source having a first field of view within a monitored environment that has an environment gateway and an off-camera region;

b) receiving a subsequent first video frame from the first image source;

c) determining a number of objects, without tracking each of the objects, included in the off-camera region of the monitored environment based on the video frames.

18. The computerized method of video analysis of claim 17 further comprising d) determining a change in the number of objects, without tracking each of the objects,included in the monitored environment based on the location of the environment gateway in relation to objects included in the video frames.

19. The computerized method of video analysis of claim 17 wherein the environment gateway is one of an exit, an entrance, and an exit-entrance.

20. The computerized method of video analysis of claim 17 further comprising:

d) receiving a second video frame, the second video frame being generated by a second image source having a second field of view within the monitored environment; and

e) receiving a subsequent second video frame from the second image source.

21. The computerized method of video analysis of claim 20 wherein the first field-of-view overlaps at least in part with the second field-of-view.

22. The computerized method of video analysis of claim 20 wherein the first field-of-view does not overlap with the second field-of-view.

23. The computerized method of video analysis of claim 20 wherein the video frames include a plurality of distinct field-of-view regions, the method further comprising:

f) Determining a number of objects, without tracking each of the objects, included in the first and second fields-of-view based on the number of objects included in the distinct field-of-view regions of the first and second video frames; and

g) determining a subsequent number of objects, without tracking each of the objects, in the first and second fields-of-view based on the distinct field-of-view regions of the subsequent first video frame and the subsequent second video frame.

24. The computerized method of video analysis of claim 23 wherein the video frames comprise a plurality of image regions, at least one image region in the first video frames and one image region from the second video frames both correspond to the same distinct field-of-view region.

25. The computerized method of video analysis of claim 17 further comprising c-1) determining a change in the number of objects included entirely in the environment gateway.

26. The computerized method of video analysis of claim 25 further comprising c-2) determining a change in the number of objects included in the monitored environment that are only partially included in the environment gateway.

27. The computerized method of video analysis of claim 17 further comprising d) determining a number of objects, without tracking each of the objects, included in the monitored environment.

28. The computerized method of video analysis of claim 26 further comprising d-1) averaging the number of objects included in the monitored environment over a plurality of subsequent video frames.

29. The computerized method of video analysis of claim 26 further comprising:

e) calculating statistical values based on the number of objects included in the monitored environment.

30. A computerized system for video analysis comprising:

a) a receiving module configured to receive a first video frame, the first video frame being generated by a first image source having a first field of view, and to receiving a second video frame, the second video frame being generated by a second image source having a second field of view, the second field of view having a known overlap with at least part of the first field-of-view; and

b) a processing module configured to determine a number of objects, without tracking the objects, in the first and second fields-of-view based on the first video frame and the second video frame and based on the known overlap.

31. The computerized system of video analysis of claim 30 wherein 1) the first video frame comprises a first plurality of image regions, 2) the second video frame comprises a second plurality of image regions, 3) each image region corresponds to a distinct field-of-view region, and 4) at least one set of overlapping image regions is known to correspond to one distinct field-of-view region, the processing module being further configured to create a data structure identifying the image regions that correspond to each distinct field-of-view region based on the known overlap.

32. The computerized system for video analysis of claim 31, the processing module further configured to determine a number of objects included in a distinct field-of-view region based on the number of objects included in the image regions that correspond to the distinct field-of-view region.

33. The computerized system for video analysis of claim 32 wherein the processing module determines the number of objects included in a distinct field-of-view region to be equal to the number of objects included in the image region, corresponding to the distinct field-of view region, that that includes the most objects.

34. The computerized system for video analysis of claim 31 wherein the processing module determines the number of objects in the first and second fields-of-view to be equal to a total number of objects included in each distinct field-of-view region.

35. A computerized system for video analysis comprising:

a) A receiving module configured to receive a plurality of first video frames, the first video frames being generated over a period time by a first image source having a first field of view and to receive a plurality of second video frames, the second video frames being generated over a period of time by a second image source having a second field of view, the second field of view having a known overlap with at least part of the first field-of-view; and

b) a processing module configured to determine a number of objects, without tracking the objects, in the first and second fields-of-view at an instant in time based on a first video frame and a second video frame generated at about the instant in time and based on the known overlap.

36. The computerized system for video analysis of claim 35 wherein 1) the first video frames comprise a first plurality of image regions, 2) the second video frames comprise a second plurality of image regions, 3) each image region corresponds to a distinct field-of-view region, and 4) at least one set of overlapping image regions is known to correspond to one distinct field-of-view region, the processing module further configured to maintain a data structure identifying the image regions that correspond to each distinct field-of-view region.

37. The computerized system for video analysis of claim 36, the processing module further configured to maintain, for a distinct field-of-view region in the data structure, an average of the number of objects included in the distinct field-of-view region in frames in which such distinct field-of-view region has included an object.

38. The computerized system for video analysis of claim 37, the processing module further configured to determine whether the maintained average for such distinct field-of-view region is greater than a threshold, and to subdivide the image regions corresponding to such distinct field-of-view region.

39. The computerized system for video analysis of claim 38, the processing module further configured to remove such distinct field-of-view region from the data structure, and to add at least one of new distinct field-of-view region to the data structure corresponding to at least one subdivided image region.

40. The computerized system for video analysis of claim 38 wherein the threshold is greater than 1.0 and less than about 2.0.

41. The computerized system for video analysis of claim 36, the processing module further configured to determine a number of objects included in a distinct field-of-view region based on the number of objects included in the image regions that correspond to the distinct field-of-view region.

42. The computerized system for video analysis of claim 41 wherein the processing module determines the number of objects included in a distinct field-of-view region to be equal to the number of objects included in the image region, corresponding to the distinct field-of view region, that that includes the most objects.

43. The computerized system for video analysis of claim 36 wherein the processing module determines the number of objects in the first and second fields-of-view to be equal to a total number of objects included in each distinct field-of-view region.

44. The computerized system for claim 36, the processing module further configured to receive updated overlap information, and to alter data in the data structure identifying image regions that correspond to a distinct field-of-view region based on the updated overlap information.

45. The computerized system for claim 35, the processing module further configured to average the number of objects included in the first and second fields-of-view over at least two instants in time.

46. A computer system for video analysis comprising:

a) a receiving module configured to receive a first video frame, the first video frame being generated by a first image source having a first field of view within a monitored environment that has an environment gateway and an off-camera region and to receive a subsequent first video frame from the first image source;

b) a processing module configured to determine a number of objects, without tracking the objects, included in the off-camera region of the monitored environment based on the video frames.

47. The computerized system for video analysis of claim 46, the processing module further configured to determine a change in the number of objects, without tracking the objects, included in the monitored environment based on the location of the environment gateway in relation to objects included in the video frames.

48. The computerized system for video analysis of claim 46 wherein the environment gateway is one of an exit, an entrance, and an exit-entrance.

49. The computerized system for video analysis of claim 46 wherein the receiving module is further configured to receive a second video frame, the second video frame being generated by a second image source having a second field of view within the monitored environment and to receiving a subsequent second video frame from the second image source.

50. The computerized system for video analysis of claim 49 wherein the first field-of-view overlaps at least in part with the second field-of-view.

51. The computerized system for video analysis of claim 49 wherein the first field-of-view does not overlap with the second field-of-view.

52. The computerized system for video analysis of claim 49 wherein the video frames include a plurality of distinct field-of-view regions, the processing module being further configured to determine a number of objects, without tracking the objects, included in the first and second fields-of-view based on the number of objects included in the distinct field-of-view regions of the first and second video frames, and to determine a subsequent number of objects, without tracking the objects, in the first and second fields-of-view based on the distinct field-of-view regions of the subsequent first video frame and the subsequent second video frame.

53. The computerized system for video analysis of claim 52 wherein the video frames comprise a plurality of image regions, at least one image region in the first video frames and one image region from the second video frames both correspond to the same distinct field-of-view region.

54. The computerized system for video analysis of claim 46, the processing module further configured to determine a change in the number of objects included entirely in the environment gateway.

55. The computerized system for video analysis of claim 54, the processing module further configured to determine a change in the number of objects included in the monitored environment that are only partially included in the environment gateway.

56. The computerized system for video analysis of claim 46, the processing module further configured to determine a number of objects, without tracking the objects, included in the monitored environment.

57. The computerized system for video analysis of claim 56, the processing module further configured to average the number of objects included in the monitored environment over a plurality of subsequent video frames.

58. The computerized system for video analysis of claim 56 the processing module further configured to calculate statistical values based on the number of objects included in the monitored environment.