Computerized method and apparatus for determining field-of-view relationships among multiple image sensors

• US 20050058321A1
• Filed: 09/11/2003
• Published: 03/17/2005
• Est. Priority Date: 09/11/2003
• Status: Active Grant

• Current Assignee: Johnson Controls Tyco IP Holdings LLP (Johnson Controls International plc)
• Original Assignee: Sensormatic Electronics Corporation (Johnson Controls International plc)
• Inventors: Buehler, Christopher J.
• Granted Patent: US 7,286,157 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/248 involving reference images or patches
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A computerized method of image analysis that includes receiving first image data for a plurality of first video frames representing a first scene. Each first video frame is composed of a plurality of image regions, and a first object is present in an image region of at least one first video frame. Second image data is received for a plurality of second video frames representing a second scene. Each second video frame is composed of a plurality of image regions, and a second object is present in an image region of at least one second video frame. The method also includes determining a relationship between a first image region and a second image region based on a probabilistic correlation between occurrences of the first object being present in the first image region and occurrences of the second object being present in the second image region.

Claims

1. A computerized method of image analysis comprising:

a) receiving first image data for a plurality of first video frames representing a first scene, wherein (i) each first video frame comprises a plurality of image regions and (ii) a first object is present in an image region of at least one first video frame;

b) receiving second image data for a plurality of second video frames representing a second scene, wherein (i) each second video frame comprises a plurality of image regions and (ii) a second object is present in an image region of at least one second video frame; and

c) determining a relationship between first and second image regions of the plurality of image regions of the first and second video frames based on a probabilistic correlation between occurrences of the first object being present in the first image region and occurrences of the second object being present in the second image region.

2. The method of claim 1 wherein step c) further comprises c-1) determining a conditional probability that the first object is present in the first image region given that the second object is present in the second image region.

3. The method of claim 1 wherein step c) further comprises c-1) determining a joint probability that the first object is present in the first object region and that the second object is present in the second image region.

4. The method of claim 1 wherein step c) further comprises c-1) determining a lift associated with the first object being present in the first image region and the second object being present in the second image region.

5. The method of claim 1 wherein step c) further comprises c-1) determining a correlation coefficient associated with the first object being present in the first image region and the second object being present in the second image region.

6. The method of claim 1 wherein the first image data originates from a first camera having a first field-of-view, and the second image data originates from a second camera having a second field-of-view.

7. The method of claim 6 wherein the first field-of-view and the second field-of-view at least in part overlap.

8. The method of claim 6 wherein the first field-of-view and the second field-of-view do not overlap.

9. The method of claim 1 wherein the first image region is one of the image regions that comprises the first video frames and the second image region is one of the image regions that comprises the second video frames.

10. The method of claim 1 wherein the first image region and the second image region are both image regions that comprise one of the first video frames and the second video frames.

11. The method of claim 1 wherein the first object and the second object are the same object.

12. The method of claim 1 wherein the first object and the second object are different objects.

13. The method of claim 1 wherein step c) comprises determining a temporal relationship between occurrences of the presence of the first object in the first image region and occurrences of the presence of the second object in the second image region.

14. The method of claim 1 wherein step c) comprises determining a spatial relationship between the first image region and the second image region.

15. The method of claim 1 further comprising d) counting the number of occurrences of any object being present in the first and second image regions.

16. The method of claim 1 further comprising d) counting the number of occurrences of any object being concurrently present in both the first and second image regions.

17. The method of claim 1 further comprising d) counting, for each of a predetermined number of earlier time instants, the number of occurrences of both any object being currently present in the first image region and any object being present in the second image region at the earlier time instants.

18. The method of claim 1 further comprising d) providing a plurality of attributes for objects.

19. The method of claim 18 further comprising e) counting for each of the plurality of attributes, the number of occurrences of an object having such attribute being present in the first and second image regions.

20. The method of claim 18 further comprising e) counting, for each of the plurality of attributes, the number of occurrences of an object with each attribute being present in the first image region concurrently with an object having each attribute being present in second image regions.

21. The method of claim 18 further comprising e) counting, for each of a predetermined number of earlier time instants, and for each of the plurality of attributes, the number of occurrences of both any object having such attribute being currently present in the first image region and any object having each attribute being present in the second image region at the earlier time instants.

22. The method of claim 1 further comprising d) correlating the first object in the first video frames with a second object present in the second video frames based on the determined relationship.

23. The method of claim 1 further comprising:

d) determining that the first object present in one first video frame was not present in a preceding first video frame; and

e) correlating the first object with the second object present in the second video frames based on the determined relationship.

24. The method of claim 1 wherein each image region has a location in one of the plurality of first and second video frames that is substantially the same across the plurality of first and second video frames.

25. The method of claim 24 further comprising d) maintaining an object list identifying at least the locations of the first object and the second object.

26. The method of claim 25 further comprising:

e) selecting an object present in one of the first and second video frames;

f) determining whether the selected object is one of the first object, the second object, and a third object based at least in part on the object list and the determined relationship.

27. A surveillance system comprising:

a) a first memory module for receiving first image data for a plurality of first video frames representing a first scene, wherein (i) each first video frame comprises a plurality of image regions and (ii) a first object is present in an image region of at least one first video frame;

b) a second memory module for receiving second image data for a plurality of second video frames representing a second scene, wherein (i) each second video frame comprises a plurality of image regions and (ii) a second object is present in an image region of at least one second video frame; and

c) a processing module for determining a relationship between first and second image regions of the plurality of image regions of the first and second video frames based on a probabilistic correlation between occurrences of the first object being present in the first image region and occurrences of the second object being present in the second image region.

28. The surveillance system of claim 27 wherein the processing module further determines a conditional probability that the first object is present in the first image region given that the second object is present in the second image region.

29. The surveillance system of claim 27 wherein the processing module further determines a joint probability that the first object is present in the first object region and that the second object is present in the second image region.

30. The surveillance system of claim 27 wherein the processing module further determines a lift associated with the first object being present in the first image region and the second object being present in the second image region.

31. The surveillance system of claim 27 wherein the processing module further determines a correlation coefficient associated with the first object being present in the first image region and the second object being present in the second image region.

32. The surveillance system of claim 27 wherein the first image data originates from a first camera having a first field-of-view, and the second image data originates from a second camera having a second field-of-view.

33. The surveillance system of claim 32 wherein the first field-of-view and the second field-of-view at least in part overlap.

34. The surveillance system of claim 32 wherein the first field-of-view and the second field-of-view do not overlap.

35. The surveillance system of claim 27 wherein the first image region is one of the image regions that comprises the first video frames and the second image region is one of the image regions that comprises the second video frames.

36. The surveillance system of claim 27 wherein the first image region and the second image region are both image regions that comprise one of the first video frames and the second video frames.

37. The surveillance system of claim 27 wherein the first object and the second object are the same object.

38. The surveillance system of claim 27 wherein the first object and the second object are different objects.

39. The surveillance system of claim 27 wherein the processing module determines a temporal relationship between occurrences of the presence of the first object in the first image region and occurrences of the presence of the second object in the second image region.

40. The surveillance system of claim 27 wherein the processing module determines a spatial relationship between the first image region and the second image region.

41. The surveillance system of claim 27 wherein the processing module further counts the number of occurrences of any object being present in the first and second image regions.

42. The surveillance system of claim 27 wherein the processing module further counts the number of occurrences of any object being concurrently present in both the first and second image regions.

43. The surveillance system of claim 27 wherein the processing module further counts, for each of a predetermined number of earlier time instants, the number of occurrences of both any object being currently present in the first image region and any object being present in the second image region at the earlier time instants.

48. The surveillance system of claim 27 wherein the processing module further correlates the first object in the first video frames with a second object present in the second video frames based on the determined relationship.

49. The surveillance system of claim 27 wherein in the processing module further determines that the first object present in one first video frame was not present in a preceding first video frame, and correlates the first object with the second object present in the second video frames based on the determined relationship.

50. The surveillance system of claim 27 wherein each image region has a location in one of the plurality of first and second video frames that is substantially the same across the plurality of first and second video frames.

51. The surveillance system of claim 50 wherein the processing module further maintains an object list identifying at least the locations of the first object and the second object.

52. The surveillance system of claim 51 wherein the processing module further selects an object present in one of the first and second video frames and determines whether the selected object is one of the first object, the second object, and a third object based at least in part on the object list and the determined relationship.

44. The surveillance system 27 wherein the processing module further r provides a plurality of attributes for objects.

45. The surveillance system of claim 44 wherein the processing module further counts for each of the plurality of attributes, the number of occurrences of an object having such attribute being present in the first and second image regions.

46. The surveillance system of claim 44 wherein the processing module further counts, for each of the plurality of attributes, the number of occurrences of an object with each attribute being present in the first image region concurrently with an object having each attribute being present in second image regions.

47. The surveillance system of claim 44 further wherein the processing module further counts, for each of a predetermined number of earlier time instants, and for each of the plurality of attributes, the number of occurrences of both any object having such attribute being currently present in the first image region and any object having each attribute being present in the second image region at the earlier time instants.

53. A surveillance system comprising:

a) a means for receiving first and second image data for a plurality of first and second video frames representing first and second scenes, wherein (i) each first and second video frame comprises a plurality of image regions and (ii) first and second objects are present in an image region of at least one first and second video frame, respectively; and

b) a processing module for determining a relationship between first and second image regions of the plurality of image regions of the first and second video frames based on a probabilistic correlation between occurrences of the first object in the first image region and the second object in the second image region.

54. The surveillance system of claim 53 wherein the first object is not the same object as the second object.

55. The surveillance system of claim 53 wherein the first object is the same object as the second object.