Intelligent camera selection and object tracking

• US 20100002082A1
• Filed: 03/24/2006
• Published: 01/07/2010
• Est. Priority Date: 03/25/2005
• Status: Active Grant

• Current Assignee: Johnson Controls Tyco IP Holdings LLP (Johnson Controls International plc)
• Original Assignee: Sensormatic Electronics LLC (Johnson Controls International plc)
• Inventors: Cannon, Howard I., Buehler, Christopher J.
• Granted Patent: US 8,174,572 B2
• Time in Patent Office: Days
• US Class Current: 348/159
• CPC Class Codes: G08B 13/19645 Multiple cameras, each having view on one of a plurality of scenes, e.g. multiple cameras for multi-room surveillance or for tracking an object by view hand-over
• CPC Class Codes: G08B 13/19693 using multiple video sources viewed on a single or compound screen

Abstract

Methods and systems for creating video from multiple sources utilize intelligence to designate the most relevant sources, facilitating their adjacent display and/or catenation of their video streams.

Claims

1. A video surveillance system comprising:

a user interface comprising;

a primary camera pane for displaying a primary video data feed captured by a primary video surveillance camera;

two or more camera panes in proximity to the primary camera pane, each proximate camera pane for displaying secondary video data feeds captured by one of a set of secondary video surveillance cameras; and

a camera selection module for determining the set of secondary video surveillance cameras in response to the primary video data displayed in the primary camera pane.

2. The system of claim 1 wherein the set of secondary video surveillance cameras is based on spatial relationships between the primary video surveillance camera and a plurality of video surveillance cameras.

3. The system of claim 1 wherein the set of secondary video surveillance cameras is inferred based on statistical relationships between the primary video surveillance camera and a plurality of video surveillance cameras.

4. The system of claim 1 wherein the video data displayed in the primary camera pane is divided into two or more sub-regions.

5. The system of claim 4 wherein the set of secondary video surveillance cameras is based on a selection of one of the two or more sub-regions.

6. The system of claim 4 further comprising an input device for facilitating selection of a sub-region of the video data displayed in the primary camera pane.

7. The system of claim 1 further comprising an input device for facilitating the selection of an object of interest within the video data shown in the primary camera pane.

8. The system of claim 7 wherein the set of secondary video surveillance cameras is based on the selected object of interest within the video data shown in the primary camera pane.

9. The system of claim 7 wherein the set of secondary video surveillance cameras is based on motion of the selected object of interest within the video data shown in the primary camera pane.

10. The system of claim 9 wherein the set of secondary video surveillance cameras is based at least in part on a likelihood-of-transition metric.

11. The system of claim 7 wherein the set of secondary video surveillance cameras is based on an image quality of the selected object of interest within the video data shown in the primary camera pane.

12. The system of claim 1 wherein the camera selection module further determines the placement of the two or more proximate camera panes with respect to each other.

13. The system of claim 1 further comprising an input device for selecting one of the secondary video data feeds and thereby causing the camera selection module to designate the selected secondary video data feed as the primary video data feed and determining a second set of secondary video data feeds to be displayed in the proximate camera panes.

15. The user interface of claim 13 where the number of available secondary video data feeds is greater than the number of adjacent video panes.

16. The user interface of claim 13 wherein an assignment of video data feeds to adjacent video panes is based on a ranking of the video data feeds.

14. A user interface for presenting video surveillance data feeds comprising:

a primary video pane for presenting a primary video data feed; and

a plurality of proximate video panes, each of the proximate video panes for presenting a video data feed from one of a set of available secondary video data feeds, the presented secondary video data feeds being determined by the primary video data feed.

17. A method of selecting video data feeds for display, comprising:

presenting a primary video data feed in a primary video data pane;

receiving an indication of an object in the primary video pane;

presenting a secondary video data feed in a secondary video data pane in response to the indication;

detecting movement of the indicated object in the secondary video data feed and, based thereon, replacing the primary data feed with the secondary video data feed in the primary video data pane; and

selecting a new secondary video data feed for display in the secondary video data pane.

18. The method of claim 17 wherein the new secondary video data feed is determined based at least in part on a likelihood-of-transition metric.

19. The method of claim 18 wherein the likelihood-of-transition metric is determined according to steps comprising:

defining a set of candidate video data feeds;

assigning, to each candidate video data feed, an adjacency probability representing a likelihood that an object tracked in the primary video data pane will transition into the candidate video data feed.

20. The method of claim 19 wherein the adjacency probabilities vary according to predefined rules.

21. The method of claim 19 wherein the candidate video data feeds represent a subset of available data feeds, the candidate video data feeds being defined according to predefined rules.

22. The method of claim 19 wherein the adjacency probabilities are stored in a multi-dimensional matrix.

23. The method of claim 22 wherein the multi-dimensional matrix comprises a dimension based on the number of candidate video data feeds.

24. The method of claim 22 wherein the multi-dimensional matrix comprises a time-based dimension.

25. The method of claim 22 further comprising segmenting the multi-dimensional matrix into sub-matrices based, at least in part, on the adjacency probabilities.

26. The method of claim 19 wherein the adjacency probabilities are based at least in part on historical data.

27. A method of compiling a surveillance video comprising:

creating a surveillance video using a primary video data feed as a source video data feed;

receiving an indication to change the source video for the surveillance video from the primary video data feed to a secondary video data feed; and

concatenating the surveillance video with video data from the secondary video data feed.

28. The method of claim 27 wherein an observer of the primary video data feed indicates the change from the primary video data to the secondary video data feed.

29. The method of claim 27 wherein the indication to change the source video is generated automatically based on movement within the primary video data feed.

30. The method of claim 27 further comprising augmenting the surveillance video with audio.

31. The method of claim 30 wherein the audio is recorded observations of an observer of the primary video data feed.

32. The method of claim 30 wherein the audio is captured by a camera supplying the primary video data feed.

33. The method of claim 27 further comprising augmenting the surveillance video with one or more of text, graphics, and audio.

34. An article of manufacture having computer-readable program portions embodied thereon for compiling a surveillance video, the article comprising computer-readable instructions for:

creating a surveillance video using a primary video data feed as a source video data feed;

receiving an indication to change the source video for the surveillance video from the primary video data feed to a secondary video data feed; and

concatenating the surveillance video with video data from the secondary video data feed.

35. A data structure for describing relationships among field-of-view of cameras in a video surveillance system, the data structure comprising an N by M matrix, N representing a first set of cameras having a field-of-view in which an observed object is located at a current time, M representing a second set of cameras having a field-of-view to which the observed object is likely to appear at a subsequent time, and entries in the matrix representing transitional probabilities between the first and second set of cameras.

36. The data structure of claim 35 wherein N and M are equal.

37. The data structure of claim 35 wherein the transitional probabilities comprise a likelihood that the observed object transitions from a camera in the first set to a camera in the second set.

38. The data structure of claim 35 wherein the transitional probabilities comprise a time-based parameter.

39. A module for selecting among cameras based on motion of an observed object in a field-of-view of a reference camera, the module comprising:

a database for specifying a prediction set of cameras having a field-of-view in which the object is likely to appear at a subsequent time, and transitional probabilities between the reference camera and the set of cameras; and

a selection module for selecting the set of cameras based on the database entries.

40. The module of claim 39, wherein the database is organized as an N by M matrix, N representing a reference set of cameras having a field-of-view in which an observed object is located at a current time, M representing the prediction, and entries in the matrix representing the transitional probabilities.