Tracking people and objects using multiple live and recorded surveillance camera video feeds

• US 8,350,908 B2
• Filed: 05/22/2008
• Issued: 01/08/2013
• Est. Priority Date: 05/22/2007
• Status: Active Grant

• Current Assignee: Ascvid IP Holdings LLC
• Original Assignee: VidSys Incorporated
• Inventors: Morris, Stephen Jeffrey, Clarkson, Richard Casey
• Primary Examiner(s): Feild, Lynn
• Assistant Examiner(s): ELFERVIG, TAYLOR A
• Application Number:
  US12/125,122
• Publication Number:
  US 20080292140A1
• Time in Patent Office: 1,692 Days
• Field of Search: 348/143
• US Class Current: 348/143
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/173 face re-identification, e.g. recognising unknown faces across different face tracks
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Tracking a target across a region is disclosed. A graphical user interface is provided that displays, in a first region, video from a field of view of a main video device, and, in a plurality of second regions, video from a field of view of each of a plurality of perimeter video devices (PVDs). The field of view of each PVD is proximate to the main video device'"'"'s field of view. A selection of one of the plurality of PVDs is received. In response, video from a field of view of the selected PVD is displayed in the first region, and a plurality of candidate PVDs is identified. Each candidate PVD has a field of view proximate to the field of view of the selected PVD. The plurality of second regions is then repopulated with video from a field of view of each of the plurality of identified candidate PVDs.

Claims

1. A method of tracking a target across a region using a plurality of video devices, comprising:

providing a graphical user interface, wherein the graphical user interface displays, in a first region, video from a current field of view of a main video device, and, in a plurality of second regions, video from a field of view corresponding to each of a plurality of perimeter video devices, the field of view of each perimeter video device being proximate to the field of view of the main video device, the field of view defining a position based on a camera pointing direction and on static or dynamic movement of the perimeter video devices independent of the location of the perimeter video devices;

receiving a selection of one of the plurality of perimeter video devices; and

in response;

displaying video from a field of view of the selected perimeter video device in the first region;

calculating a distance and direction from the field of view of the selected perimeter video device to proximate fields of view of other proximate video devices;

identifying a plurality of candidate perimeter video devices based on the calculated distance and direction, the calculated distance and direction based on the field of view of the corresponding video device;

using the calculated distance and direction to discard one or more candidate perimeter video devices from the plurality of candidate perimeter video devices; and

repopulating the plurality of second regions with video from a field of view corresponding to each of the plurality of identified candidate perimeter video devices.

2. The method of claim 1 comprising:

recording video from a subset of the plurality of video devices; and

playing a subset of the recorded video backwards in the graphical user interface to track a target back through a period of time by repeating the steps of receiving, displaying, identifying, and repopulating, the subset of the recorded video used throughout the repeated steps.

3. The method of claim 2 comprising:

ceasing backwards playback of the subset of the recorded video at a point; and

playing a subset of the recorded video forward from the point to track a second target, the second target tracked by repeating the steps of receiving, displaying, identifying, and repopulating, the subset of the recorded video used throughout the repeated steps.

4. The method of claim 1 comprising:

prior to displaying video from a field of view of a video device, identifying a plurality of fields of view associated with that video device, each field of view associated with a position of the video device;

selecting a field of view of the video device that is optimally proximate to the field of view of the main video device;

directing the video device to move to the position associated with the selected field of view;

recording an initial position of the video device; and

upon video from the video device no longer being displayed in the graphical user interface, directing the video device to return to the initial position.

5. The method of claim 1 wherein identifying comprises:

identifying a plurality of candidate perimeter video devices, wherein at least one candidate perimeter video device has an include list, the include list comprising one or more video devices associated with the candidate perimeter video device, any video device on the include list automatically added to the plurality of candidate perimeter video devices.

6. The method of claim 1 wherein identifying comprises:

identifying a plurality of candidate perimeter video devices, wherein at least one candidate perimeter video device has an exclude list, the exclude list comprising one or more video devices associated with the candidate perimeter video device, any video device on the exclude list automatically disqualified from being part of the plurality of candidate perimeter video devices.

7. The method of claim 1 wherein receiving comprises:

selecting one of the plurality of perimeter video devices automatically through use of analytics, the analytics including relations of distance, probability, passability or priority to the field of view of the selected perimeter video device.

8. The method of claim 1 wherein identifying candidate perimeter video devices includes probability, further comprising:

computing, for each candidate perimeter video device, the most likely fields of view for which a target will move into to determine a probability for that candidate perimeter video device, and

selecting the perimeter video devices based on the computed probability.

9. The method of claim 1 wherein identifying candidate perimeter video devices includes distance, further comprising

computing, for each candidate perimeter video device, a distance from the selected perimeter video device, and

selecting the perimeter video devices based on the computed distance.

10. The method of claim 1 further comprising defining an obstruction mapping for each of the video devices, the obstruction mapping modeling a region in which a target may be tracked by using a coordinate system for assigning a location to each video device, the obstruction mapping defining a passability relation indicating physical deterrents to passage that affect likelihood of transition to a field of view of each video device.

11. The method of claim 10 wherein the coordinate space includes z coordinate indicative of a vertical orientation in addition to x, y horizontal plane coordinates, the z component indicative of multiple floors or elevators.

12. The method of claim 10 wherein identifying candidate perimeter video devices includes priorities, the priorities indicative of video devices having an enhanced need for scrutiny which results the video devices to be selected when they otherwise might not, further comprising assigning stronger priorities to video devices having a field of view more prone to obscurement of tracked targets.

13. The method of claim 1, further comprising:

modeling a region in which a target may be tracked by using a coordinate space; and

relating the fields of view of the perimeter video devices to a coordinate space.

14. The method of claim 13 further comprising:

assigning units of the coordinate space to the modeled region; and

defining the fields of view to the units of the coordinate space relative to the modeled region.

15. The method of claim 14 wherein the units of the coordinate space are in terms of a building fixture or latitude/longitude.

16. The method of claim 15 wherein the coordinate space defines an origin based on a structural landmark in a monitored space, and the units are based on a fixture distance in the monitored space.

17. The method of claim 1 further comprising:

computing a distance and direction of each candidate perimeter video device in a coordinate space; and

using the computed distance and direction to identify candidate perimeter video devices for repopulating the plurality of second regions.

18. The method of claim 1 wherein the identified candidate perimeter video devices define the plurality of perimeter video devices, further comprising:

iteratively receiving the selection of one of the plurality of perimeter video devices for display in the first region; and

repeating identifying the candidate perimeter video devices based on an expected path of a tracked subject.

19. The method of claim 18 further comprising identifying the candidate perimeter video devices further based on a line approximating a line of travel of a subject, the line based on previous fields of view traversed by the subject.

20. The method of claim 1 wherein the plurality of candidate perimeter video devices are further identified based on a likely next field of view of the subject, the next likely field of view based on a line of travel of the subject and physical barriers in the field of view of the main and candidate perimeter video devices.

21. The method of claim 1 further comprising:

identifying a spot indicative of a center of a field of view corresponding to the selection;

determining a coordinate position of the identified spot;

calculating a distance between adjacent fields of view based on the determined coordinate positions; and

selecting candidate perimeter video devices based on an ordering of the calculated distance.

22. The method of claim 21 wherein identified spot is based on a field of view obtained from multiple positions of a video device including a PTZ (pan/tilt/zoom) video camera.

23. The method of claim 1 wherein the position includes a range of fields of view based on dynamic movement of the perimeter video device.

24. A computer program product, stored on a non transitory computer readable medium having instructions that, when executed by a processor, cause the processor to perform a method for tracking a target across a region using a plurality of video devices, comprising:

a processor responsive to the instructions;

computer program code for providing a graphical user interface, wherein the graphical user interface displays, in a first region, video from a field of view corresponding to a main video device, and, in a plurality of second regions, video from a field of view of each of a plurality of perimeter video devices, the field of view of each perimeter video device being proximate to the field of view of the main video device;

computer program code for receiving a selection of one of the plurality of perimeter video devices;

and in response;

computer program code for displaying video from a field of view of the selected perimeter video device in the first region, the field of view defining a position based on a camera pointing direction and on movement of the perimeter video device independent of the location of the perimeter video device;

computer program code for calculating a distance and direction from the field of view of the selected perimeter video device to proximate fields of view of other proximate video devices;

computer program code for identifying a plurality of candidate perimeter video devices based on the calculated distance and direction, the calculated distance and direction based on the field of view of the corresponding perimeter video device;

computer program code for using the calculated distance and direction to discard one or more candidate perimeter video devices from the plurality of candidate perimeter video devices; and

computer program code for repopulating the plurality of second regions with video from a field of view corresponding to each of the plurality of identified candidate perimeter video devices.

25. The computer program product of claim 24 comprising:

computer program code for recording video from a subset of the plurality of video devices; and

computer program code for playing a subset of the recorded video backwards in the graphical user interface to track a target back through a period of time by repeating the steps of receiving, displaying, identifying, and repopulating, the subset of the recorded video used throughout the repeated steps.

26. The computer program product of claim 25 comprising:

computer program code for ceasing backwards playback of the subset of the recorded video at a point; and

computer program code for playing a subset of the recorded video forward from the point to track a second target, the second target tracked by repeating the steps of receiving, displaying, identifying, and repopulating, the subset of the recorded video used throughout the repeated steps.

27. A computer system comprising:

a memory;

a processor;

a display;

a network interface; and

an interconnection mechanism coupling the memory, the processor, the display, and the network interface, allowing communication there between;

wherein the memory of the computer system is encoded with a virtual tracking application, that when executed in the processor, provides a virtual tracking process that tracks a target across a region using a plurality of video devices, by causing the computer system to perform operations of;

providing a graphical user interface, wherein the graphical user interface displays, in a first region, video from a field of view of a main video device, and, in a plurality of second regions, video from a field of view corresponding to each of a plurality of perimeter video devices, the field of view of each perimeter video device being proximate to the field of view of the main video device, the field of view defining a position based on a camera pointing direction and on static or dynamic movement of the perimeter video device independent of the location of the perimeter video device;

receiving a selection of one of the plurality of perimeter video devices; and

in response;

displaying video from a field of view of the selected perimeter video device in the first region;

calculating a distance from the field of view of the selected perimeter video device to proximate fields of view of other proximate video devices;

identifying a plurality of candidate perimeter video devices based on the calculated distance and direction, the calculated distance and direction based on the field of view of the corresponding video device;

using the calculated distance and direction to discard one or more candidate perimeter video devices from the plurality of candidate perimeter video devices; and

repopulating the plurality of second regions with video from a field of view corresponding to each of the plurality of identified candidate perimeter video devices.

28. The computer system of claim 27 comprising:

recording video from a subset of the plurality of video devices; and

playing a subset of the recorded video backwards in the graphical user interface to track a target back through a period of time by repeating the steps of receiving, displaying, identifying, and repopulating, the subset of the recorded video used throughout the repeated steps.

29. The computer system of claim 28 comprising:

ceasing backwards playback of the subset of the recorded video at a point; and

playing a subset of the recorded video forward from the point to track a second target, the second target tracked by repeating the steps of receiving, displaying, identifying, and repopulating, the subset of the recorded video used throughout the repeated steps.

30. The computer system of claim 27 comprising:

prior to displaying video from a field of view of a video device, identifying a plurality of fields of view associated with that video device, each field of view associated with a position of the video device;

selecting a field of view of the video device that is optimally proximate to the field of view of the main video device; and

directing the video device to move to the position associated with the selected field of view.

31. The computer system of claim 27 wherein identifying comprises:

identifying a plurality of candidate perimeter video devices, each candidate perimeter video device having a location in a coordinate space indicating its distance from the selected perimeter video device; and

using the locations to discard one or more candidate perimeter video devices from the plurality of candidate perimeter video devices.

32. In a surveillance area for monitoring subjects of interest by selective observation from a plurality of video cameras, a method of rendering video streams received from the cameras comprising:

identifying a subject of interest in the surveillance area, each camera covering a respective field of view in the surveillance area, the field of view defining a position based on a camera pointing direction and on static or dynamic movement of the camera independent of the location of the camera;

rendering the subject on a display by identifying a camera as a main video camera having a main field of view occupied by the subject of interest;

calculating a distance and direction from the field of view of the selected camera to proximate fields of view of other cameras;

computing, based on the calculated distance and direction, the calculated distance and direction based on the field of view of the corresponding camera, a set of candidate perimeter cameras covering fields of view to which the subject of interest is most likely to proceed, each candidate perimeter camera having a location in a coordinate space indicating its distance from the main video camera;

the set of candidate perimeter cameras computed by;

applying a series of filters to eliminate, from the set of candidate perimeter cameras, fields of view unlikely to provide a trackable view of the subject, the series of filters including elimination of cameras either too distant from or too close to the main field of view;

evaluating the candidate perimeter cameras by accumulating weighting factors indicative of a strategic tracking value of the fields of view, accumulating including considering a high passability to the field of view, a positioning priority of the field of view and a history of previous subject visits to the field of view as positive weighting factors and considering physical passability obstructions and field of view obscurement as negative weighting factors;

ranking the candidate perimeter cameras according to the accumulated weighting factors;

using the locations to discard one or more candidate perimeter cameras from the plurality of candidate perimeter cameras; and

rendering the video streams from the highest ranked candidate perimeter cameras on the video display adjacent to the video stream received from the main camera;

identifying one of the candidate perimeter cameras as a successive main camera depending on which field of view the subject of interest is determined to advance to;

designating the successive main camera as the main camera; and

repeating rendering, computing a set of candidate perimeter cameras, and identifying a successive main camera as the subject of interest advances among the fields of view in the surveillance area.