Camera surveillance planning and tracking system

• US 10,318,819 B2
• Filed: 01/05/2016
• Issued: 06/11/2019
• Est. Priority Date: 01/05/2016
• Status: Active Grant

• Current Assignee: Mitre Corporation
• Original Assignee: Mitre Corporation
• Inventors: Stelzer, Emily, Chong, Ronald S., Chen, Shuo, Richkus, Jacob, Tuomey, Dave, Nene, Vilas
• Primary Examiner(s): Kelley, Christopher S
• Assistant Examiner(s): Nirjhar, Nasim N
• Application Number:
  US14/988,603
• Publication Number:
  US 20170193307A1
• Time in Patent Office: 1,253 Days
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/90 Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Disclosed herein are system, method, and computer program product embodiments for camera surveillance planning and tracking using, for example, infrared technologies. An embodiment operates by receiving a camera specification input and placement input, calculating a surveillance coverage based on the camera specification and the placement input, displaying the surveillance coverage on a map, and identifying at least one stereo pair.

Claims

1. A method, comprising:

receiving a camera specification and placement input;

calculating a surveillance coverage of a camera according to the camera specification and the placement input, wherein the surveillance coverage comprises a geographic area;

displaying the surveillance coverage of the camera on a graphical user interface map, wherein the graphical user interface map displays the geographic area of the surveillance coverage;

identifying a valid video stereo pair according to the surveillance coverage;

identifying, from images received from the valid video stereo pair, a first foreground object having a first trajectory;

identifying, from the images, a second foreground object having a second trajectory;

comparing the second trajectory to the first trajectory to determine a difference between the second trajectory and the first trajectory;

determining that the difference between the second trajectory and the first trajectory exceeds a first threshold and that the second foreground object is a new object that differs from the first foreground object;

in response to the determining, applying size and trajectory filtering to the second foreground image to determine that a size of the second foreground object and the second trajectory exceed a second threshold to selectively designate the second foreground object as an object of interest; and

in response to the applying, updating the graphical user interface map to display the second trajectory on the geographic area of the surveillance coverage.

2. The method of claim 1, further comprising:

calibrating the camera, wherein calibrating the camera comprises generating a plurality of camera parameters.

3. The method of claim 2, wherein the camera specification comprises the plurality of camera parameters.

4. The method of claim 1, further comprising:

receiving a simulation object input, wherein the simulation object input comprises a point in the geographic area.

5. The method of claim 4, the identifying comprising:

highlighting an overlap of the simulation object input and the surveillance coverage on the graphical user interface map.

6. The method of claim 1, the identifying comprising:

eliminating a pair of cameras from a plurality of cameras based upon the surveillance coverage.

7. The method of claim 1, further comprising:

receiving a plurality of simulation object inputs, wherein the simulation object inputs comprise points on the graphical user interface map.

21. The method of claim 1, further comprising:

determining a first 2D trajectory of the first foreground object based on images captured from a first camera of the valid video stereo pair;

determining a second 2D trajectory of the first foreground object based on images captured from a second camera of the valid video stereo pair; and

generating a 3D triangulated trajectory for the first foreground object based on the first 2D trajectory and the second 2D trajectory.

24. The method of claim 1, wherein updating the graphical user interface map further comprises:

displaying an altitude measurement with the second trajectory.

8. A system, comprising:

a memory; and

at least one processor coupled to the memory and configured to;

receive an infrared camera specification and placement input;

calculate a surveillance coverage of an infrared camera according to the infrared camera specification and the placement input, wherein the surveillance coverage comprises a geographic area;

display the surveillance coverage of the infrared camera on a graphical user interface map, wherein the graphical user interface map displays the geographic area of the surveillance coverage;

identify a valid video stereo pair according to the surveillance coverage;

identify, from images received from the valid video stereo pair, a first foreground object having a first trajectory;

identify, from the images, a second foreground object having a second trajectory;

compare the second trajectory to the first trajectory to determine a difference between the second trajectory and the first trajectory;

determine that the difference between the second trajectory and the first trajectory exceeds a first threshold and that the second foreground object is a new object that differs from the first foreground object;

in response to the determining, apply size and trajectory filtering to the second foreground image to determine that a size of the second foreground object and the second trajectory exceed a second threshold to selectively designate the second foreground object as an object of interest; and

in response to the applying, update the graphical user interface map to display the second trajectory on the geographic area of the surveillance coverage.

9. The system of claim 8, the at least one processor further configured to:

calibrate the infrared camera, wherein calibrating the camera comprises generating a plurality of camera parameters.

10. The system of claim 9, wherein the infrared camera specification comprises the plurality of camera parameters.

11. The system of claim 8, the at least one processor further configured to:

receive a simulation object input, wherein the simulation object input comprises a point in the geographic area.

12. The system of claim 11, wherein to identify, the at least one processor is further configured to:

highlight an overlap of the simulation object input and the surveillance coverage on the graphical user interface map.

13. The system of claim 8, wherein to identify, the at least one processor is further configured to:

eliminate a pair of cameras from a plurality of cameras based upon the surveillance coverage.

22. The system of claim 8, wherein the at least one processor is further configured to:

determine a first 2D trajectory of the first foreground object based on images captured from a first camera of the valid video stereo pair;

determine a second 2D trajectory of the first foreground object based on images captured from a second camera of the valid video stereo pair; and

generate a 3D triangulated trajectory for the first foreground object based on the first 2D trajectory and the second 2D trajectory.

14. A tangible, non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising:

receiving a camera specification and placement input;

calculating a surveillance coverage of a camera according to the camera specification and the placement input, wherein the surveillance coverage comprises a geographic area;

displaying the surveillance coverage of the camera on a graphical user interface map, wherein the graphical user interface map displays the geographic area of the surveillance coverage;

identifying a valid video stereo pair according to the surveillance coverage;

identifying, from images received from the valid video stereo pair, a first foreground object having a first trajectory;

identifying, from the images, a second foreground object having a second trajectory;

comparing the second trajectory to the first trajectory to determine a difference between the second trajectory and the first trajectory;

determining that the difference between the second trajectory and the first trajectory exceeds a first threshold and that the second foreground object is a new object that differs from the first foreground object;

in response to the determining, applying size and trajectory filtering to the second foreground image to determine that a size of the second foreground object and the second trajectory exceed a second threshold to selectively designate the second foreground object as an object of interest; and

in response to the applying, updating the graphical user interface map to display the second trajectory on the geographic area of the surveillance coverage.

15. The non-transitory computer-readable device of claim 14, the operations further comprising:

calibrating the camera, wherein calibrating the camera comprises generating a plurality of camera parameters.

16. The non-transitory computer-readable device of claim 15, wherein the camera specification comprises the plurality of camera parameters.

17. The non-transitory computer-readable device of claim 14, the operations further comprising:

receiving a simulation object input, wherein the simulation object input comprises a point in the geographic area.

18. The non-transitory computer-readable device of claim 17, the identifying comprising:

highlighting an overlap of the simulation object input and the surveillance coverage on the graphical user interface map.

19. The non-transitory computer-readable device of claim 14, the identifying comprising:

eliminating a pair of cameras from a plurality of cameras based upon the surveillance coverage.

20. The non-transitory computer-readable device of claim 14, the operations further comprising:

receiving a plurality of simulation object inputs, wherein the simulation object inputs comprise points on the graphical user interface map.

23. The non-transitory computer-readable device of claim 14, wherein the operations further comprise:

determining a first 2D trajectory of an object based on images captured from a first camera of the valid video stereo pair;

determining a second 2D trajectory of the object based on images captured from a second camera of the valid video stereo pair; and

generating a 3D triangulated trajectory for the object based on the first 2D trajectory and the second 2D trajectory.