Automatic counter-surveillance detection camera and software

• US 9,172,913 B1
• Filed: 09/19/2011
• Issued: 10/27/2015
• Est. Priority Date: 09/24/2010
• Status: Active Grant

• Current Assignee: JETprotect Corporation
• Original Assignee: JETprotect Corporation
• Inventors: Johnston, Gregory Edward, Hubble, Larry
• Primary Examiner(s): Czekaj, Dace
• Assistant Examiner(s): Rahman, Mohammad J
• Application Number:
  US13/236,201
• Time in Patent Office: 1,499 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06V 10/145 Illumination specially adapted for pattern recognition, e.g. using gratings
• CPC Class Codes: H04N 13/117 the virtual viewpoint locations being selected by the viewers or determined by viewer tracking
• CPC Class Codes: H04N 13/279 the virtual viewpoint locations being selected by the viewers or determined by tracking
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 7/155 involving storage of or access to video conference sessions tracking arrangements for later retrieval of a computer conference content or participants activities H04L12/1831
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

A counter-surveillance detection system includes a camera with an integrated illumination source and a processor for performing analytic software algorithms. The counter-surveillance detection system and method is configured to detect foreign objects within a defined field of view (FOV) using an analytic software method operating on imagery data from the camera that is synchronized with the illuminator source. In some applications, the system is used to detect a human performing remote observation of snooping.

Claims

1. An apparatus comprising:

a. an illumination source configured to illuminate a field of view;

b. a camera configured to capture a series of image frames corresponding to the field of view, wherein the camera and the illumination source are substantially co-aligned, further wherein each image frame includes auto-reflected light from within the field of view resulting from illumination by the illumination source, wherein the auto-reflected light is light from the illumination source that is reflected from an object in the field of view at a same angle as impinging light originating from the illumination source; and

c. a means for processing the captured series of image frames to determine a set of extracted objects, the set of extracted objects comprises a set of glint objects corresponding to the auto-reflected light and a set of kinetic objects, and to process each extracted object to determine if the extracted object is a foreign object in the field of view compared to a known background determined from a series of previous image frames, wherein determining the set of glint objects is not a function of determining the set of kinetic objects, and determining the set of kinetic objects is not a function of determining the set of glint objects.

2. The apparatus of claim 1 further comprising means for adapting an exposure of the camera according to changing time of day and other lighting conditions, thereby normalizing pixels from image frame to image frame.

3. The apparatus of claim 1 further comprising means for calibrating one or more known object locations in the known background, each of the one or more known object locations correspond to a known glint object when illuminated by the illumination source, and means for determining an ON/OFF state of the illumination source by determining if the known glint object is present at each of the one or more known object locations when the illumination source is ON.

4. The apparatus of claim 1 wherein the foreign object is an observer and the means for processing is configured to detect the observer in the field of view looking in the direction of the camera.

5. The apparatus of claim 1 wherein the foreign object is an observer looking through an optical device and the means for processing is configured to detect the observer in the field of view looking through the optical device in the direction of the camera.

6. An apparatus comprising:

a. an illumination source configured to illuminate a field of view;

b. a camera configured to capture a series of image frames corresponding to the field of view, wherein each image frame includes auto-reflected light from within the field of view resulting from illumination by the illumination source;

c. a memory configured to store the series of image frames; and

d. a processor coupled to the memory, the processor comprises program instructions configured to;

i. determine a known background, wherein the known background comprises a known value for each pixel in the field of view;

ii. determine a difference between a current image frame and the known background thereby forming a difference frame;

iii. determine a set of extracted objects from the difference frame, wherein the set of extracted objects comprises a set of glint objects and a set of kinetic objects, further wherein determining the set of glint objects is performed independently of and is not a function of determining the set of kinetic objects; and

iv. process each extracted object to determine if the extracted object is a foreign object within the field of view.

7. The apparatus of claim 6 wherein determining the set of glint objects comprises determining pixels from the difference frame that exceed a light intensity threshold, dilating an area surrounding each pixel exceeding the light intensity threshold to include other pixels in the area exceeding the light intensity threshold to form one or more groupings of pixels exceeding the light intensity threshold, and eroding each grouping to remove outlier pixels, whereby each eroded grouping forms a glint object.

8. The apparatus of claim 6 wherein processing each extracted object comprises processing each glint object by pattern matching each glint object to a known pattern database and assigning a first solution metric to the glint object based on a closeness of the pattern matching, wherein if the first solution metric exceeds a first threshold value then the glint object is determined to be a foreign object.

9. The apparatus of claim 8 wherein processing each glint object further comprises comparing a location of the glint object to a location of each of the set of kinetic objects and if the location of the glint object matches the location of one of the kinetic object then the glint object is determined to be a foreign object.

10. The apparatus of claim 8 wherein processing each glint object further comprises comparing a location of the glint object to a location of each of the set of kinetic objects and assigning a second solution metric to the glint object according to the comparison.

11. The apparatus of claim 10 wherein processing each glint object further comprises comparing the glint object to a persistent object list and assigning a third solution metric to the glint object according to the comparison, wherein the persistent object list comprises a list of extracted objects identified in previous image frames.

12. The apparatus of claim 11 wherein if the third solution metric exceeds a third solution metric threshold value then the glint object is determined to be a foreign object.

13. The apparatus of claim 11 wherein the first solution metric, the second solution metric, and the third solution metric are individually weighted and summed together to form a cumulative solution metric, wherein if the cumulative solution metric exceeds a cumulative solution metric threshold value then the glint object is determined to be a foreign object.

14. The apparatus of claim 6 wherein the set of kinetic objects comprise a set of kinetic ON objects corresponding to kinetic objects determined from the image frame captured when the illumination source is ON or a set of kinetic OFF objects corresponding to kinetic objects determined from the image frame when the illumination source is OFF.

15. The apparatus of claim 6 wherein the program instructions are further configured to trigger an alarm in response to determining that the extracted object is a foreign object.

16. The apparatus of claim 6 wherein the illumination source and the camera are co-aligned.

17. The apparatus of claim 6 wherein the illumination source comprises a laser.

18. The apparatus of claim 6 wherein the camera includes a filter to selectively capture predefined light wavelengths.

19. The apparatus of claim 18 wherein the illumination source is configured to emit light having the predefined wavelengths.

20. The apparatus of claim 6 wherein the known background comprises a first known background corresponding to when the illumination source is ON and a second known background corresponding to when the illumination source is OFF.

21. The apparatus of claim 20 wherein each image frame is associated with the illumination source ON or OFF, and the set of extracted objects are determined by determining the difference between the current frame corresponding to the illumination source ON and the first known background, or by determining the difference between the current frame corresponding to the illumination source OFF and the second known background.

22. The apparatus of claim 6 further comprising means for adapting an exposure of the camera according to changing time of day and other lighting conditions, thereby normalizing pixels from image frame to image frame.

23. The apparatus of claim 6 further comprising means for calibrating one or more known object locations in the known background, each of the one or more known object locations correspond to a known glint object when illuminated by the illumination source, and means for determining an ON/OFF state of the illumination source by determining if the known glint object is present at each of the one or more known object locations when the illumination source is ON.

24. The apparatus of claim 6 wherein the program instructions are further configured to detect an observer in the field of view looking in the direction of the camera.

25. The apparatus of claim 6 wherein the program instructions are further configured to detect an observer in the field of view looking through an optical device in the direction of the camera.

26. The apparatus of claim 6 wherein the set of glint objects are determined using the auto-reflected light.

27. The apparatus of claim 6 wherein the illumination source is configured to repeatedly turn ON and OFF to illuminate or not illuminate the field of view, and the series of image frames captured by the camera includes first image frames corresponding to image frames captured when the illumination source is ON and second image frames corresponding to image frames captured when the illumination source is OFF, further wherein determining a known background comprises determining a first known background frame and a second known background frame, wherein the first known background frame comprises a known value for each pixel in the filed of view when the illumination source is ON and the second known background frame comprises a known value for each pixel in the field of view when the illumination source is OFF, further wherein forming the difference frame comprises forming a first difference frame and a second difference frame, wherein the first difference frame is a difference between a current image frame captured when the illumination source is ON and the first known background frame, and the second difference frame is a difference between a current image frame captured when the illumination source is OFF and the second known background frame, further wherein determining the set of extracted objects from the difference frame comprises determining a set of extracted objects from the first difference frame and determining a set of extracted objects from the second difference frame, wherein the set of extracted objects from the first difference frame comprises a set of glint objects and a set of kinetic objects from the first difference frame, and the set of extracted objects from the second difference frame comprises a set of kinetic objects from the second difference frame, further wherein each glint object from the set of glint objects determined from the first difference frame is compared to a collection of known optical targets and a degree of matching is quantified as a first solution metric, if the first solution metric is greater than a first threshold value then the glint object is determined to be a foreign object in the field of view, further wherein the glint object is compared for spatial overlap with each kinetic object from the set of kinetic objects determined from the first difference frame and a degree of overlap is quantified as a second solution metric, if the first solution metric plus the second solution metric is greater than a second threshold value then the glint object is determined to be a foreign object in the field of view, further wherein each kinetic object from the set of kinetic objects determined from the first difference frame is compared is compared for spatial overlap with each kinetic object from the set of kinetic objects determined from the second difference frame of an immediately preceding image frame and a degree of overlap is quantified as a third solution metric, if the first solution metric plus the second solution metric plus the third solution metric is greater than a third threshold value then the kinetic object from the first difference frame is determined to be a foreign object in the field of view.