Tailgating and reverse entry detection, alarm, recording and prevention using machine vision

• US 7,382,895 B2
• Filed: 04/08/2003
• Issued: 06/03/2008
• Est. Priority Date: 04/08/2002
• Status: Active Grant

• Current Assignee: Newton Security Incorporated
• Original Assignee: Newton Security Incorporated
• Inventors: Witty, Carl R., Bramblet, John W., LaForge, Seth Michael
• Primary Examiner(s): Vo; Tung
• Assistant Examiner(s): SENFI, BEHROOZ M
• Application Number:
  US10/410,884
• Publication Number:
  US 20040017929A1
• Time in Patent Office: 1,883 Days
• Field of Search: 348/385.1, 348/159, 348/169, 348/387.1, 348/43, 348/150, 348/152, 348/153, 348/154, 348/155, 348/167, 348/44, 348/135, 348/143, 348/144, 348/148, 348/149, 382/103, 382/100, 382/291, 382/104, 382/191, 382/286, 382/318, 382/154, 340/541, 340/545.1
• US Class Current: 382/103
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G07C 9/00 Individual registration on entry or exit
• CPC Class Codes: G07C 9/15 with arrangements to prevent the passage of more than one individual at a time

Abstract

Unauthorized entry into controlled access areas using tailgating or reverse entry methods is detected using machine vision methods. Camera images of the controlled area are processed to identify and track objects in the controlled area. In a preferred embodiment, this processing includes 3D surface analysis to distinguish and classify objects in the field of view. Feature extraction, color analysis, and pattern recognition may also be used for identification and tracking of objects. Integration with security monitoring and control systems provides notification when a tailgating or reverse entry event has occurred. More reliable operation in practical circumstances is thus obtained, such as when multiple people are using an entrance or exit under variable light and shadow conditions. Electronic access control systems may further be combined with the machine vision methods of the invention to more effectively prevent tailgating or reverse entry.

Claims

1. A method for detecting a violation of security of a controlled access area, comprising:

(a) acquiring one or more stereoscopic images of an area of observation, wherein objects in the area of observation are characterized by (1) an object type and (2) an authorization status indicating whether the object is authorized with respect to the controlled access area;

(b) analyzing the one or more images using a machine vision processing system to identify a first object in the area of observation and to classify the first object in a first object type among a plurality of object types that are pre-defined in the machine vision processing system, said first object having supplied an authorization with respect to the controlled access area, and further using the machine vision processing system to identify a second object in the area of observation and to classify the second object in a second object type among the plurality of pre-defined object types, wherein the authorization status of each of the objects is determined separately from classifying the object in an object type; and

(c) applying one or more access control rules to the information obtained from the image analysis to determine whether the second object is attempting to breach the controlled access area by utilizing the authorization supplied by the first object in violation thereof,

wherein the controlled access area is limited to objects that are classified in a defined object type and have a status that is authorized with respect to the controlled access area, and

wherein the one or more access control rules determine whether the second object is attempting to breach the controlled access area by separately determining (1) whether the classification of the second object is in the defined object type and (2) whether the second object has a status that is authorized with respect to the controlled access area.

2. The method of claim 1, in which the one or more access control rules determine whether the second object is attempting to breach the controlled access area based on proximity of the second object to the first object in the area of observation.

3. The method of claim 2, in which the authorization of the first object is to enter the controlled access area, and the one or more access control rules determine whether the second object is attempting to enter the controlled access area by maintaining proximity to the first object as the first object enters the controlled access area.

4. The method of claim 2, in which the authorization of the first object is to exit the controlled access area, and the one or more access control rules determine whether the second object is attempting to enter the controlled access area by maintaining proximity to the first object as the first object exits the controlled access area.

5. The method of claim 1, in which analyzing the one or more images further comprises using the machine vision processing system to identify motion of the first object and the second object.

6. The method of claim 5, in which the one or more access control rules determine whether the second object is attempting to breach the controlled access area based on motion of the second object relative to the first object.

7. The method of claim 6, in which the one or more access control rules determine whether the second object is following the motion of the first object with respect to the controlled access area.

8. The method of claim 1, further comprising triggering an alarm when the one or more access control rules determine that the second object is attempting to breach the controlled access area.

9. The method of claim 1, further comprising, in response to determining that the second object is attempting to breach the controlled access area, causing an access limiting device associated with the controlled access area to prevent the breach of the controlled access area by the second object.

10. The method of claim 9, in which causing the access limiting device to prevent the breach of the controlled access area comprises causing the access limiting device to block passage by the second object with respect to the controlled access area.

11. The method of claim 9, in which causing the access limiting device to prevent the breach of the controlled access area comprises locking the access limiting device.

12. The method of claim 9, in which the access limiting device has a normal direction of operation and in which causing the access limiting device to prevent the breach of the controlled access area comprises reversing the direction of operation of the access limiting device to be opposite the normal direction of operation.

13. The method of claim 12, in which the access limiting device is a revolving structure and the direction of rotation of the revolving structure is reversed.

14. The method of claim 1, further comprising causing one or more access limiting devices to block passage of the second object and contain the second object within a defined area.

15. The method of claim 1, further comprising receiving access control information from an access control system and applying the one or more access control rules to the access control information in combination with the information obtained from the image analysis to determine whether the second object is attempting to breach the controlled access area.

16. The method of claim 15, further comprising triggering an alarm when the one or more access control rules applied to the access control information and image analysis information determine that the second object is attempting to breach the controlled access area.

17. The method of claim 15, further comprising, in response to determining that the second object is attempting to breach the controlled access area, causing an access limiting device associated with the controlled access area to prevent the breach of the controlled access area by the second object.

18. The method of claim 17, in which causing the access limiting device to prevent the breach of the controlled access area comprises causing the access limiting device to block passage by the second object with respect to the controlled access area.

19. The method of claim 17, in which causing the access limiting device to prevent the breach of the controlled access area comprises locking the access limiting device.

20. The method of claim 17, in which the access limiting device has a normal direction of operation and in which causing the access limiting device to prevent the breach of the controlled access area comprises reversing the direction of operation of the access limiting device to be opposite the normal direction of operation.

21. The method of claim 20, in which the access limiting device is a revolving door and the direction of rotation of the revolving door is reversed.

22. The method of claim 1, in which analyzing the one or more images using a machine vision processing system further comprises using a three-dimensional surface analysis to identify the first or second object.

23. The method of claim 1, in which analyzing the one or more images using a machine vision processing system further comprises using a three-dimensional surface analysis to classify the first or second object.

24. The method of claim 1, in which analyzing the one or more images further comprises determining the presence of a connection between the first and second objects.

25. The method of claim 1, further comprising recording the one or more images.

26. The method of claim 1, further comprising recording information obtained from the image analysis.

27. The method of claim 1, further comprising recording information resulting from applying the one or more access control rules to the information obtained from the image analysis.

28. The method of claim 1, further comprising defining a region of interest within the area of observation.

29. The method of claim 28, further comprising defining a plurality of regions of interest and applying the one or more access control rules based on the region of interest in which the first or second objects are located.

30. The method of claim 28, further comprising using an interactive user interface that displays the area of observation to define the region of interest.

31. The method of claim 1, further comprising using an interactive user interface to configure properties of the image analysis.

32. The method of claim 1, further comprising using an interactive user interface to configure properties of the machine vision processing system.

33. The method of claim 1, further comprising using an interactive user interface to configure properties of an access control system that provides access control information to the machine vision processing system.

49. The method of claim 1, in which the controlled access area is limited to objects classified as vehicles.

50. The method of claim 1, in which the controlled access area is limited to objects classified as packages.

51. The method of claim 1, in which the controlled access area is limited to objects classified as persons.

52. The method of claim 1, wherein if the second object is classified in an object type that is not allowed for the authorization supplied by the first object, the method further comprises actuating an alarm.

53. The method of claim 1, wherein if the second object is classified in an object type that is not allowed for any authorization, the method further comprises actuating an alarm.

54. The method of claim 1, wherein the second object type of the second object is different than the first object type of the first object.

34. An apparatus for detecting a security violation with respect to a controlled access area, comprising:

(a) a stereoscopic imaging device for acquiring one or more stereoscopic images of an area of observation, wherein objects in the area of observation are characterized by (1) an object type and (2) an authorization status indicating whether the object is authorized with respect to the controlled access area; and

(b) a machine vision processing system in communication with the stereoscopic imaging device to receive the one or more images, the machine vision processing system being further configured to analyze the one or more images to identify an object in the area of observation and to classify the object in an object type among a plurality of object types that are pre-defined in the machine vision processing system, and apply one or more access control rules to determine whether the object is attempting to breach the controlled access area in violation of security of the controlled access area,

wherein the controlled access area is limited to objects that are classified in a defined object type and have a status that is authorized with respect to the controlled access area, and

wherein the one or more access control rules applied by the machine vision processing system determine whether the object is attempting to breach the controlled access area by separately determining (1) whether the classification of the object is in the defined object type and (2) whether the second object has a status that is authorized with respect to the controlled access area.

35. The apparatus of claim 34, in which the object in the area of observation is a suspect object, the apparatus further comprising a position sensor disposed on a background object in the area of observation, the machine vision processing system being further configured to use information received from the position sensor to define the background object from the suspect object.

36. The apparatus of claim 35, in which the position sensor is disposed on a door.

37. The apparatus of claim 35, in which the machine vision processing system is further configured to perform a calibration procedure that correlates position sensor information to a position of the background object for determining a background image.

38. The apparatus of claim 34, in which the controlled access area has a designated access point, the machine vision processing system being further configured to determine the direction of motion of the object with respect to the designated access point and apply the one or more access control rules to determine whether the object is attempting to breach the controlled access area by using the designated access point in violation of the security of the controlled access area.

39. The apparatus of claim 38, in which the machine vision processing system is further configured to analyze the one or more images to determine the speed of motion of the object and apply the one or more access control rules to determine whether the speed of motion of the object indicates a thrown object.

40. The apparatus of claim 38, in which the machine vision processing system is further configured to analyze the one or more images to determine the trajectory of motion of the object and apply the one or more access control rules to determine whether the trajectory of motion of the object indicates a thrown object.

41. The apparatus of claim 34, further comprising an alarm that is triggered when the one or more access control rules determine that the object is attempting to breach the controlled access area in violation of the security of the controlled access area.

42. The apparatus of claim 34, in which the object in the area of observation is a second object, the machine vision processing system being further configured to identify and classify a first object in the area of observation having supplied an authorization with respect to the controlled access area, and to apply the one or more access control rules to determine whether the second object is attempting to breach the controlled access area by utilizing the authorization supplied by the first object in violation thereof.

43. The apparatus of claim 42, in which the one or more access control rules are configured to determine whether the second object is attempting to breach the controlled access area based on proximity of the second object to the first object in the area of observation.

44. The apparatus of claim 43, in which the authorization of the first object is to enter the controlled access area, and the one or more access control rules are configured to determine whether the second object is attempting to enter the controlled access area by maintaining proximity to the first object as the first object enters the controlled access area.

45. The apparatus of claim 43, in which the authorization of the first object is to exit the controlled access area, and the one or more access control rules are configured to determine whether the second object is attempting to enter the controlled access area by maintaining proximity to the first object as the first object exits the controlled access area.

46. The apparatus of claim 34, further comprising an access control system associated with the controlled access area in which the access control system is configured to receive information from the object for accessing the controlled access area.

47. The apparatus of claim 46, further comprising an access limiting device in communication with the access control system, in which the access control system is configured to control the operation of the access limiting device.

48. The apparatus of claim 34, wherein the stereoscopic imaging device for acquiring the one or more stereographic images is placed directly overhead the area of observation.