Adaptive multi-modal integrated biometric identification detection and surveillance systems

• US 8,976,237 B2
• Filed: 01/10/2013
• Issued: 03/10/2015
• Est. Priority Date: 09/17/2004
• Status: Active Grant

• Current Assignee: Sensormatic Electronics LLC (Johnson Controls International plc)
• Original Assignee: Proximex Corporation (Apollo Global Management, Inc.)
• Inventors: Cheng, Ken P., Chang, Edward Y., Wang, Yuan-Fang
• Primary Examiner(s): Grant, II, Jerome
• Application Number:
  US13/738,655
• Publication Number:
  US 20130141559A1
• Time in Patent Office: 789 Days
• Field of Search: 713/186, 713/202, 713/320, 713/201, 713/200, 340/506, 358/143, 358/147, 358/161, 358/169, 707/103, 707/4, 382/103, 382/209, 382/276, 382/277, 382/289, 382/291, 382/293, 382/294, 382/295, 382/282, 382/305, 382/115, 382/107, 382/190, 382/116, 345/629, 348/143, 348/151, 348/155, 348/169, 348/77, 709/227, 709/215, 709/203, 709/315
• US Class Current: 348/77
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/70 Multimodal biometrics, e.g. combining information from different biometric modalities
• CPC Class Codes: G08B 13/196 using television cameras
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A surveillance system is provided that includes at least one sensor disposed in a security area of a surveillance region to sense an occurrence of a potential security breach event; a plurality of cameras is disposed in the surveillance region; at least one camera thereof has a view of the security area and can be configured to automatically gather biometric information concerning at least one subject person in the vicinity of the security area in response to the sensing of a potential security breach event; one or more other of the plurality of cameras can be configured to search for the at least one subject person; a processing system is programmed to produce a dossier corresponding to the at least one subject person to match biometric information of one or more persons captured by one or more of the other cameras with corresponding biometric information in the dossier.

Claims

1. A surveillance method comprising:

using at least one event sensor disposed in a security area of a surveillance region to sense an occurrence of a potential security breach event;

using at least one camera with a view of the security area in which the event is sensed to gather biometric information concerning at least one person in the view of the one camera at a first time that is about the time of the sensing of the event;

producing a subject dossier corresponding to the at least one person;

using the at least one camera and one or more other cameras that have different views of different regions in the surveillance region such that, as the at least one person travels out of a first region and into a second region of the surveillance region, the at least one person travels out of the view of the at least one camera and into the view of at least a second camera from the one or more other cameras; and

matching biometric information of one or more persons, that is captured by the one or more other cameras at a second time that differs from the first time, with corresponding biometric information in the subject dossier.

2. The method of claim 1,

wherein the at least one camera gathers clothing color;

and

wherein matching involves matching clothing color of one or more persons captured by the one or more other cameras with clothing color in the subject dossier.

3. The method of claim 1,

wherein the at least one camera gathers a facial feature signature;

and

wherein matching involves matching facial feature signature of one or more persons captured by the one or more other cameras with a facial feature signature in the subject dossier.

4. The method of claim 1,

wherein the at least one camera gathers hair color;

and

wherein matching involves matching hair color of one or more persons captured by the one or more other cameras with hair color in the subject dossier.

5. The method of claim 1,

wherein the at least one camera gathers skin color;

and

wherein matching involves matching skin color of one or more persons captured by the one or more other cameras with skin color in the subject dossier.

6. The method of claim 1,

wherein the at least one camera gathers one or more of clothing color, facial features, skin color and hair color;

and

wherein matching involves matching at least one of clothing color, facial features, skin color and hair color of one or more persons captured by the one or more other cameras with corresponding one or more of clothing color, facial features, skin color and hair color in the subject dossier.

7. The method of claim 1,

wherein the at least one camera gathers one or both of estimated height and weight;

and

wherein matching involves matching at least one of height and weight of one or more persons captured by the one or more other cameras with corresponding or both of estimated height and weight in the subject dossier.

8. The method of claim 1, wherein the second time occurs when the at least one person has moved outside of the view of the at least one camera.

10. The method of claim 1,

wherein the at least one camera gathers estimated speed of travel of the at least one person;

and

further including determining which cameras are in the vicinity of the at least one camera based upon the estimated speed of travel.

11. The method of claim 1,

wherein the at least one camera gathers estimated direction of travel and estimated speed of travel of the at least one person;

and

further including determining which cameras are in the vicinity of the at least one camera based upon the estimated direction of travel and the estimated speed of travel.

12. The method of claim 1,

wherein the at least one camera gathers one or more of clothing color, facial features, skin color, hair color, estimated height and estimated weight;

wherein matching involves matching at least one of clothing color, facial features, skin color, hair color, estimated height and estimated weight of one or more persons captured by the one or more other cameras with corresponding one or more of clothing color, facial features, skin color, hair color, estimated height and estimated weight in the subject dossier.

13. The method of claim 1,

wherein the at least one camera gathers one or more of clothing color, facial features, skin color, hair color, estimated height and estimated weight;

wherein the at least one camera gathers one or both of estimated direction of travel and estimated speed of travel of the at least one person; and

further including determining which cameras are in the vicinity of the at least one camera based upon at least one of the estimated direction of travel and the estimated speed of travel; and

wherein matching involves matching at least one of clothing color, facial features, skin color, hair color, estimated height and estimated weight of one or more persons captured by the one or more other cameras with corresponding one or more of clothing color, facial features, skin color, hair color, estimated height and estimated weight in the subject dossier.

14. The method of claim 1 further comprising:

providing to a control console display an image of a person captured by the one or more other cameras having biometric information matching biometric information in the subject dossier.

15. The method of claim 1,

wherein the at least one camera with a view of the security area also serves as an event sensor.

9. A surveillance method comprising:

using at least one event sensor disposed in a security area of a surveillance region to sense an occurrence of a potential security breach event;

using a plurality of cameras that have different views of different regions in the surveillance region, wherein as at least one person travels out of a first region and into a second region of the surveillance region, the at least one person travels out of the view of at least one camera and into the view of at least a second camera from the one or more other cameras;

using the at least one camera with a view of the security area in which the event is sensed to gather biometric information concerning at least one person in the vicinity of the security area at about the time of the sensing of the event, wherein the at least one camera gathers estimated direction of travel of the at least one person;

producing a subject dossier corresponding to the at least one person; and

determining which one or more other cameras are in the vicinity of the at least one camera based upon the estimated direction of travel; and

matching biometric information of one or more persons captured by the one or more other cameras in the vicinity of the at least one camera with corresponding biometric information in the subject dossier.

16. A surveillance system comprising:

at least one sensor disposed in a security area of a surveillance region to sense an occurrence of a potential security breach event;

a plurality of cameras having corresponding views of different regions in the surveillance region, wherein as at least one person travels out of a first region and into a second region of the surveillance region, the at least one person travels out of the view of at least one camera and into the view of one or more other cameras;

wherein the at least one camera of the plurality has a view of the first regions which corresponds to the security area and can be configured to automatically gather biometric information concerning at least one subject person in the vicinity of the security area at a first time in response to the sensing of a potential security breach event in the security area;

wherein one or more other of the plurality of cameras can be configured to capture biometric information for one or more persons in the corresponding views, at a second time, while the at least one subject person has traveled outside of the view of the at least one camera;

a processing system;

wherein the processing system is programmable to produce a subject dossier corresponding to the at least one subject person; and

wherein the processing system is programmable to search for the at least one subject person by matching biometric information of one or more persons captured by the one or more of the other of the plurality of cameras with corresponding biometric information in the subject dossier.

17. The system of claim 16, wherein the at least one camera serves as an event sensor.

18. The system of claim 16,

wherein the processing system is programmable to produce a subject dossier that includes at least one or more of clothing color, facial features, skin color, hair color, estimated height and estimated weight;

wherein the processing system is programmable to match at least one of clothing color, facial features, skin color, hair color, estimated height and estimated weight of one or more persons captured by the one or more other cameras with corresponding one or more of clothing color, facial features, skin color, hair color, estimated height and estimated weight in the subject dossier.

19. The system of claim 16, wherein the processing system is configured to estimate at least one of direction of travel and speed of travel of the at least one subject person;

and to determine which of the one or more other cameras have a view of the vicinity to which the at least one subject person is moving.

20. A surveillance system comprising:

a plurality of sensors disposed in a surveillance region to sense an occurrence of a potential security breach event;

a plurality of cameras disposed in the surveillance region and having corresponding views of different regions in the surveillance region such that as at least one subject person travels out of a first region and into a second region of the surveillance region, the at least one subject person travels out of the view of at least one camera and into the view of one or more other cameras, wherein one or more of the plurality of cameras can be configured to automatically gather one or both of biometric information and temporal identifying information concerning at least one subject person in the vicinity of the corresponding camera, in response to sensing of a potential security breach event in the vicinity of such camera;

wherein one or more of the plurality of cameras can be configured to capture biometric information for one or more persons in the corresponding views, when the at least one subject person has traveled outside of the view of the corresponding camera;

a processing system wherein the processing system is programmable to produce a subject dossier including one or both of biometric information or temporal identifying information corresponding to the at least one subject person; and

wherein the processing system is programmable to search for the at least one subject person by matching one or both of biometric information or temporal identifying information of one or more persons captured by one or more cameras.

21. The system of claim 20, wherein the processing system is configured to estimate at least one of direction of travel and speed of travel of the at least one subject person;

and to determine which of the one or more other cameras have a view of the vicinity to which the at least one subject person is moving.