Adaptive multi-modal integrated biometric identification detection and surveillance systems

• US 8,373,753 B2
• Filed: 05/05/2011
• Issued: 02/12/2013
• Est. Priority Date: 09/17/2004
• Status: Active Grant

• Current Assignee: Sensormatic Electronics LLC (Johnson Controls International plc)
• Original Assignee: Proximex Corporation (Apollo Global Management, Inc.)
• Inventors: Cheng, Ken P., Chang, Edward Y., Wang, Yuan-Fang
• Primary Examiner(s): Grant, II, Jerome
• Application Number:
  US13/101,149
• Publication Number:
  US 20110228094A1
• Time in Patent Office: 649 Days
• Field of Search: 713/186, 713/202, 713/320, 713/201, 713/200, 340/506, 358/143, 358/147, 358/161, 358/169, 707/103, 707/4, 382/103, 382/209, 382/276, 382/277, 382/289, 382/291, 382/293, 382/294, 382/295, 382/305, 382/282, 382/115, 382/107, 382/190, 382/116, 345/629, 348/143, 348/151, 348/155, 348/161, 348/169, 709/227, 709/215, 709/203, 709/315
• US Class Current: 348/143
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/70 Multimodal biometrics, e.g. combining information from different biometric modalities
• CPC Class Codes: G08B 13/196 using television cameras
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A surveillance system is provided that includes at least one sensor disposed in a security area of a surveillance region to sense an occurrence of a potential security breach event; a plurality of cameras is disposed in the surveillance region; at least one camera thereof has a view of the security area and can be configured to automatically gather biometric information concerning at least one subject person in the vicinity of the security area in response to the sensing of a potential security breach event; one or more other of the plurality of cameras can be configured to search for the at least one subject person; a processing system is programmed to produce a dossier corresponding to the at least one subject person to match biometric information of one or more persons captured by one or more of the other cameras with corresponding biometric information in the dossier.

Claims

1. A surveillance method comprising;

using at least one event sensor disposed in a security area of a surveillance region to sense an occurrence of a potential security breach event, the surveillance region having multiple areas;

using a first camera, with a view of a first area in the surveillance region in which the event is sensed, to gather subject biometric information concerning at least a first person in the first area at a first time that corresponds to the time of the sensing of the event;

producing a subject dossier corresponding to the first person in response to sensing the occurrence of the event, the subject dossier including the subject biometric information;

using at least a second camera, with a view of a second area in the surveillance region, to gather additional biometric information concerning persons in a second area at a second time that differs from the first time;

matching the additional biometric information for one of the persons captured by at least the second camera with the corresponding subject biometric information in the subject dossier for the first person.

2. The method of claim 1,

wherein the first and second cameras gather clothing color;

and

wherein matching involves matching clothing color of one or more persons captured by the second camera with clothing color in the subject dossier.

3. The method of claim 1,

wherein the first and second cameras gather a facial feature signature;

and

wherein matching involves matching facial feature signature of one or more persons captured by the second camera with a facial feature signature in the subject dossier.

4. The method of claim 1,

wherein the first and second cameras gather hair color;

and

wherein matching involves matching hair color of one or more persons captured by the second camera with hair color in the subject dossier.

5. The method of claim 1,

wherein the first and second cameras gather skin color;

and

wherein matching involves matching skin color of one or more persons captured by the second cameras with skin color in the subject dossier.

6. The method of claim 1,

wherein the first and second cameras gather one or more of clothing color, facial features, skin color and hair color;

and

wherein matching involves matching at least one of clothing color, facial features, skin color and hair color of one or more persons captured by the second camera with corresponding one or more of clothing color, facial features, skin color and hair color in the subject dossier.

7. The method of claim 1,

wherein the first and second cameras gather one or both of estimated height and weight;

and wherein matching involves matching at least one of height and weight of one or more persons captured by the second cameras with corresponding or both of estimated height and weight in the subject dossier.

8. The method of claim 1,

wherein the first and second cameras gather estimated direction of travel of the first person;

and

further including determining which cameras are in the vicinity of the first camera based upon the estimated direction of travel.

9. The method of claim 1,

wherein the first and second cameras gather estimated speed of travel of the first person;

and

further including determining which cameras are in the vicinity of the first camera based upon the estimated speed of travel.

10. The method of claim 1,

wherein the first and second cameras gather estimated direction of travel and estimated speed of travel of the first person;

and

further including determining which cameras are in the vicinity of the first camera based upon the estimated direction of travel and the estimated speed of travel.

11. The method of claim 1,

wherein the first and second cameras gather one or more of clothing color, facial features, skin color, hair color, estimated height and estimated weight;

wherein matching involves matching at least one of clothing color, facial features, skin color, hair color, estimated height and estimated weight of one or more persons captured by at least the second with corresponding one or more of clothing color, facial features, skin color, hair color, estimated height and estimated weight in the subject dossier.

12. The method of claim 1,

wherein the first and second cameras gather one or more of clothing color, facial features, skin color, hair color, estimated height and estimated weight;

wherein the first and second cameras gather one or both of estimated direction of travel and estimated speed of travel of the at least one person; and

further including determining which cameras are in the vicinity of the first camera based upon at least one of the estimated direction of travel and the estimated speed of travel; and

wherein matching involves matching at least one of clothing color, facial features, skin color, hair color, estimated height and estimated weight of one or more persons captured by the second camera with corresponding one or more of clothing color, facial features, skin color, hair color, estimated height and estimated weight in the subject dossier.

13. The method of claim 1 further comprising:

providing to a control console display an image of the one person captured by the second camera having the additional biometric information matching the subject biometric information in the subject dossier.

14. The method of claim 1,

wherein the first camera with a view of the security area also serves as an event sensor.

15. The method of claim 1, further comprising prioritizing the subject biometric information in the subject dossier to be used to make an estimate of which persons in view of the second camera are candidates for a match.

16. The method of claim 1, further comprising adding features incrementally to the subject dossier as the features become available.

17. The method of claim 1, further comprising wherein the matching includes employing the subject biometric information incrementally based on a prioritization of the subject and additional biometric information.

18. The method of claim 1, wherein the first and second areas are not continuous.

19. The method of claim 1, wherein the first and second areas are disparate areas.

20. The method of claim 1, wherein the first and second areas are partially overlapping.

21. A surveillance system comprising:

at least one sensor disposed in a security area of a surveillance region to sense an occurrence of a potential security breach event, the surveillance region having multiple areas;

a plurality of cameras;

wherein a first camera of the plurality of cameras has a view of a first area in the surveillance region and can be configured to automatically gather subject biometric information concerning at least a first subject person in the first area at a first time in response to the sensing of a potential security breach event in the security area;

wherein one or more other of the plurality of cameras, with a view of a second area in the surveillance region, can be configured to search for persons in the second area at a second time that differs from the first time to gather additional biometric information;

a processing system;

wherein the processing system is programmable to produce a subject dossier corresponding to the first subject person, the subject dossier including the subject biometric information; and

wherein the processing system is programmable to match the addition biometric information for one of the persons captured by the one or more of the other cameras with corresponding subject biometric information in the subject dossier for the first person.

22. The system of claim 21, wherein the first camera serves as an event sensor.

23. The system of claim 21,

wherein the processing system is programmable to produce a subject dossier that includes at least one or more of clothing color, facial features, skin color, hair color, estimated height and estimated weight as the subject biometric information;

wherein the processing system is programmable to match at least one of clothing color, facial features, skin color, hair color, estimated height and estimated weight of one or more persons captured by the one or more other cameras with corresponding one or more of clothing color, facial features, skin color, hair color, estimated height and estimated weight in the subject dossier.

24. The surveillance system of claim 21, wherein the processing system prioritizes the subject biometric information in the subject dossier to be used to make an estimate of which one of the persons in view of the second camera are candidates for a match.

25. The surveillance system of claim 21, wherein the processing system adds features incrementally to the subject dossier as the features become available.

26. The surveillance system of claim 21, wherein the processing system employs the subject biometric information incrementally based on a prioritization of the subject and additional biometric information.

27. The surveillance system of claim 21, wherein the first and second areas are not continuous.

28. The surveillance system of claim 21, wherein the first and second areas are disparate areas.

29. The surveillance system of claim 21, wherein the first and second areas are partially overlapping.