ADAPTIVE MULTI-MODAL INTEGRATED BIOMETRIC IDENTIFICATION AND SURVEILLANCE SYSTEMS

• US 20150138332A1
• Filed: 01/28/2015
• Published: 05/21/2015
• Est. Priority Date: 09/17/2004
• Status: Active Grant

• Current Assignee: Sensormatic Electronics LLC (Johnson Controls International plc)
• Original Assignee: Proximex Corporation (Apollo Global Management, Inc.)
• Inventors: Cheng, Ken P., Chang, Edward Y., Wang, Yuan-Fang
• Granted Patent: US 9,432,632 B2
• Time in Patent Office: Days
• US Class Current: 348/77
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/70 Multimodal biometrics, e.g. combining information from different biometric modalities
• CPC Class Codes: G08B 13/196 using television cameras
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A surveillance system is provided that includes at least one sensor disposed in a security area of a surveillance region to sense an occurrence of a potential security breach event; a plurality of cameras is disposed in the surveillance region; at least one camera thereof has a view of the security area and can be configured to automatically gather biometric information concerning at least one subject person in the vicinity of the security area in response to the sensing of a potential security breach event; one or more other of the plurality of cameras can be configured to search for the at least one subject person; a processing system is programmed to produce a dossier corresponding to the at least one subject person to match biometric information of one or more persons captured by one or more of the other cameras with corresponding biometric information in the dossier.

Claims

1-18. -18. (canceled)

19. A surveillance system comprising:

multiple data gathering agents disposed throughout a surveillance region to monitor the surveillance region, the data gathering agents to provide corresponding signatures identifying aspects of a subject person in the surveillance region, the signatures including one or both of biometric information or temporal information;

a database storing a map corresponding to the surveillance region;

a processing system programmed to aggregate the signatures associated with the subject person to produce a subject dossier including one or both of biometric information or temporal information corresponding to the subject person, the processing system programmed to correlate the data gathering agents, the subject dossier and the map; and

a control terminal to display the map for at least a portion of the surveillance region and produce indicia on the map indicative of a path followed by the subject person, the indicia based on the signatures in the subject dossier and the data gathering agents.

20. The system of claim 19, wherein the indicia represent a dashed line produced on the map on the display showing the path.

21. The system of claim 19, wherein the processing system correlating different types of the data gathering agents to facilitate tracking of the subject person.

22. The system of claim 19, wherein the data gathering agents including at least one of a camera, digital video recorder, access control or bio-chemical detector.

23. The system of claim 19, Wherein the database stores zoom maps, and wherein the control terminal is further configured to display zoom maps associated with locations in the surveillance region, the zoom maps representing at least one of security gates, check-in counters, airport fairway, parking area, access entrance, or check-in counters.

24. The system of claim 19, wherein the data gathering agents include a group of cameras and event sensors.

25. The system of claim 19, wherein the processing system is programmed to index the signatures within the subject dossier to permit incremental searches for individuals within the surveillance region.

26. The system of claim 19, wherein the processing system is programmed to perform classification and matching of the signatures.

27. The system of claim 19, wherein the data gathering agents collect information about at least one of a walking pattern or gait of the subject, the processing system programmed to analyze the at least one of a walking pattern or gait of the subject person.

28. The system of claim 19, wherein the data gathering agents collect information about at least one of an iris scan and facial features of the subject person.

29. The system of claim 19, wherein the data gathering agents collect information in connection with multiple individuals, the processing system programmed to prioritize the information and analyzing a portion of the information to make an initial estimate of which of the individuals are candidate individuals for a potential match to the subject person, the processing system programmed to incrementally add other portions of the information to the analysis to assess which of the candidate individuals match the subject person.

30. The system of claim 19, wherein the processing system is programmed to apply a monitoring rule wherein at least one of the data gathering agents monitors a particular area in the surveillance region and when an individual loiters in the particular area for more than a select period of time, the processing system declares an alert.

31. The system of claim 19, wherein the processing system is programmed to direct at least one of the data gathering agents to zoom in on a face of the individual, and the processing system programmed to attempt to match the face of the individual to a watch list.

32. The system of claim 19, wherein at least one of the data gathering agents scans data on at least one of a smart card or an identification (ID) card.

33. The system of claim 19, wherein the data gathering agents collect biometric signatures, temporal information and object tracking locations, the biometric signatures including at least one of hair color, skin tone/color, weight, build, or height, the temporal information including at least one of speed, direction, location, past places visited, path, or past activities.

34. A surveillance method comprising:

monitoring a surveillance region with multiple data gathering agents disposed throughout the surveillance region to provide corresponding signatures identifying aspects of a subject person in the surveillance region, the signatures including one or both of biometric information or temporal information;

storing a map corresponding to the surveillance region;

aggregating the signatures associated with the subject person to produce a subject dossier including one or both of biometric information or temporal information corresponding to the subject person;

correlating the data gathering agents, the subject dossier and the map; and

displaying the map for at least a portion of the surveillance region and producing indicia on the map indicative of a path followed by the subject person, the indicia based on the signatures in the subject dossier and the data gathering agents.

35. The method of claim 34, wherein the indicia represent a dashed line produced on the map when displayed showing the path.

36. The method of claim 34, further comprising correlating signatures from different types of the data gathering agents to facilitate tracking of the subject person.

37. The method of claim 34, wherein the data gathering agents including at least two of a camera, digital video recorder, access control or bio-chemical detector.

38. The method of claim 34, further comprising storing and displaying zoom maps associated with locations in the surveillance region, the zoom maps representing at least one of security gates, check-in counters, airport fairway, parking area, access entrance, or check-in counters.

39. The method of claim 34, further comprising indexing the signatures within the subject dossier and performing incremental searches for individuals within the surveillance region.

40. The method of claim 34, further comprising performing classification and matching of the signatures.

41. The method of claim 34, wherein the information in the signatures includes collecting information about at least one of a walking pattern or gait of the subject, the method analyzing the at least one of a walking pattern or gait of the subject person.

42. The method of claim 34, wherein the information in the signatures includes information about at least one of an iris scan and facial features of the subject person.

43. The method of claim 34, further comprising collecting information in connection with multiple individuals, prioritizing the information, analyzing a portion of the information to make an initial estimate of which of the individuals are candidate individuals for a potential match to the subject person, and incrementally adding other portions of the information to the analysis to assess which of the candidate individuals match the subject person.

44. The method of claim 34, further comprising\applying a monitoring rule wherein at least one of the data gathering agents monitors a particular area in the surveillance region and when an individual biters in the particular area for more than a select period of time, declaring an alert.

45. The method of claim 34, further comprising directing at least one of the data gathering agents to zoom in on a face of the individual, and attempting to match the face of the individual to a watch list.

46. The method of claim 34, further comprising scanning data on at least one of a smart card or an identification (ID) card.

47. The method of claim 34, further comprising collecting biometric signatures, temporal information and object tracking locations, the biometric signatures including at least one of hair color, skin tone/color, weight, build, or height, the temporal information including at least one of speed, direction, location, past places visited, path, or past activities.