Biometric matching technology

• US 9,195,893 B2
• Filed: 12/30/2014
• Issued: 11/24/2015
• Est. Priority Date: 04/09/2012
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture Global Services Limited (Accenture PLC)
• Inventors: Tiwari, Abhinav, Paul, Sanjoy
• Primary Examiner(s): Rahmjoo, Manuchehr
• Application Number:
  US14/585,480
• Publication Number:
  US 20150178569A1
• Time in Patent Office: 329 Days
• Field of Search: 382/115, 382/209, 382/276, 382/277, 382/289, 382/291, 382/293, 382/294, 382/295
• US Class Current: 1/1
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 30/242 Division of the character sequences into groups prior to recognition; Selection of dictionaries
• CPC Class Codes: G06V 30/244 using graphical properties, e.g. alphabet type or font
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/70 Multimodal biometrics, e.g. combining information from different biometric modalities

Abstract

Biometric matching technology, in which a watch list is managed, multiple images of a potential suspect are accessed, and parallel pre-processing of the multiple images is controlled. Based on the pre-processing, an image of the potential suspect to use in matching against the watch list is determined and the determined image is used to search sorted biometric data included in the watch list. A subset of persons from the watch list is identified based on the search and parallel analysis of the determined image of the potential suspect against detailed biometric data associated with the subset of persons in the watch list is controlled. Based on the parallel analysis, it is determined whether the potential suspect matches a person in the watch list and a result is outputted based on the determination.

Claims

1. A system comprising:

a database storage;

at least one processor; and

at least one memory coupled to the at least one processor having stored thereon instructions which, when executed by the at least one processor, causes the at least one processor to perform operations comprising;

managing, in the database storage, a watch list that includes sorted biometric data for persons in the watch list and associations to additional biometric data for the persons in the watch list;

accessing multiple images of a potential suspect to use to determine if the potential suspect matches one of the persons in the watch list;

determining at least one criterion relevant to the multiple images of the potential suspect based on at least one of sensor and camera data;

selecting two or more images to use in pre-processing based on the at least one criterion;

controlling parallel pre-processing of the selected two or more images of the potential suspect to enhance a quality of each of the selected two or more images;

based on the pre-processing, determining an image of the potential suspect to use in matching against the watch list;

using the determined image of the potential suspect to search the sorted biometric data included in the watch list;

identifying a subset of persons from the watch list based on the search of the sorted biometric data included in the watch list;

controlling parallel analysis of the determined image of the potential suspect against biometric data associated with the subset of persons in the watch list;

determining whether the potential suspect matches a person in the subset of persons in the watch list using a result of the controlled parallel analysis of the determined image of the potential suspect against the biometric data associated with the subset of persons in the watch list; and

outputting a result based on the determination of whether the potential suspect matches a person in the subset of persons in the watch list.

2. The system of claim 1, wherein managing the watch list that includes sorted biometric data for persons in the watch list and associations to additional biometric data for persons in the watch list comprises maintaining numeric index values for persons in the watch list sorted in the watch list and maintaining blobs of images of the persons in the watch list stored on different partition servers.

3. The system of claim 1, wherein managing the watch list that includes sorted biometric data for persons in the watch list and associations to additional biometric data for persons in the watch list comprises:

creating, in a relational database storage, a table for the watch list;

accessing biometric data for a person to include on the watch list;

determining an index value from the accessed biometric data;

storing the index value with an identifier for the person at an appropriate location in the table;

determining whether blob data for the accessed biometric data of the person meets a threshold storage size;

based on a determination that the blob data for the accessed biometric data of the person does not meet the threshold storage size, storing the blob data in a table with an identifier for the person; and

based on a determination that the blob data for the accessed biometric data of the person meets the threshold storage size, storing the blob data in non-relational storage and storing a link to the blob data in the table with the identifier.

4. The system of claim 3:

wherein creating, in the relational database storage, the table for the watch list comprises creating the table for the watch list in a relational database included in a storage account provided by a cloud service provider; and

wherein storing the blob data in the non-relational storage comprises storing the blob data in a non-relational storage included in the storage account provided by the cloud service provider and controlling the blob data to be stored on a different partition server than all other blob data stored for the persons on the watch list.

5. The system of claim 1, wherein controlling parallel pre-processing of the selected two or more images of the potential suspect to enhance a quality of each of the selected two or more images comprises:

determining at least one criterion relevant to the multiple images of the potential suspect based on at least one of sensor and camera data;

selecting types of operations to perform in pre-processing based on the at least one criterion; and

controlling parallel pre-processing of the selected two or more images of the potential suspect using the selected types of operations to perform in pre-processing.

6. The system of claim 1, wherein controlling parallel pre-processing of the selected two or more images of the potential suspect to enhance a quality of each of the selected two or more images comprises:

determining at least one criterion relevant to the multiple images of the potential suspect based on at least one of sensor and camera data;

selecting a second two or more images to use in pre-processing based on the at least one criterion;

selecting types of operations to perform in pre-processing based on the at least one criterion; and

controlling parallel pre-processing of the selected second two or more images of the potential suspect using the selected types of operations to perform in pre-processing.

7. The system of claim 1 where:

using the determined image of the potential suspect to search the sorted biometric data included in the watch list comprises;

accessing a reference image,

computing a similarity score that represents similarity between the determined image of the potential suspect and the reference image based on the accessed reference image, and

searching the watch list using the computed similarity score; and

identifying the subset of persons from the watch list based on the search of the sorted biometric data included in the watch list comprises identifying a subset of persons in the watch list having a similarity score within a threshold of the computed similarity score for the potential suspect.

8. The system of claim 7, wherein accessing the reference image comprises:

determining at least one criterion relevant to the multiple images of the potential suspect based on at least one of sensor and camera data;

selecting, from among multiple reference images, a reference image based on the at least one criterion; and

accessing the selected reference image.

9. The system of claim 7, wherein accessing the reference image comprises:

evaluating distribution of similarity scores in the watch list;

selecting, from among multiple reference images, a reference image based on the evaluation of the distribution of similarity scores in the watch list; and

accessing the selected reference image.

10. The system of claim 7, wherein accessing the reference image comprises:

determining at least one criterion relevant to the multiple images of the potential suspect based on at least one of sensor and camera data;

evaluating distribution of similarity scores in the watch list;

selecting, from among multiple reference images, a reference image based on the at least one criterion and the evaluation of the distribution of similarity scores in the watch list; and

accessing the selected reference image.

11. The system of claim 7, wherein identifying the subset of persons in the watch list having a similarity score within a threshold of the computed similarity score for the potential suspect comprises:

determining a quality measure for the determined image of the potential suspect and images of persons in the watch list;

setting a score threshold based on the determined quality measure for the determined image of the potential suspect and images of persons in the watch list; and

identifying a subset of persons in the watch list having a similarity score within the set score threshold of the computed similarity score for the potential suspect.

12. The system of claim 7, wherein identifying the subset of persons in the watch list having a similarity score within a threshold of the computed similarity score for the potential suspect comprises:

determining a number of matches within a set of matches representative of the subset of persons in the watch list;

determining whether the number of matches within the set of matches is within a match threshold;

based on a determination that the number of matches within the set of matches is within the match threshold, using the set of matches as the subset of persons in the watch list for which to perform additional processing; and

based on a determination that the number of matches within the set of matches is not within the match threshold;

selecting a new reference image that is different than the accessed reference image;

identifying a new set of matches within the score threshold for the new reference image;

determining the number of matches within each of the set of matches and the new set of matches;

determining whether the number of matches within each of the set of matches and the new set of matches is within the match threshold;

based on a determination that the number of matches within each of the set of matches and the new set of matches is within the match threshold, using the matches in each of the set of matches and the new set of matches as the subset of persons in the watch list for which to perform additional processing; and

based on a determination that the number of matches within each of the set of matches and the new set of matches is not within the match threshold, continuing to narrow potential matches using an additional reference image.

13. The system of claim 1, wherein controlling parallel analysis of the determined image of the potential suspect against biometric data associated with the subset of persons in the watch list comprises:

determining a context of a situation associated with the determined image of the potential suspect;

setting a batch size based on the determined context of the situation;

selecting a batch of images from the subset of persons in the watch list based on the batch size;

controlling parallel analysis of the determined image of the potential suspect against the batch of images; and

controlling parallel analysis of the determined image of the potential suspect against additional batches of images until images of all of the subset of persons in the watch list have been analyzed.

14. The system of claim 1, wherein controlling parallel analysis of the determined image of the potential suspect against biometric data associated with the subset of persons in the watch list comprises:

determining a criticality of a situation associated with the determined image of the potential suspect;

setting a batch size based on the determined criticality of the situation;

selecting a batch of images from the subset of persons in the watch list based on the batch size;

controlling parallel analysis of the determined image of the potential suspect against the batch of images; and

controlling parallel analysis of the determined image of the potential suspect against additional batches of images until images of all of the subset of persons in the watch list have been analyzed.

15. The system of claim 1, wherein controlling parallel analysis of the determined image of the potential suspect against biometric data associated with the subset of persons in the watch list comprises:

determining a context of a situation associated with the determined image of the potential suspect;

determining a criticality of a situation associated with the determined image of the potential suspect;

setting a batch size based on the determined context of the situation and the determined criticality of the situation;

selecting a batch of images from the subset of persons in the watch list based on the batch size;

controlling parallel analysis of the determined image of the potential suspect against the batch of images; and

controlling parallel analysis of the determined image of the potential suspect against additional batches of images until images of all of the subset of persons in the watch list have been analyzed.

16. The system of claim 1, wherein controlling parallel analysis of the determined image of the potential suspect against biometric data associated with the subset of persons in the watch list comprises:

determining whether a number of unprocessed persons in the subset of persons in the watch list is greater than a batch size; and

based on a determination that the number of unprocessed persons in the subset is less than or equal to the batch size, performing a comparison of the determined image of the potential suspect against all remaining unprocessed persons in the subset of persons in the watch list.

17. The system of claim 16, wherein controlling parallel analysis of the determined image of the potential suspect against biometric data associated with the subset of persons in the watch list further comprises:

selecting a batch of persons from the subset of persons in the watch list based on a determination that the number of unprocessed persons in the subset is greater than the batch size;

performing a comparison of the determined image of the potential suspect against all persons in the selected batch of persons;

determining whether a match is found within the selected batch based on the comparison of the determined image of the potential suspect against all persons in the selected batch;

based on a determination that a match is found within the selected batch, ending processing and outputting the match; and

based on a determination that a match is not found within the selected batch, processing one or more additional batches of images in parallel until all of the subset of persons in the watch list have been processed or a match is found.

18. The system of claim 1, wherein the operations further comprise:

identifying a person of interest from the watch list;

determining a profile for the identified person of interest from the watch list;

accessing one or more images of a crowd that includes multiple potential suspects;

comparing the profile for the identified person of interest from the watch list to profiles of the multiple potential suspects included in the one or more images of the crowd; and

selecting, from among the multiple potential suspects included in the one or more images of the crowd, the potential suspect for further processing based on the comparison revealing that the profile for the identified person of interest from the watch list matches a profile of the potential suspect,

wherein the operations of accessing multiple images of the potential suspect to use to determine if the potential suspect matches one of the persons in the watch list, controlling parallel pre-processing of the selected two or more images of the potential suspect to enhance a quality of each of the selected two or more images, determining the image of the potential suspect to use in matching against the watch list, using the determined image of the potential suspect to search the sorted biometric data included in the watch list, identifying the subset of persons from the watch list, controlling parallel analysis of the determined image of the potential suspect against biometric data associated with the subset of persons in the watch list, determining whether the potential suspect matches a person in the subset of persons in the watch list, and outputting the result based on the determination of whether the potential suspect matches a person in the subset of persons in the watch list are conditioned on the selection of the potential suspect for further processing.

19. A method comprising:

managing, in a database storage, a watch list that includes sorted biometric data for persons in the watch list and associations to additional biometric data for the persons in the watch list;

accessing multiple images of a potential suspect to use to determine if the potential suspect matches one of the persons in the watch list;

determining at least one criterion relevant to the multiple images of the potential suspect based on at least one of sensor and camera data;

selecting two or more images to use in pre-processing based on the at least one criterion;

controlling parallel pre-processing of the selected two or more images of the potential suspect to enhance a quality of each of the selected two or more images;

based on the pre-processing, determining an image of the potential suspect to use in matching against the watch list;

using the determined image of the potential suspect to search the sorted biometric data included in the watch list;

identifying a subset of persons from the watch list based on the search of the sorted biometric data included in the watch list;

controlling parallel analysis of the determined image of the potential suspect against biometric data associated with the subset of persons in the watch list;

determining, by at least one processing device, whether the potential suspect matches a person in the subset of persons in the watch list using a result of the controlled parallel analysis of the determined image of the potential suspect against the biometric data associated with the subset of persons in the watch list; and

outputting a result based on the determination of whether the potential suspect matches a person in the subset of persons in the watch list.

20. At least one non-transitory computer-readable storage medium encoded with executable instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:

managing, in a database storage, a watch list that includes sorted biometric data for persons in the watch list and associations to additional biometric data for the persons in the watch list;

accessing multiple images of a potential suspect to use to determine if the potential suspect matches one of the persons in the watch list;

determining at least one criterion relevant to the multiple images of the potential suspect based on at least one of sensor and camera data;

selecting two or more images to use in pre-processing based on the at least one criterion;

controlling parallel pre-processing of the selected two or more images of the potential suspect to enhance a quality of each of the selected two or more images;

based on the pre-processing, determining an image of the potential suspect to use in matching against the watch list;

using the determined image of the potential suspect to search the sorted biometric data included in the watch list;

identifying a subset of persons from the watch list based on the search of the sorted biometric data included in the watch list;

controlling parallel analysis of the determined image of the potential suspect against biometric data associated with the subset of persons in the watch list;

determining whether the potential suspect matches a person in the subset of persons in the watch list using a result of the controlled parallel analysis of the determined image of the potential suspect against the biometric data associated with the subset of persons in the watch list; and

outputting a result based on the determination of whether the potential suspect matches a person in the subset of persons in the watch list.