Biometric matching technology

• US 9,483,689 B2
• Filed: 06/09/2016
• Issued: 11/01/2016
• Est. Priority Date: 04/09/2012
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture Global Services Limited (Accenture PLC)
• Inventors: Tiwari, Abhinav, Paul, Sanjoy
• Primary Examiner(s): Rahmjoo, Manuchehr
• Application Number:
  US15/177,714
• Publication Number:
  US 20160283782A1
• Time in Patent Office: 145 Days
• Field of Search: 382/118
• US Class Current: 1/1
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 30/242 Division of the character sequences into groups prior to recognition; Selection of dictionaries
• CPC Class Codes: G06V 30/244 using graphical properties, e.g. alphabet type or font
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/70 Multimodal biometrics, e.g. combining information from different biometric modalities

Abstract

Biometric matching technology, in which a watch list is managed, multiple images of a potential suspect are accessed, and parallel pre-processing of the multiple images is controlled. Based on the pre-processing, an image of the potential suspect to use in matching against the watch list is determined and the determined image is used to search sorted biometric data included in the watch list. A subset of persons from the watch list is identified based on the search and parallel analysis of the determined image of the potential suspect against detailed biometric data associated with the subset of persons in the watch list is controlled. Based on the parallel analysis, it is determined whether the potential suspect matches a person in the watch list and a result is outputted based on the determination.

Claims

1. A system comprising:

a database storage;

at least one processor; and

at least one memory coupled to the at least one processor having stored thereon instructions which, when executed by the at least one processor, causes the at least one processor to perform operations comprising;

managing, in the database storage, a) data for persons on a watch list sorted using similarity scores for a reference image that represent, for each of the persons on the watch list, a similarity between the reference image and an image of the person and b) associations to biometric data for each of the persons on the watch list;

accessing two or more images of a potential suspect;

determining at least one criterion relevant to the image of the potential suspect based on at least one of sensor or camera data;

selecting types of operations to perform in parallel pre-processing based on the at least one criterion;

controlling parallel pre-processing of the two or more images of the potential suspect to enhance a quality of each of the two or more images using the selected types of operations to perform in the parallel pre-processing;

determining an image of the potential suspect to use in computing a similarity score that represents similarity between the image of the potential suspect and the reference image based on the pre-processing;

computing a similarity score that represents a similarity between the determined image of the potential suspect and the reference image based on selecting the reference image;

searching the watch list using the computed similarity score;

identifying, using a result of the search of the watch list using the computed similarity score, a subset of persons on the watch list each having a similarity score within a threshold of the computed similarity score for the reference image;

determining whether the potential suspect matches a person in the subset of the persons on the watch list; and

outputting a result based on determining whether the potential suspect matches a person in the subset of the persons on the watch list.

2. The system of claim 1, wherein managing a) the data for the persons on the watch list sorted using the similarity scores for the reference image and b) associations to biometric data for each of the persons on the watch list comprises maintaining numeric index values for the persons on the watch list using a similarity score for an image of the person and the reference image and maintaining blobs of images of the persons on the watch list stored on different partition servers.

3. The system of claim 1, wherein managing a) the data for the persons on the watch list sorted using the similarity scores for the reference image and b) associations to biometric data for each of the persons on the watch list comprises:

creating, in a relational database storage, a table for the watch list;

accessing biometric data for a person to be included on the watch list;

determining, for each person on the watch list, an index value from the accessed biometric data for the person and that represents a similarity score for the reference image and an image of the person;

storing, for each person on the watch list, the index value with an identifier for the person at an appropriate location in the table;

determining, for each person on the watch list, whether blob data for the accessed biometric data of the person meets a threshold storage size; and

based on determining whether the blob data for the accessed biometric data of the person meets the threshold storage size, performing one of;

for each person on the watch list for which the blob data for the accessed biometric data of the person does not meet the threshold storage size, storing the blob data in the table with an identifier for the person, or

for each person on the watch list for which the blob data for the accessed biometric data of the person meets the threshold storage size, storing the blob data in non-relational storage and storing a link to the blob data in the table with the identifier.

4. The system of claim 1, wherein identifying the subset of the persons on the watch list each having a similarity score within a threshold of the computed similarity score for the reference image comprises:

determining a quality measure for the image of the potential suspect and the images of the persons on the watch list;

setting a score threshold based on the determined quality measure for the image of the potential suspect and the images of the persons on the watch list; and

identifying a subset of the persons on the watch list each having a similarity score within the set score threshold of the computed similarity score for the reference image.

5. The system of claim 1, wherein identifying the subset of the persons on the watch list each having a similarity score within a threshold of the computed similarity score for the reference image comprises:

determining a number of matches within a set of matches representative of the subset of the persons on the watch list;

determining whether the number of matches within the set of matches is within a match threshold; and

based on determining whether the number of matches within the set of matches is within the match threshold, performing one of;

in response to determining that the number of matches within the set of matches is within the match threshold, using the set of matches as the subset of the persons on the watch list for which to perform additional processing, or

in response to determining that the number of matches within the set of matches is not within the match threshold;

selecting a new reference image that is different than the reference image;

identifying a new set of matches within the score threshold for the selected new reference image; and

using at least a portion of the new set of matches as the subset of the persons on the watch list.

6. The system of claim 5, wherein using at least the portion of the new set of matches as the subset of the persons on the watch list comprises:

determining a set of common matches that includes matches which are in both a) the set of matches and b) the new set of matches;

determining a number of matches within the set of common matches;

determining whether the number of matches within the set of common matches is within a match threshold; and

based on determining whether the number of matches within the set of common matches is within the match threshold, performing one of;

based on determining that the number of matches within the set of common matches is within the match threshold, using the matches in the set of common matches as the subset of the persons on the watch list for which to perform additional processing, or

based on determining that the number of matches within the set of common matches is not within the match threshold, continuing to narrow potential matches using an additional reference image.

7. The system of claim 1, wherein controlling the parallel pre-processing of the two or more images of the potential suspect to enhance a quality of each of the two or more images comprises:

selecting a number of images from the two or more images of the potential suspect to use in the parallel pre-processing based on the at least one criterion; and

controlling the parallel pre-processing of the selected number of images of the potential suspect to enhance a quality of each of the two or more images using the selected types of operations to perform in the parallel pre-processing.

8. The system of claim 1, wherein the operations further comprise:

determining a context of a situation associated with the image of the potential suspect;

setting a batch size based on the determined context of the situation;

selecting a batch of images from the subset of the persons on the watch list based on the batch size;

controlling parallel analysis of the image of the potential suspect against the batch of images; and

controlling parallel analysis of the image of the potential suspect against additional batches of images until images of all the persons in the subset of the persons on the watch list have been analyzed, wherein determining whether the potential suspect matches a person in the subset of the persons on the watch list comprises determining whether the potential suspect matches a person in the subset of the persons on the watch list using a result of the controlled parallel analysis of the image of the potential suspect against the batches of images from the subset of the persons on the watch list.

9. The system of claim 1, wherein the operations further comprise:

determining a criticality of a situation associated with the image of the potential suspect;

setting a batch size based on the determined criticality of the situation;

selecting a batch of images from the subset of the persons on the watch list based on the batch size;

controlling parallel analysis of the image of the potential suspect against the batch of images; and

controlling the parallel analysis of the image of the potential suspect against additional batches of images until images of all persons in the subset of the persons on the watch list have been analyzed, wherein determining whether the potential suspect matches a person in the subset of the persons on the watch list comprises determining whether the potential suspect matches a person in the subset of the persons on the watch list using a result of the controlled parallel analysis of the image of the potential suspect against the batches of images from the subset of the persons on the watch list.

10. The system of claim 1, wherein the operations further comprise:

determining a context of a situation associated with the image of the potential suspect;

determining a criticality of a situation associated with the image of the potential suspect;

setting a batch size based on the determined context of the situation and the determined criticality of the situation;

selecting a batch of images from the subset of the persons on the watch list based on the batch size;

controlling parallel analysis of the image of the potential suspect against the batch of images; and

controlling parallel analysis of the image of the potential suspect against additional batches of images until images of all persons in the subset of the persons on the watch list have been analyzed, wherein determining whether the potential suspect matches a person in the subset of the persons on the watch list comprises determining whether the potential suspect matches a person in the subset of the persons on the watch list using a result of the controlled parallel analysis of the image of the potential suspect against the batches of images from the subset of the persons on the watch list.

11. The system of claim 1, wherein determining whether the potential suspect matches a person in the subset of the persons on the watch list comprises:

determining a number of unprocessed persons in the subset of the persons on the watch list for which the image of the potential suspect has not been compared to an image of the person;

determining whether the number of unprocessed persons in the subset of the persons on the watch list is greater than a batch size; and

in response to determining that the number of unprocessed persons in the subset is less than or equal to the batch size and for at least some potential suspects, performing a comparison of the image of the potential suspect against all remaining unprocessed persons in the subset of the persons on the watch list, wherein determining whether the potential suspect matches a person in the subset of the persons on the watch list comprises determining whether the potential suspect matches a person in the subset of the persons on the watch list using a result the performance of the comparison of the image of the potential suspect against all remaining unprocessed persons in the subset of the persons on the watch list.

12. The system of claim 11, wherein the operations further comprise:

for at least some potential suspects, selecting a batch of persons from the subset of the persons on the watch list in response to determining that the number of unprocessed persons in the subset is greater than the batch size;

performing a comparison of the image of the potential suspect against all persons in the selected batch of persons;

determining whether a match is found within the selected batch using a result of the comparison of the image of the potential suspect against all persons in the selected batch; and

based on determining whether the match is found within the selected batch using the result of the comparison of the image of the potential suspect against all persons in the selected batch, performing one of;

in response to determining that a match is found within the selected batch, ending processing and outputting the match, or

in response to determining that a match is not found within the selected batch, processing one or more additional batches of images in parallel until all of the subset of the persons on the watch list have been processed or a match is found.

13. The system of claim 1, wherein the operations further comprise:

identifying a person of interest from the watch list;

determining a profile for the identified person of interest from the watch list;

accessing one or more images of a crowd that includes multiple potential suspects;

comparing the profile for the identified person of interest from the watch list to profiles of the multiple potential suspects included in the one or more images of the crowd; and

selecting, from among the multiple potential suspects included in the one or more images of the crowd, the potential suspect for further processing based on determining, in response to comparing the profile for the identified person of interest from the watch list to profiles of the multiple potential suspects included in the one or more images of the crowd, that the profile for the identified person of interest from the watch list matches a profile of the potential suspect,

wherein accessing the image of the potential suspect, identifying the subset of the persons from the watch list, determining whether the potential suspect matches a person in the subset of the persons on the watch list, and outputting the result based on determining whether the potential suspect matches a person in the subset of the persons on the watch list are conditioned on the selection of the potential suspect for further processing.

14. A method comprising:

managing, in a database storage, a) data for persons on a watch list sorted using similarity scores for a reference image that represent, for each of the persons on the watch list, a similarity between the reference image and an image of the person and b) associations to biometric data for each of the persons on the watch list;

accessing two or more images of a potential suspect;

determining at least one criterion relevant to the image of the potential suspect based on at least one of sensor or camera data;

selecting types of operations to perform in parallel pre-processing based on the at least one criterion;

controlling parallel pre-processing of the two or more images of the potential suspect to enhance a quality of each of the two or more images using the selected types of operations to perform in the parallel pre-processing;

determining an image of the potential suspect to use in computing a similarity score that represents similarity between the image of the potential suspect and the reference image based on the pre-processing;

computing a similarity score that represents a similarity between the determined image of the potential suspect and the reference image based on selecting the reference image;

searching the watch list using the computed similarity score;

identifying, using a result of the search of the watch list using the computed similarity score, a subset of persons on the watch list each having a similarity score within a threshold of the computed similarity score for the reference image;

determining whether the potential suspect matches a person in the subset of the persons on the watch list; and

outputting a result based on determining whether the potential suspect matches a person in the subset of the persons on the watch list.

15. The method of claim 14, wherein managing a) the data for the persons on the watch list sorted using the similarity scores for the reference image and b) associations to biometric data for each of the persons on the watch list comprises maintaining numeric index values for the persons on the watch list using a similarity score for an image of the person and the reference image and maintaining blobs of images of the persons on the watch list stored on different partition servers.

16. The method of claim 14, wherein managing a) the data for the persons on the watch list sorted using the similarity scores for the reference image and b) associations to biometric data for each of the persons on the watch list comprises:

creating, in a relational database storage, a table for the watch list;

accessing biometric data for a person to be included on the watch list;

determining, for each person on the watch list, an index value from the accessed biometric data for the person and that represents a similarity score for the reference image and an image of the person;

storing, for each person on the watch list, the index value with an identifier for the person at an appropriate location in the table;

determining, for each person on the watch list, whether blob data for the accessed biometric data of the person meets a threshold storage size; and

based on determining whether the blob data for the accessed biometric data of the person meets the threshold storage size, performing one of;

for each person on the watch list for which the blob data for the accessed biometric data of the person does not meet the threshold storage size, storing the blob data in the table with an identifier for the person, or

for each person on the watch list for which the blob data for the accessed biometric data of the person meets the threshold storage size, storing the blob data in non-relational storage and storing a link to the blob data in the table with the identifier.

17. The method of claim 14, wherein identifying the subset of the persons on the watch list each having a similarity score within a threshold of the computed similarity score for the reference image comprises:

determining a quality measure for the image of the potential suspect and the images of the persons on the watch list;

setting a score threshold based on the determined quality measure for the image of the potential suspect and the images of the persons on the watch list; and

identifying a subset of the persons on the watch list each having a similarity score within the set score threshold of the computed similarity score for the reference image.

18. The method of claim 14, wherein identifying the subset of the persons on the watch list each having a similarity score within a threshold of the computed similarity score for the reference image comprises:

determining a number of matches within a set of matches representative of the subset of the persons on the watch list;

determining whether the number of matches within the set of matches is within a match threshold; and

based on determining whether the number of matches within the set of matches is within the match threshold, performing one of;

in response to determining that the number of matches within the set of matches is within the match threshold, using the set of matches as the subset of the persons on the watch list for which to perform additional processing, or

in response to determining that the number of matches within the set of matches is not within the match threshold;

selecting a new reference image that is different than the reference image;

identifying a new set of matches within the score threshold for the selected new reference image; and

using at least a portion of the new set of matches as the subset of the persons on the watch list.

19. The method of claim 18, wherein using at least the portion of the new set of matches as the subset of the persons on the watch list comprises:

determining a set of common matches that includes matches which are in both a) the set of matches and b) the new set of matches;

determining a number of matches within the set of common matches;

determining whether the number of matches within the set of common matches is within a match threshold; and

based on determining whether the number of matches within the set of common matches is within the match threshold, performing one of;

based on determining that the number of matches within the set of common matches is within the match threshold, using the matches in the set of common matches as the subset of the persons on the watch list for which to perform additional processing, or

based on determining that the number of matches within the set of common matches is not within the match threshold, continuing to narrow potential matches using an additional reference image.

20. At least one non-transitory computer-readable storage medium encoded with executable instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:

managing, in a database storage, a) data for persons on a watch list sorted using similarity scores for a reference image that represent, for each of the persons on the watch list, a similarity between the reference image and an image of the person and b) associations to biometric data for each of the persons on the watch list;

accessing two or more images of a potential suspect;

determining at least one criterion relevant to the image of the potential suspect based on at least one of sensor or camera data;

selecting types of operations to perform in parallel pre-processing based on the at least one criterion;

controlling parallel pre-processing of the two or more images of the potential suspect to enhance a quality of each of the two or more images using the selected types of operations to perform in the parallel pre-processing;

determining an image of the potential suspect to use in computing a similarity score that represents similarity between the image of the potential suspect and the reference image based on the pre-processing;

computing a similarity score that represents a similarity between the determined image of the potential suspect and the reference image based on selecting the reference image;

searching the watch list using the computed similarity score;

identifying, using a result of the search of the watch list using the computed similarity score, a subset of persons on the watch list each having a similarity score within a threshold of the computed similarity score for the reference image;

determining whether the potential suspect matches a person in the subset of the persons on the watch list; and

outputting a result based on determining whether the potential suspect matches a person in the subset of the persons on the watch list.