Biometric matching technology

• US 9,292,749 B2
• Filed: 10/14/2015
• Issued: 03/22/2016
• Est. Priority Date: 04/09/2012
• Status: Active Grant

• Current Assignee: Accenture Global Services Limited (Accenture PLC)
• Original Assignee: Accenture Global Services Limited (Accenture PLC)
• Inventors: Tiwari, Abhinav, Paul, Sanjoy
• Primary Examiner(s): Rahmjoo, Manuchehr
• Application Number:
  US14/882,552
• Publication Number:
  US 20160034765A1
• Time in Patent Office: 160 Days
• Field of Search: 382/115
• US Class Current: 1/1
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5854 using shape and object relationship
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/23 Clustering techniques
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 30/242 Division of the character sequences into groups prior to recognition; Selection of dictionaries
• CPC Class Codes: G06V 30/244 using graphical properties, e.g. alphabet type or font
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/161 Detection; Localisation; Normalisation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G06V 40/70 Multimodal biometrics, e.g. combining information from different biometric modalities

Abstract

Biometric matching technology, in which a watch list is managed, multiple images of a potential suspect are accessed, and parallel pre-processing of the multiple images is controlled. Based on the pre-processing, an image of the potential suspect to use in matching against the watch list is determined and the determined image is used to search sorted biometric data included in the watch list. A subset of persons from the watch list is identified based on the search and parallel analysis of the determined image of the potential suspect against detailed biometric data associated with the subset of persons in the watch list is controlled. Based on the parallel analysis, it is determined whether the potential suspect matches a person in the watch list and a result is outputted based on the determination.

Claims

1. A system comprising:

a database storage;

at least one processor; and

at least one memory coupled to the at least one processor having stored thereon instructions which, when executed by the at least one processor, causes the at least one processor to perform operations comprising;

for each of a plurality of potential reference images;

comparing, for each of a plurality of images of persons on a watch list, the potential reference image with the image; and

determining, for each of the plurality of images of the persons on the watch list, a similarity score that represents a similarity between the potential reference image and the image based on comparing the potential reference image with the image;

determining which of the potential reference images have similarity scores that are most evenly distributed for the plurality of images of the persons on the watch list;

selecting, as a reference image, a single potential reference image that has similarity scores that are most evenly distributed for the plurality of images of the persons on the watch list;

managing, in the database storage, a) biometric data for the persons on the watch list using the similarity scores for the selected reference image and b) associations to additional biometric data for the persons on the watch list;

accessing an image of a potential suspect;

computing a similarity score that represents similarity between the image of the potential suspect and the selected reference image in response to accessing the image of the potential suspect;

searching the watch list using the computed similarity score;

identifying, using a result of the search of the watch list using the computed similarity score, a subset of the persons on the watch list each having a similarity score within a threshold of the computed similarity score for the potential suspect;

determining whether the potential suspect matches a person in the subset of the persons on the watch list; and

outputting a result based on determining whether the potential suspect matches a person in the subset of the persons on the watch list.

2. The system of claim 1, wherein managing a) the biometric data for the persons on the watch list using the similarity scores for the selected reference image and b) associations to additional biometric data for the persons on the watch list comprises maintaining numeric index values for the persons on the watch list using a similarity score for an image of the person and the selected reference image and maintaining blobs of images of the persons on the watch list stored on different partition servers.

3. The system of claim 1, wherein managing a) the biometric data for the persons on the watch list using the similarity scores for the selected reference image and b) associations to additional biometric data for the persons on the watch list comprises:

creating, in a relational database storage, a table for the watch list;

accessing biometric data for a person to be included on the watch list;

determining, for each person on the watch list, an index value from the accessed biometric data for the person and that represents a similarity score for the selected reference image and an image of the person;

storing, for each person on the watch list, the index value with an identifier for the person at an appropriate location in the table;

determining, for each person on the watch list, whether blob data for the accessed biometric data of the person meets a threshold storage size; and

based on determining whether the blob data for the accessed biometric data of the person meets the threshold storage size, performing one of;

for each person on the watch list for which the blob data for the accessed biometric data of the person does not meet the threshold storage size, storing the blob data in the table with an identifier for the person, or

for each person on the watch list for which the blob data for the accessed biometric data of the person meets the threshold storage size, storing the blob data in a non-relational storage and storing a link to the blob data in the table with the identifier.

4. The system of claim 1, wherein the operations further comprise:

determining at least one criterion relevant to the image of the potential suspect based on at least one of sensor or camera data;

selecting, as the reference image, the single potential reference image from among multiple reference images based on the at least one criterion; and

accessing the selected reference image, wherein computing the similarity score that represents similarity between the image of the potential suspect and the selected reference image comprises computing the similarity score that represents similarity between the image of the potential suspect and the selected reference image selected from among the multiple reference images.

5. The system of claim 1, wherein the operations further comprise:

adding one or more additional persons to the watch list;

evaluating a distribution of similarity scores in the watch list;

selecting, as the reference image, the single potential reference image from among multiple reference images based on evaluating the distribution of similarity scores in the watch list; and

accessing the selected reference image, wherein computing the similarity score that represents similarity between the image of the potential suspect and the selected reference image comprises computing the similarity score that represents similarity between the image of the potential suspect and the selected reference image selected from among the multiple reference images.

6. The system of claim 1, wherein the operations further comprise:

determining at least one criterion relevant to the image of the potential suspect based on at least one of sensor or camera data;

evaluating distribution of similarity scores in the watch list;

selecting, as the reference image, the single potential reference image from among multiple reference images based on the at least one criterion and evaluating the distribution of similarity scores in the watch list; and

accessing the selected reference image, wherein computing the similarity score that represents similarity between the image of the potential suspect and the selected reference image comprises computing the similarity score that represents similarity between the image of the potential suspect and the selected reference image selected from among the multiple reference images.

7. The system of claim 1, wherein identifying the subset of the persons on the watch list each having a similarity score within a threshold of the computed similarity score for the potential suspect comprises:

determining a quality measure for the image of the potential suspect and the images of the persons on the watch list;

setting a score threshold based on the determined quality measure for the image of the potential suspect and the images of the persons on the watch list; and

identifying a subset of the persons on the watch list each having a similarity score within the set score threshold of the computed similarity score for the potential suspect.

8. The system of claim 1, wherein identifying the subset of the persons on the watch list each having a similarity score within a threshold of the computed similarity score for the potential suspect comprises:

determining a number of matches within a set of matches representative of the subset of the persons on the watch list;

determining whether the number of matches within the set of matches is within a match threshold; and

based on determining whether the number of matches within the set of matches is within the match threshold, performing one of;

in response to determining that the number of matches within the set of matches is within the match threshold, using the set of matches as the subset of the persons on the watch list for which to perform additional processing, or

in response to determining that the number of matches within the set of matches is not within the match threshold;

selecting a new reference image that is different than the selected reference image;

identifying a new set of matches within the score threshold for the selected new reference image; and

using at least a portion of the new set of matches as the subset of the persons on the watch list.

9. The system of claim 8, wherein using at least the portion of the new set of matches as the subset of the persons on the watch list comprises:

determining a set of common matches that includes matches which are in both a) the set of matches and b) the new set of matches;

determining a number of matches within the set of common matches;

determining whether the number of matches within the set of common matches is within a match threshold; and

based on determining whether the number of matches within the set of common matches is within the match threshold, performing one of;

based on determining that the number of matches within the set of common matches is within the match threshold, using the matches in the set of common matches as the subset of the persons on the watch list for which to perform additional processing, or

based on a determining that the number of matches within the set of common matches is not within the match threshold, continuing to narrow potential matches using an additional reference image.

10. The system of claim 1, wherein:

accessing the image of the potential suspect comprises accessing two or more images of the potential suspect; and

the operations further comprise;

controlling parallel pre-processing of the two or more images of the potential suspect to enhance a quality of each of the two or more images; and

based on the parallel pre-processing, determining an image of the potential suspect to use in computing the similarity score that represents similarity between the image of the potential suspect and the selected reference image, wherein computing the similarity score that represents similarity between the image of the potential suspect and the selected reference image comprises computing the similarity score that represents similarity between the determined image of the potential suspect and the selected reference image.

11. The system of claim 10, wherein controlling the parallel pre-processing of the two or more images of the potential suspect to enhance a quality of each of the two or more images comprises:

determining at least one criterion relevant to the two or more images of the potential suspect based on at least one of sensor or camera data;

selecting a number of images of the potential suspect to use in the parallel pre-processing based on the at least one criterion; and

controlling the parallel pre-processing of the selected number of images of the potential suspect.

12. The system of claim 10, wherein controlling the parallel pre-processing of the two or more images of the potential suspect to enhance the quality of each of the two or more images comprises:

determining at least one criterion relevant to the two or more images of the potential suspect based on at least one of sensor or camera data;

selecting types of operations to perform in the parallel pre-processing based on the at least one criterion; and

controlling the parallel pre-processing of the two or more images of the potential suspect using the selected types of operations to perform in the parallel pre-processing.

13. The system of claim 10, wherein controlling the parallel pre-processing of the two or more images of the potential suspect to enhance the quality of each of the two or more images comprises:

determining at least one criterion relevant to the two or more images of the potential suspect based on at least one of sensor or camera data;

selecting a number images of the potential suspect to use in the parallel pre-processing based on the at least one criterion;

selecting types of operations to perform in the parallel pre-processing based on the at least one criterion; and

controlling the parallel pre-processing of the selected number of images of the potential suspect using the selected types of operations to perform in the parallel pre-processing.

14. The system of claim 1, wherein the operations further comprise:

determining a context of a situation associated with the image of the potential suspect;

setting a batch size based on the determined context of the situation;

selecting a batch of images from the subset of the persons on the watch list based on the batch size;

controlling parallel analysis of the image of the potential suspect against the batch of images; and

controlling parallel analysis of the image of the potential suspect against additional batches of images until images of all the persons in the subset of the persons on the watch list have been analyzed, wherein determining whether the potential suspect matches a person in the subset of the persons on the watch list comprises determining whether the potential suspect matches a person in the subset of the persons on the watch list using a result of the controlled parallel analysis of the image of the potential suspect against the batches of images from the subset of the persons on the watch list.

15. The system of claim 1, wherein the operations further comprise:

determining a criticality of a situation associated with the image of the potential suspect;

setting a batch size based on the determined criticality of the situation;

selecting a batch of images from the subset of the persons on the watch list based on the batch size;

controlling parallel analysis of the image of the potential suspect against the batch of images; and

controlling the parallel analysis of the image of the potential suspect against additional batches of images until images of all persons in the subset of the persons on the watch list have been analyzed, wherein determining whether the potential suspect matches a person in the subset of the persons on the watch list comprises determining whether the potential suspect matches a person in the subset of the persons on the watch list using a result of the controlled parallel analysis of the image of the potential suspect against the batches of images from the subset of the persons on the watch list.

16. The system of claim 1, wherein the operations further comprise:

determining a context of a situation associated with the image of the potential suspect;

determining a criticality of a situation associated with the image of the potential suspect;

setting a batch size based on the determined context of the situation and the determined criticality of the situation;

selecting a batch of images from the subset of the persons on the watch list based on the batch size;

controlling parallel analysis of the image of the potential suspect against the batch of images; and

controlling parallel analysis of the image of the potential suspect against additional batches of images until images of all persons in the subset of the persons on the watch list have been analyzed, wherein determining whether the potential suspect matches a person in the subset of the persons on the watch list comprises determining whether the potential suspect matches a person in the subset of the persons on the watch list using a result of the controlled parallel analysis of the image of the potential suspect against the batches of images from the subset of the persons on the watch list.

17. The system of claim 1, wherein determining whether the potential suspect matches a person in the subset of the persons on the watch list comprises:

determining a number of unprocessed persons in the subset of the persons on the watch list for which the image of the potential suspect has not been compared to an image of the person;

determining whether the number of unprocessed persons in the subset of the persons on the watch list is greater than a batch size; and

in response to determining that the number of unprocessed persons in the subset is less than or equal to the batch size and for at least some potential suspects, performing a comparison of the image of the potential suspect against all remaining unprocessed persons in the subset of the persons on the watch list, wherein determining whether the potential suspect matches a person in the subset of the persons on the watch list comprises determining whether the potential suspect matches a person in the subset of the persons on the watch list using a result the performance of the comparison of the image of the potential suspect against all remaining unprocessed persons in the subset of the persons on the watch list.

18. The system of claim 17, wherein the operations further comprise:

for at least some potential suspects, selecting a batch of persons from the subset of the persons on the watch list in response to determining that the number of unprocessed persons in the subset is greater than the batch size;

performing a comparison of the image of the potential suspect against all persons in the selected batch of persons;

determining whether a match is found within the selected batch using a result of the comparison of the image of the potential suspect against all persons in the selected batch; and

based on determining whether the match is found within the selected batch using the result of the comparison of the image of the potential suspect against all persons in the selected batch, performing one of;

in response to determining that a match is found within the selected batch, ending processing and outputting the match, or

in response to determining that a match is not found within the selected batch, processing one or more additional batches of images in parallel until all of the subset of the persons on the watch list have been processed or a match is found.

19. The system of claim 1, wherein the operations further comprise:

identifying a person of interest from the watch list;

determining a profile for the identified person of interest from the watch list;

accessing one or more images of a crowd that includes multiple potential suspects;

comparing the profile for the identified person of interest from the watch list to profiles of the multiple potential suspects included in the one or more images of the crowd; and

selecting, from among the multiple potential suspects included in the one or more images of the crowd, the potential suspect for further processing based on determining, in response to comparing the profile for the identified person of interest from the watch list to profiles of the multiple potential suspects included in the one or more images of the crowd, that the profile for the identified person of interest from the watch list matches a profile of the potential suspect,

wherein the operations of accessing an image of the potential suspect, identifying the subset of the persons from the watch list, determining whether the potential suspect matches a person in the subset of the persons on the watch list, and outputting the result based on determining whether the potential suspect matches a person in the subset of the persons on the watch list are conditioned on the selection of the potential suspect for further processing.

20. A method comprising:

for each of a plurality of potential reference images;

comparing, for each of a plurality of images of persons on a watch list, the potential reference image with the image; and

determining, for each of the plurality of images of the persons on the watch list, a similarity score that represents a similarity between the potential reference image and the image based on comparing the potential reference image with the image;

determining which of the potential reference images have similarity scores that are most evenly distributed for the plurality of images of the persons on the watch list;

selecting, as a reference image, a single potential reference image that has similarity scores that are most evenly distributed for the plurality of images of the persons on the watch list;

managing, in a database storage, a) biometric data for the persons on the watch list using the similarity scores for the selected reference image and b) associations to additional biometric data for the persons on the watch list;

accessing an image of a potential suspect;

computing a similarity score that represents similarity between the image of the potential suspect and the selected reference image in response to accessing the image of the potential suspect;

searching the watch list using the computed similarity score;

identifying, using a result of the search of the watch list using the computed similarity score, a subset of the persons on the watch list each having a similarity score within a threshold of the computed similarity score for the potential suspect;

determining whether the potential suspect matches a person in the subset of the persons on the watch list; and

outputting a result based on determining whether the potential suspect matches a person in the subset of the persons on the watch list.

21. At least one non-transitory computer-readable storage medium encoded with executable instructions that, when executed by at least one processor, cause the at least one processor to perform operations comprising:

for each of a plurality of potential reference images;

comparing, for each of a plurality of images of persons on a watch list, the potential reference image with the image; and

determining, for each of the plurality of images of the persons on the watch list, a similarity score that represents a similarity between the potential reference image and the image based on comparing the potential reference image with the image;

determining which of the potential reference images have similarity scores that are most evenly distributed for the plurality of images of the persons on the watch list;

selecting, as a reference image, a single potential reference image that has similarity scores that are most evenly distributed for the plurality of images of the persons on the watch list;

managing, in a database storage, a) biometric data for the persons on the watch list using the similarity scores for the selected reference image and b) associations to additional biometric data for the persons on the watch list;

accessing an image of a potential suspect;

computing a similarity score that represents similarity between the image of the potential suspect and the selected reference image in response to accessing the image of the potential suspect;

searching the watch list using the computed similarity score;

identifying, using a result of the search of the watch list using the computed similarity score, a subset of the persons on the watch list each having a similarity score within a threshold of the computed similarity score for the potential suspect;

determining whether the potential suspect matches a person in the subset of the persons on the watch list; and

outputting a result based on determining whether the potential suspect matches a person in the subset of the persons on the watch list.