Methods and apparatus for false positive minimization in facial recognition applications

• US 10,654,942 B2
• Filed: 10/21/2016
• Issued: 05/19/2020
• Est. Priority Date: 10/21/2015
• Status: Active Grant

• Current Assignee: 15 Seconds Of Fame Inc. (Forwardly, Inc.)
• Original Assignee: 15 Seconds Of Fame Inc. (Forwardly, Inc.)
• Inventors: Resnick, Adam, Sabitov, Ruslan, Joshpe, Brett
• Primary Examiner(s): Roz, Mark
• Application Number:
  US15/299,934
• Publication Number:
  US 20170116466A1
• Time in Patent Office: 1,306 Days
• Field of Search: None
• CPC Class Codes: A61K 39/39591 Stabilisation, fragmentation
• CPC Class Codes: A61K 47/26 Carbohydrates, e.g. sugar alcohols, amino sugars, nucleic acids, mono-, di- or oligo-saccharides; Derivatives thereof, e.g. polysorbates, sorbitan fatty acid esters or glycyrrhizin
• CPC Class Codes: C07K 16/18 against material from animals or humans
• CPC Class Codes: C07K 16/40 against enzymes
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06V 10/761 Proximity, similarity or dissimilarity measures
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 20/49 Segmenting video sequences, i.e. computational techniques such as parsing or cutting the sequence, low-level clustering or determining units such as shots or scenes
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

An apparatus can include a processor that can receive location data from a user device, and store the location data in a user profile data structure also storing facial recognition data. The processor can also receive at least one image, and can identify a location based at least in part on a set of characteristics within the at least one image. The processor can, for each user profile data structure stored in a database, compare location data in that user profile data structure to the location. The processor can, when the location data of the user profile data structure and the location match, conduct facial recognition to determine whether the user associated with the user profile data structure can be identified in the at least one image. The processor can then associate the at least one image with the user profile data structure if the user can be identified.

Claims

1. An apparatus, comprising:

a memory; and

a processor operatively coupled to the memory, the processor configured to, at a first time, receive data from a user device, the processor configured to store the data in a user profile data structure, the user profile data structure including facial recognition data of a user associated with the user device and defined based on at least one of two-dimensional facial recognition analytics, three-dimensional facial recognition analytics, or convolutional neural nets (CNN), the processor configured to receive, at a second time different from the first time, at least one image from an image capture device,

the processor configured to identify (1) a venue based at least in part on data associated with the at least one image and (2) an image location within the venue based at least in part on a set of image characteristics within the received at least one image, the processor configured to retrieve, from a database, a plurality of user profile data structures including the user profile data structure, the processor configured to determine whether the user is at the venue based on at least a first portion of the data stored in the user profile data structure from the plurality of user profile data structures,

the processor configured to, when the user is at the venue, determine whether the user associated with the user profile data structure can be identified in the at least one image by analyzing the at least one image with respect to the facial recognition data of the user based on at least one of the two-dimensional facial recognition analytics, the three-dimensional facial recognition analytics, or the CNN to identify a facial recognition confidence score,

the processor configured to (1) determine a user location within the venue based on at least a second portion of the data stored in the user profile data structure, (2) compare the user location within the venue to the image location, and (3) increase the facial recognition confidence score when the user location within the venue and the image location are within a predetermined distance of each other, the processor configured to associate the at least one image with the user profile data structure based on the facial recognition confidence score meeting a predetermined criterion,

the processor configured to not perform facial recognition analysis on the at least one image with respect to the facial recognition data when the user is not at the venue.

2. The apparatus of claim 1, wherein the data received from the user device is at least one of iBeacon data, Global Positioning Service (GPS) data, a seat identifier, social media data, internet web browsing data, purchase data, near field communication (NFC) verification data, cellular network triangulation data, or a Wi-Fi network identifier.

3. The apparatus of claim 1, wherein the user is a first user, the image capture device is one of an autonomous camera or a user client device associated with a second user different from the first user.

4. The apparatus of claim 1, wherein the processor is configured to identify the image location by:

performing image processing on the at least one image;

identifying at least one venue landmark based on the image processing; and

identifying the image location by determining a location of the venue landmark.

5. The apparatus of claim 1, wherein the facial recognition data of the user includes data relating to at least one photograph of the user that is associated with the user profile data structure.

6. The apparatus of claim 1, wherein the at least one image is stored in the database if the user can be identified in the at least one image.

15. The apparatus of claim 1, wherein the processor is configured to automatically send to the user device the at least one image when the facial recognition confidence score satisfies the predetermined criterion.

16. The apparatus of claim 1, wherein the processor is configured to determine the user location within the venue by inferring a section within the venue based on at least the second portion of the data stored in the user profile data structure.

17. The apparatus of claim 1, wherein the first portion of the data stored in the user profile data structure includes location data, the processor is configured to determine the user is at the venue based on the location data stored in the user profile data structure.

18. The apparatus of claim 1, wherein the first portion of the data stored in the user profile data structure includes historical location data associated with the user device, and

the processor is configured to determine the user is at the venue based on the historical location data stored in the user profile data structure.

19. The apparatus of claim 1, wherein the first portion of the data stored in the user profile data structure includes location data and the second portion of the data stored in the user profile data structure includes non-location data, the processor is configured to determine the user is at the venue based on the location data and to determine the user location within the venue based on the non-location data.

7. A method, comprising:

receiving, at a first time, data from a user device;

storing the data in a user profile data structure in a database;

receiving, at a second time different from the first time, at least one image from an image capture device;

identifying (1) a venue based at least in part on data associated with the at least one image and (2) an image location within the venue based at least in part on a set of image characteristics within the at least one image;

determining a user is at the venue based on a first portion of the data stored in the user profile data structure;

determining a user location within the venue based on a second portion of the data stored in the user profile data structure, the second portion of the data being different from the first portion of the data;

comparing the user location to the image location; and

when a distance between the user location and the image location is less than a predetermined distance;

analyzing facial recognition data within the user profile data structure with respect to the at least one image, facial recognition analysis not being performed on the facial recognition data within the user profile data structure with respect to the at least one image when the distance is greater than the predetermined distance, and

storing the at least one image as associated with the user profile data structure when the at least one image matches the facial recognition data.

8. The method of claim 7, wherein the data received from the user device is at least one of iBeacon data, Global Positioning Service (GPS) data, a seat number, social media data, internet web browsing data, purchase data, near field communication (NFC) verification data, cellular network triangulation data, or a Wi-Fi network identifier.

9. The method of claim 7, further comprising:

pre-processing the at least one image to determine contextual information before analyzing the facial recognition data with respect to the at least one image,

the contextual information including at least one of an identifier associated with the venue, a time the at least one image was captured, or a coinciding event that occurred when the at least one image was captured.

10. The method of claim 7, further comprising:

calculating a confidence level for the at least one image based on the analyzing of the facial recognition data with respect to the at least one image; and

determining that the at least one image matches the facial recognition data when the confidence level exceeds a predetermined threshold.

11. The method of claim 7, wherein the at least one image is a video including the at least one image, the method further comprising:

dividing the video into a series of images such that the facial recognition data is analyzed with respect to each image in the series of images when the facial recognition data is analyzed with respect to the at least one image.

12. The method of claim 7, further comprising:

sending a signal indicative of an instruction to graphically render the at least one image at the user device when the at least one image matches the facial recognition data.

13. The method of claim 7, wherein the identifying of the image location includes identifying the image location based at least in part on background scenery or a background landmark included in the at least one image.

14. The method of claim 7, wherein the user profile data structure is from a plurality of user profile data structures stored in the database, each user profile data structure from the plurality of user profile data structures including facial recognition data of a corresponding user, the method further comprising:

discarding the at least one image if the at least one image does not match facial recognition data of at least one user profile data structure from the plurality of user profile data structures.

20. The method of claim 7, further comprising:

sending, automatically and to the user device, the at least one image when the at least one image matches the facial recognition data.

21. The method of claim 7, wherein the determining the user location within the venue includes inferring a section within the venue based on the second portion of the data.

22. The method of claim 7, wherein the first portion of the data includes location data, the determining the user is at the venue includes determining the user is at the venue based on the location data.

23. The method of claim 7, wherein the first portion of the data includes location data and the second portion of the data includes non-location data, the determining the user is at the venue includes determining the user is at the venue based on the location data and the determining the user location within the venue includes determining the user location within the venue based on the non-location data.