Apparatus and methods for facial recognition and video analytics to identify individuals in contextual video streams

• US 10,591,281 B2
• Filed: 10/03/2018
• Issued: 03/17/2020
• Est. Priority Date: 07/15/2015
• Status: Active Grant

• Current Assignee: 15 Seconds Of Fame Inc. (Forwardly, Inc.)
• Original Assignee: 15 Seconds Of Fame Inc. (Forwardly, Inc.)
• Inventors: Sabitov, Ruslan, Betser, Alexander, Resnick, Adam, Huestis, E. Michael, Joshpe, Brett Terry
• Primary Examiner(s): Mariam, Daniel G
• Application Number:
  US16/150,717
• Publication Number:
  US 20190137261A1
• Time in Patent Office: 531 Days
• Field of Search: 382118, 382209, 382218
• CPC Class Codes: G01B 11/14 for measuring distance or clearance between spaced objects or spaced apertures G01B11/26 takes precedence; rangefinders G01C3/00
• CPC Class Codes: G01J 3/26 using multiple reflection, e.g. Fabry-Perot interferometer, variable interference filters
• CPC Class Codes: G01J 3/2823 Imaging spectrometer
• CPC Class Codes: G01J 3/4535 Devices with moving mirror G01J3/4532 takes precedence
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

An apparatus includes a memory, a communication interface in communication with the memory and configured to communicate via a network, and a processor in communication with the memory and the communication interface. The processor receives facial image data associated with a user of a client device, registers the facial image data, and stores the facial image data and contextual data associated with the user in a database. The processor also receives video stream data from at least one image capture device in communication with the network, analyzes the video stream data and contextual data associated with the video stream data to define analyzed video data and analyzed contextual data, respectively, and defines a confidence level based on comparing the data associated with the video stream data to the data stored in the database. The processor defines a user-specific contextual video stream when the confidence level satisfies a criterion.

Claims

1. An apparatus, comprising:

a memory;

a communication interface in communication with the memory and configured to communicate via a network; and

a processor in communication with the memory and the communication interface, the processor configured to;

receive, via the network and the communication interface, a signal including data representing at least one image captured by an image capture device and contextual data associated with the at least one image,

identify a venue based on the contextual data associated with the at least one image,

request location data associated with a client device,

analyze the at least one image via facial recognition with respect to facial image data of a user associated with the client device when the location data associated with the client device is within a threshold distance from a location associated with the venue, the facial image data of the user included in a user profile data structure stored in the memory,

define user-specific data when a result of the analyzing indicates an identification of the user in the at least one image, the user-specific data including an instance of data representing at least one image of the user from the at least one image received from the image capture device and data representing at least an indication of a coinciding event occurring at the venue, and

send, via the network and the communication interface, the user-specific data such that the user can access the user-specific data via the client device.

2. The apparatus of claim 1, wherein the client device is one of a tablet, a smartphone, or a wearable electronic device.

3. The apparatus of claim 1, wherein the coinciding event is an event occurring at the venue at a time that at least partially coincides with a time at which the image capture device captures the at least one image.

4. The apparatus of claim 1, wherein each of the user associated with the client device and the coinciding event are depicted in the at least one image received from the image capture device.

5. The apparatus of claim 1, wherein the processor is configured to define a confidence level associated with identifying the user based on analyzing the at least one image via facial recognition with respect to the facial image data of the user,

the processor is configured to define the user-specific data when the confidence level satisfies a confidence threshold.

6. The apparatus of claim 1, wherein the location data associated with the client device is global positioning system (GPS) location data.

7. The apparatus of claim 1, wherein the requesting location data associated with the client device includes requesting the location data and other contextual data from the client device, the other contextual data including at least one of calendar data, preference data, social media data, check-in data, or purchase data.

8. A method, comprising:

receiving, at a host device and via a network, a signal including data representing at least one image captured by an image capture device;

identifying a venue based on contextual data associated with the at least one image, the contextual data including an indication of a coinciding event occurring at the venue;

requesting location data associated with a client device, data associated with the client device being stored in a database, the data including at least facial image data of a user associated with the client device;

analyzing the at least one image via facial recognition with respect to the facial image data of the user when the location data associated with the client device is within a threshold distance from a location associated with the venue;

defining user-specific data including (1) an instance of data representing at least one image of the user from the at least one image received from the image capture device and (2) data representing at least an indication of the coinciding event, when a result of the analyzing indicates an identification of the user in the at least one image; and

sending, from the host device to the client device, a signal indicative of an instruction to present the user-specific data on a display of the client device.

9. The method of claim 8, wherein the client device is one of a tablet, a smartphone, or a wearable electronic device.

10. The method of claim 8, wherein the coinciding event is an event occurring at the venue at a time that at least partially coincides with a time at which the image capture device captures the at least one image.

11. The method of claim 8, wherein each of the user and the coinciding event is depicted in the at least one image received from the image capture device.

12. The method of claim 8, further comprising:

defining a confidence level associated with identifying the user based on the analyzing the at least one image via facial recognition with respect to the facial image data of the user,

the defining the user-specific data is in response to the confidence level satisfying a confidence threshold.

13. The method of claim 8, wherein the requesting location data associated with the client device includes requesting global positioning system (GPS) location data.

14. The method of claim 8, wherein the requesting location data associated with the client device includes requesting the location data and other contextual data from the client device, the other contextual data including at least one of calendar data, preference data, social media data, check-in data, or purchase data.

15. A method, comprising:

receiving, at a host device and via a network, facial image data of a user associated with a client device;

storing data associated with the user in a database, the data including at least the facial image data and data associated with the client device;

receiving, at the host device and via the network, a signal including image data captured by an image capture device and contextual data associated with the image data;

identifying a venue and a coinciding event occurring at the venue based at least in part on the contextual data;

analyzing the image data via facial recognition with respect to the facial image data of the user stored in the database when global positioning system (GPS) location data associated with the client device is within a proximity threshold associated with the venue;

defining user-specific data when a confidence level associated with identifying the user in the image data during the analyzing satisfies a criterion, the user-specific data including an instance of data representing at least one image of the user from the image data and data representing at least an indication of the coinciding event; and

sending, from the host device to the client device, a signal indicative of an instruction to present the user-specific data on a display of the client device.

16. The method of claim 15, wherein the client device is one of a tablet, a smartphone, or a wearable electronic device.

17. The method of claim 15, wherein the image data includes at least one image of the user captured during a time contemporaneous with the coinciding event.

18. The method of claim 15, wherein the criterion is a minimum confidence level associated with identifying the user in the image data.

19. The method of claim 15, further comprising:

sending, from the host device and to the client device via the network, a request for the GPS location data prior to the analyzing the image data.

20. The method of claim 15, further comprising:

sending, from the host device and to the client device via the network, a request for contextual data associated with the client device, the contextual data including at least one of the GPS location data, calendar data, preference data, social media data, check-in data, or purchase data.