Biometric notification system

• US 9,721,167 B2
• Filed: 02/22/2016
• Issued: 08/01/2017
• Est. Priority Date: 07/21/2008
• Status: Active Grant

• Current Assignee: FaceFirst, LLC
• Original Assignee: FaceFirst, LLC
• Inventors: Hesketh, Gifford, Rosenkrantz, Joseph Ethan
• Primary Examiner(s): Park, Chan
• Assistant Examiner(s): Thomas, Mia M
• Application Number:
  US15/050,326
• Publication Number:
  US 20160224841A1
• Time in Patent Office: 526 Days
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06V 10/95 structured as a network, e.g. client-server architectures
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification

Abstract

The present invention provides a biometric notification system for selectively sending messages to interested recipients. In various embodiments, message trigger criteria, interested recipients, and message content may vary depending upon, among other things, the service being provided.

Claims

1. A biometric notification system (“

BNS”

) with facial recognition and related services, the system comprising;

plural digital video cameras (C₁, C₂ . . . C_n) that monitor respective zones (Z₁, Z₂ . . . Z_n) at a first site;

a data processing system including a processing facility and a database facility;

a network interconnecting the video cameras and the data processing facility;

a video frame acquired from a video camera C_x in zone Z_x includes a first content;

a first facial feature set derived from the first content;

a database of the database facility includes plural cases, each case identifies a person by linking a name with a second content, a second facial feature set, and a case group;

a matched case discovered when the processing facility finds the first feature set matches a second feature set;

a message sent to one or more interested recipients when the processing facility finds that message trigger criteria are met;

wherein message trigger criteria include a requirement that a matched case is discovered;

a service for recognizing persons of interest;

wherein the message trigger criteria include a requirement that the case group of the matched case identify a person known to site operators as a person of interest;

a recipient selection service based on a case group/user group association;

wherein the interested recipients are limited to members of one or more user groups, each such user group associated with a case group of the matched case;

a service for selecting an interested recipient based on proximity to an event;

wherein the first site is a building, Z_x includes a locked door, a person in the video frame is matched with a case including an authorized entry case group, the interested recipient is limited to a building occupant that is closest to the door, and a notification module message instructs the interested recipient to grant access to the particular person identified in the matched case; and

,

wherein the BNS determines the building occupant closest to the door by limiting the cases available for comparison to site staff and by searching zone Zx and adjacent zones for a matched case.

2. A biometric notification system (“

BNS”

) with facial recognition and related services, the system comprising;

plural digital video cameras (C₁, C₂ . . . C_n) that monitor respective zones (Z₁, Z₂ . . . Z_n) at a first site;

a data processing system including a processing facility and a database facility;

a network interconnecting the video cameras and the data processing facility;

a video frame acquired from a video camera C_x in zone Z_x includes a first content;

a first facial feature set derived from the first content;

a database of the database facility includes plural cases, each case identifies a person by linking a name with a second content, a second facial feature set, and a case group;

a matched case discovered when the processing facility finds the first feature set matches a second feature set;

a message sent to one or more interested recipients when the processing facility finds that message trigger criteria are met;

wherein message trigger criteria include a requirement that a matched case is discovered;

a zone alert service;

wherein plural cases include respective different zone alerts;

wherein when a matched case includes a zone alert, message trigger criteria include a requirement that a zone identified by the zone alert correspond with the zone where the first content was acquired; and

,

wherein the intended recipients are selected based on job function and territory, the pool of potential recipients representing multiple different job functions and multiple different territories.

3. A biometric notification system (“

BNS”

) with facial recognition and related services, the system comprising;

plural digital video cameras (C₁, C₂ . . . C_n) that monitor respective zones (Z₁, Z₂ . . . Z_n) at a first site;

a data processing system including a processing facility and a database facility;

a network interconnecting the video cameras and the data processing facility;

a video frame acquired from a video camera C_x in zone Z_x includes a first content;

a first facial feature set derived from the first content;

a database of the database facility includes plural cases, each case identifies a person by linking a name with a second content, a second facial feature set, and a case group;

a matched case discovered when the processing facility finds the first feature set matches a second feature set;

a message sent to one or more interested recipients when the processing facility finds that message trigger criteria are met;

wherein message trigger criteria include a requirement that a matched case is discovered;

a first geofencing service for recognizing a person excluded from a zone;

wherein only a subset of the cases in the database is available for comparison with the first feature set, the subset of cases consisting of persons excluded from zone Z_x; and

,

wherein discovery of a matched case results in a message being sent to site security, the message advising the occurrence of a security breach and a name, image, and zone of the person of the matched case.

4. A biometric notification system (“

BNS”

) with facial recognition and related services, the system comprising;

plural digital video cameras (C₁, C₂ . . . C_n) that monitor respective zones (Z₁, Z₂ . . . Z_n) at a first site;

a data processing system including a processing facility and a database facility;

a network interconnecting the video cameras and the data processing facility;

a video frame acquired from a video camera C_x in zone Z_x includes a first content;

a first facial feature set derived from the first content;

a database of the database facility includes plural cases, each case identifies a person by linking a name with a second content, a second facial feature set, and a case group;

a matched case discovered when the processing facility finds the first feature set matches a second feature set;

a message sent to one or more interested recipients when the processing facility finds that message trigger criteria are met;

wherein message trigger criteria include a requirement that a matched case is discovered;

a second geofencing service for recognizing a person not excluded from a zone;

wherein only a subset of cases in the database is available for comparison with a first feature set, the subset of cases consisting of persons not excluded from zone Z_x; and

,

wherein failure to discover a matched case results in a message being sent to site security, the message advising the occurrence of a security breach and an image and zone of the person that could not be matched.

5. A biometric notification system (“

BNS”

) with facial recognition and related services, the system comprising;

plural digital video cameras (C₁, C₂ . . . C_n) that monitor respective zones (Z₁, Z₂ . . . Z_n) at a first site;

a data processing system including a processing facility and a database facility;

a network interconnecting the video cameras and the data processing facility;

a video frame acquired from a video camera C_x in zone Z_x includes a first content;

a first facial feature set derived from the first content;

a database of the database facility includes plural cases, each case identifies a person by linking a name with a second content, a second facial feature set, and a case group;

a matched case discovered when the processing facility finds the first feature set matches a second feature set;

a message sent to one or more interested recipients when the processing facility finds that message trigger criteria are met;

wherein message trigger criteria include a requirement that a matched case is discovered;

a first intelligent video storage service;

wherein a sequential stream of video frames from a match camera C_x is received by a circular memory buffer and the buffer contents B_x are saved in video storage memory when the second facial feature set is matched with the first facial feature set derived from a target frame in the circular memory buffer; and

,

wherein an interested recipient that receives a message triggered by the match receives with the message a hyperlink for viewing the buffer contents saved in video memory storage.

6. The BNS of claim 5 wherein at the time the match is completed, the circular memory buffer holds x frames acquired before the target frame, pre-roll frames“

, and y frames acquired after the target frame, post-roll frames”

, each of the x frames and the y frames acquired during a respective acquisition time period t1, t2 in excess of a time period t3 required to compare the first facial feature set of the target frame with all of the cases in the database.

7. The BNS of claim 5 wherein the circular buffer size varies with the number of second facial feature sets available for comparison with the first facial feature set of the target frame.

8. The BNS of claim 5 wherein plural circular memory buffers receive sequential video streams from respective cameras that monitor zones adjacent to Zx and these buffer contents are saved in video storage memory when the buffer contents Bx are saved.

9. A biometric notification system (“

BNS”

) with facial recognition and related services, the system comprising;

plural digital video cameras (C₁, C₂ . . . C_n) that monitor respective zones (Z₁, Z₂ . . . Z_n) at a first site;

a data processing system including a processing facility and a database facility;

a network interconnecting the video cameras and the data processing facility;

a video frame acquired from a video camera C_x in zone Z_x includes a first content;

a first facial feature set derived from the first content;

a database of the database facility includes plural cases, each case identifies a person by linking a name with a second content, a second facial feature set, and a case group;

a matched case discovered when the processing facility finds the first feature set matches a second feature set;

a message sent to one or more interested recipients when the processing facility finds that message trigger criteria are met;

wherein message trigger criteria include a requirement that a matched case is discovered;

a user preference service; and

,

wherein the first site is a hotel and the camera Cx is located in a hotel reception area Zx, first and second messages are sent when a matched case includes a case group that identifies the person as an expected guest, the first message content includes identification of the guest by name and image and a recipient interested in the first message is a hotel staff member, and the second message content includes a controller instruction for adjusting operation of an appliance located in the hotel room of the recognized guest.

10. A biometric notification system (“

BNS”

) with facial recognition and related services, the system comprising;

plural digital video cameras (C₁, C₂ . . . C_n) that monitor respective zones (Z₁, Z₂ . . . Z_n) at a first site;

a data processing system including a processing facility and a database facility;

a network interconnecting the video cameras and the data processing facility;

a video frame acquired from a video camera C_x in zone Z_x includes a first content;

a first facial feature set derived from the first content;

a database of the database facility includes plural cases, each case identifies a person by linking a name with a second content, a second facial feature set, and a case group;

a matched case discovered when the processing facility finds the first feature set matches a second feature set;

a message sent to one or more interested recipients when the processing facility finds that message trigger criteria are met;

wherein message trigger criteria include a requirement that a matched case is discovered;

an archive memory with plural archive folders; and

,

a service for building an archive in the archive memory;

wherein (i) each archive folder receives sequential video frames from a respective video camera Cx in zone Zx and (ii) for each video frame including a facial image, a second facial feature set derived from the facial image is associated with the video frame and a commonly referenced date time and stamp is associated with the video frame.

11. The BNS of claim 10 further comprising:

a particular person of interest;

wherein a facial feature set of the particular person of interest is compared with the facial feature sets of archive folders and matched folders are identified;

wherein the video frames of the matched folders are arranged for playback in chronological order.

12. The BNS of claim 11 further comprising:

a first automated video editing feature;

wherein if the sequence of frames from a first camera to see the person of interest overlaps chronologically with a sequence of frames from a second camera to see the person of interest, then sufficient frames are removed from the overlap to eliminate the overlap.

13. The BNS of claim 12 wherein the first and second camera frames that are removed result in a splice of the first camera frames to the second camera frames at an approximate overlap midpoint.

14. The BNS of claim 12 further comprising:

proximity sensing for determining a distance between the camera and the person of interest; and

,

a second automated video editing feature;

wherein if the initial frame of the first camera frame sequence and the initial frame of the second camera frame sequence are temporally coincident, then the frame sequences are spliced to provide playback of frames of the camera closest to the person of interest before playback of frames of the other camera.