BIOMETRIC NOTIFICATION SYSTEM

• US 20170323151A1
• Filed: 07/16/2017
• Published: 11/09/2017
• Est. Priority Date: 07/21/2008
• Status: Active Grant

• Current Assignee: FaceFirst, LLC
• Original Assignee: FaceFirst, LLC
• Inventors: Rosenkrantz, Joseph Ethan, Hesketh, Gifford
• Granted Patent: US 10,043,060 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/51 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/739 in form of a video summary, e.g. the video summary being a video sequence, a composite still image or having synthesized frames
• CPC Class Codes: G06F 16/784 the detected or recognised objects being people
• CPC Class Codes: G06V 10/95 structured as a network, e.g. client-server architectures
• CPC Class Codes: G06V 20/41 Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items segmenting video sequences G06V20/49
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G06V 40/172 Classification, e.g. identification
• CPC Class Codes: G08B 21/18 Status alarms G08B21/02 takes precedence
• CPC Class Codes: G11B 27/031 Electronic editing of digitised analogue information signals, e.g. audio or video signals
• CPC Class Codes: G11B 27/3081 used signal is a video-frame or a video-field (P.I.P)
• CPC Class Codes: H04N 5/77 between a recording apparatus and a television camera
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

The present invention provides a biometric notification system for selectively sending messages to interested recipients. In various embodiments, message trigger criteria, interested recipients, and message content may vary depending upon, among other things, the service being provided.

Claims

1. A biometric notification system (“

BNS”

) with facial recognition and related services, the system comprising;

plural digital video cameras (C₁ . . . C_n) that monitor respective zones (Z₁ . . . Z_n) at a first site;

a processing system including a processing facility and a database facility;

a network interconnecting the video cameras and the data processing facility;

a video frame acquired from a video camera C_x in zone Z_x includes a first content and a first facial feature set derived from the first content;

a database of the database facility includes plural cases, each case identifies a person by linking a name with a second facial feature set and a case group; and

,

a matched case discovered when the processing facility finds the first feature set matches a second feature set;

wherein message trigger criteria include a requirement that a matched case is discovered.

2. The BNS of claim 1 further comprising:

a user preference service wherein the site is a building and a camera C_x is located in a building reception area Z_x;

first and second messages are sent when a matched case includes a case group that identifies the person as an expected guest;

the first message content includes identification of the guest by name and image and a recipient of the first message is a security staff member; and

,

the second message content includes a controller instruction for adjusting operation of an appliance that attends to the guest.

3. The BNS of claim 1 further comprising:

a first geofencing service for recognizing a person excluded from a zone;

wherein only a subset of the cases in the database is available for comparison with the first feature set, the subset of cases consisting of persons excluded from zone Z_x;

wherein discovery of a matched case results in a message being sent to site security, the message advising the occurrence of a security breach and the name, image, and zone of the person of the matched case.

4. The BNS of claim 1 further comprising:

a second geofencing service for recognizing a person not excluded from a zone;

wherein only a subset of cases in the database is available for comparison with a first feature set, the subset of cases consisting of persons not excluded from zone Z_x;

wherein failure to discover a matched case results in a message being sent to site security, the message advising the occurrence of a security breach and the image and zone of the person that could not be matched.

5. The BNS of claim 1 further comprising:

a first intelligent video storage service;

wherein a sequential stream of video frames from a match camera C_x is received by a circular memory buffer and the buffer contents B_x are saved in video storage memory when a second facial feature set is matched with a first facial feature set derived from a target frame in the circular memory buffer;

wherein an interested recipient that receives a message triggered by the match receives with the message a hyperlink for viewing the buffer contents saved in video memory storage.

6. The BNS of claim 5 wherein at the time the match is completed, the circular memory buffer holds x frames acquired before the target frame (“

pre-roll frames”

) and y frames acquired after the target frame (“

post-roll frames”

), each of the x frames and the y frames acquired during a respective acquisition time period t1, t2 in excess of a time period t3 required to compare the first facial feature set of the target frame with all of the cases in the database.

7. The BNS of claim 5 wherein the circular buffer size varies with the number of second facial feature sets available for comparison with the first facial feature set of the target frame.

8. The BNS of claim 5 wherein plural circular memory buffers receive sequential video streams from respective cameras that monitor zones adjacent to Z_x and these buffer contents are saved in video storage memory when the buffer contents B_x are saved.

9. The BNS of claim 1 further comprising:

an archive memory with plural archive folders; and

,

a service for building an archive in the archive memory;

wherein (i) each archive folder receives sequential video frames from a respective video camera C_x in zone Z_x and (ii) for each video frame including a facial image, a second facial feature set derived from the facial image is associated with the video frame and a commonly referenced date time and stamp is associated with the video frame.

10. The BNS of claim 9 further comprising:

a particular person of interest;

wherein a facial feature set of the particular person of interest is compared with the facial feature sets of archive folders and matched folders are identified;

wherein the video frames of the matched folders are arranged for playback in chronological order.

11. The BNS of claim 10 further comprising:

a first automated video editing feature;

wherein if a sequence of frames from a first camera to see the person of interest overlaps chronologically with a sequence of frames from a second camera to see the person of interest, then sufficient frames are removed from the overlap to eliminate the overlap.

12. The BNS of claim 11 wherein the first and second camera frames that are removed result in a splice of the first camera frames to the second camera frames at an approximate overlap midpoint.

13. The BNS of claim 11 further comprising:

proximity sensing for determining a distance between the camera and the person of interest; and

,

a second automated video editing feature;

wherein if the initial frame of the first camera frame sequence and the initial frame of the second camera frame sequence are temporally coincident, then the frame sequences are spliced to provide playback of frames of the camera closest to the person of interest before playback of frames of the other camera.

14. A biometric notification method with facial recognition and related services, the method comprising the steps of:

providing plural digital video cameras (C₁ . . . C_n) for monitoring respective zones (Z₁ . . . Z_n) at a first site, a processing system including a processing facility and a database facility, and a network interconnecting the video cameras and the data processing facility;

acquiring a video frame from a video camera C_x in zone Z_x including a first content and a first facial feature set derived from the first content;

in a database of a database facility including plural cases, identifying each case with a name that is linked to a second facial feature set and a case group; and

,

discovering a matched case if the processing facility finds the first feature set matches a second feature set, message trigger criteria including a requirement that a matched case be discovered.

15. The method of claim 14 further comprising the steps of:

providing a user preference service wherein the site is a building and a camera C_x is located in a building reception area Z_x;

sending first and second messages when a matched case includes a case group that identifies the person as an expected guest;

wherein the first message content includes identification of the guest by name and image and a recipient of the first message is a security staff member;

wherein the second message content includes a controller instruction for adjusting operation of an appliance that attends to the guest.

16. The method of claim 14 further comprising the steps of:

providing a first geofencing service for recognizing a person excluded from a zone;

wherein only a subset of the cases in the database is available for comparison with the first feature set, the subset of cases consisting of persons excluded from zone Z_x;

wherein discovery of a matched case results in a message being sent to site security, the message advising the occurrence of a security breach and the name, image, and zone of the person of the matched case.

17. The method of claim 14 further comprising the steps of:

providing a second geofencing service for recognizing a person not excluded from a zone;

wherein only a subset of cases in the database is available for comparison with a first feature set, the subset of cases consisting of persons not excluded from zone Z_x;

wherein failure to discover a matched case results in a message being sent to site security, the message advising the occurrence of a security breach and the image and zone of the person that could not be matched.

18. The method of claim 14 further comprising the steps of:

providing a first intelligent video storage service;

wherein a sequential stream of video frames from a match camera C_x is received by a circular memory buffer and the buffer contents B_x are saved in video storage memory when a second facial feature set is matched with a first facial feature set derived from a target frame in the circular memory buffer;

wherein an interested recipient that receives a message triggered by the match receives with the message a hyperlink for viewing the buffer contents saved in video memory storage.

19. The method of claim 14 wherein at the time the match is completed, the circular memory buffer holds x frames acquired before the target frame (“

pre-roll frames”

) and y frames acquired after the target frame (“

post-roll frames”

), each of the x frames and the y frames acquired during a respective acquisition time period t1, t2 in excess of a time period t3 required to compare the first facial feature set of the target frame with all of the cases in the database.

20. The method of claim 14 wherein the circular buffer size varies with the number of second facial feature sets available for comparison with the first facial feature set of the target frame.

21. The method of claim 14 wherein plural circular memory buffers receive sequential video streams from respective cameras that monitor zones adjacent to Z_x and these buffer contents are saved in video storage memory when the buffer contents B_x are saved.

22. The method of claim 14 further comprising the steps of:

providing an archive memory with plural archive folders; and

,

providing a service for building an archive in the archive memory;

wherein (i) each archive folder receives sequential video frames from a respective video camera C_x in zone Z_x and (ii) for each video frame including a facial image, a second facial feature set derived from the facial image is associated with the video frame and a commonly referenced date time and stamp is associated with the video frame.

27. The method of claim 14 further comprising the steps of:

providing proximity sensing for determining a distance between the camera and the person of interest; and

,

providing a second automated video editing feature;

wherein if the initial frame of the first camera frame sequence and the initial frame of the second camera frame sequence are temporally coincident, then the frame sequences are spliced to provide playback of frames of the camera closest to the person of interest before playback of frames of the other camera.

24. The method of claim 23 wherein for a particular person of interest, a facial feature set of the person is compared with the facial feature sets of archive folders and matched folders are identified and the video frames of the matched folders are arranged for playback in chronological order.

25. The method of claim 23 further comprising the steps of:

providing a first automated video editing feature;

wherein if a sequence of frames from a first camera to see the person of interest overlaps chronologically with a sequence of frames from a second camera to see the person of interest, then sufficient frames are removed from the overlap to eliminate the overlap.

26. The method of claim 25 wherein the first and second camera frames that are removed result in a splice of the first camera frames to the second camera frames at an approximate overlap midpoint.

28. A biometric notification system (“

BNS”

) with facial recognition and related services, the system comprising;

plural digital video cameras (C₁ . . . C_n) that monitor respective zones (Z₁ . . . Z_n);

a processing system including a processing facility and a database facility;

a network interconnecting the video cameras and the data processing facility;

a database of the database facility includes plural cases, each case identifies a person by linking a name with a case facial feature set and a case group;

a person of interest selected from the database of cases;

for each camera, collecting video frames that include a facial image that matches the case facial feature set and associating each such frame with a date and time; and

,

assembling the collected video frames such that the frames are presented in chronological order;

wherein the only video frames subjected to the matching process are video frames that correspond to the case group of the selected person.