SELF LEARNING FACE RECOGNITION USING DEPTH BASED TRACKING FOR DATABASE GENERATION AND UPDATE
First Claim
1. A computer-implemented process for generating a face recognition training database for each person detected as being located in an environment, comprising:
- using a computer to perform the following process actions;
(a) inputting a sequence of contemporaneously-captured frame pairs, each frame pair comprising a frame output from a color video camera and a frame output from a depth video camera;
(b) using a face detection method and the color video camera frames to detect potential persons in a environment;
(c) using a motion detection method and the depth video camera frames to detect potential persons in the environment;
(d) using detection results generated via the face detection method and motion detection method to determine the location of one or more persons in the environment, said detection results generated via the face detection method comprising for each person detected, a facial characterization of the portion of a color video camera frame depicting the person'"'"'s face;
(e) for each person detected solely via the motion detection method,identifying the corresponding location of that person in the contemporaneously-captured frame of the color video camera,generating said facial characterization of the portion of the color video camera frame depicting that person'"'"'s face;
(f) for each person detected in the environment,assigning each facial characterization generated for that person to an unknown person identifier established for the person,storing each of said facial characterizations in a memory associated with the computer,attempting to ascertain the identity of the person, andwhenever the identity of the person is ascertained, re-assigning each facial characterization assigned to the unknown person identifier established for the person to a face recognition training database established for that person.
3 Assignments
0 Petitions
Accused Products
Abstract
Face recognition training database generation technique embodiments are presented that generally involve collecting characterizations of a person'"'"'s face that are captured over time and as the person moves through an environment, to create a training database of facial characterizations for that person. As the facial characterizations are captured over time, they are will represent the person'"'"'s face as viewed from various angles and distances, different resolutions, and under different environmental conditions (e.g., lighting and haze conditions). Further, over a long period of time where facial characterizations of a person are collected periodically, these characterizations can represent an evolution in the appearance of the person. This produces a rich training resource for use in face recognition systems. In addition, since a person'"'"'s face recognition training database can be established before it is needed by a face recognition system, once employed, the training will be quicker.
-
Citations
20 Claims
-
1. A computer-implemented process for generating a face recognition training database for each person detected as being located in an environment, comprising:
using a computer to perform the following process actions; (a) inputting a sequence of contemporaneously-captured frame pairs, each frame pair comprising a frame output from a color video camera and a frame output from a depth video camera; (b) using a face detection method and the color video camera frames to detect potential persons in a environment; (c) using a motion detection method and the depth video camera frames to detect potential persons in the environment; (d) using detection results generated via the face detection method and motion detection method to determine the location of one or more persons in the environment, said detection results generated via the face detection method comprising for each person detected, a facial characterization of the portion of a color video camera frame depicting the person'"'"'s face; (e) for each person detected solely via the motion detection method, identifying the corresponding location of that person in the contemporaneously-captured frame of the color video camera, generating said facial characterization of the portion of the color video camera frame depicting that person'"'"'s face; (f) for each person detected in the environment, assigning each facial characterization generated for that person to an unknown person identifier established for the person, storing each of said facial characterizations in a memory associated with the computer, attempting to ascertain the identity of the person, and whenever the identity of the person is ascertained, re-assigning each facial characterization assigned to the unknown person identifier established for the person to a face recognition training database established for that person. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18)
-
15. A computer-implemented process for generating a face recognition training database for each person detected as being located in an environment, comprising:
using a computer to perform the following process actions; (a) inputting a sequence of frames output from a color video camera; (b) using a face detection method and the color video camera frames to detect potential persons in a environment; (c) using a motion detection method and the color video camera frames to detect potential persons in a environment; (d) using detection results generated via the face detection method and motion detection method to determine the location of one or more persons in the environment, said detection results generated via the face detection method comprising for each person detected, a facial characterization of the portion of a color video camera frame depicting the person'"'"'s face; (e) for each person detected solely via the motion detection method, locating a portion of a color video camera frame depicting that person'"'"'s face, and generating said facial characterization of the portion of the color video camera frame depicting that person'"'"'s face; (f) for each person detected in the environment, assigning each facial characterization generated for that person to an unknown person identifier established for the person, storing each of said facial characterizations in a memory associated with the computer, attempting to ascertain the identity of the person, and whenever the identity of the person is ascertained, re-assigning each facial characterization assigned to the unknown person identifier established for the person to a face recognition training database established for that person.
-
19. A computer-implemented process for detecting persons located in an environment, comprising:
-
using a computer to perform the following process actions; inputting a sequence of frames output from a depth video camera; designating all the pixels in the first depth video camera frame as background pixels; for each pixel of each of the subsequently-captured depth frames contained in the sequence of frames, in the order in which the frame was captured; identifying if the depth value of the pixel has changed more than a prescribed amount from the value of a pixel in the depth frame captured immediately before the frame currently under consideration that represents the same location within the environment; whenever the depth value of the pixel has changed more than the prescribed amount, designating the pixel to be a foreground pixel; once the last frame contained in the sequence of frames has been processed to identify if its pixel depth values have changed more than the prescribed amount, (i) establishing a seed point amongst the foreground pixels in said last frame and assigning the pixel associated therewith to be a part of a separate blob, (ii) recursively determining for each pixel neighboring a pixel assigned to the blob, which is not already assigned to that blob, if its depth value is the same within a prescribed tolerance as the current average of the pixels assigned to the blob, and if so, assigning that neighboring pixel to be a part of the blob, until no neighboring pixel can be found that is unassigned to a blob and which has a depth value that is the same within said prescribed tolerance of the current average of the pixels assigned to the blob, and (iii) whenever a neighboring pixel is found during the performance of the recursive determining action (ii) that is assigned to a different blob, combining the two blobs into one and continuing the recursive determining action (ii), and (iv) repeating process actions (i) through (iii) for the unassigned foreground pixels, until no more blobs can be formed, once no more blobs can be formed, for each blob, determining if the blob meets a set of prescribed criteria that is indicative of the blob representing a human, eliminating each blob not meeting the set of prescribed criteria, and designating each remaining blob to represent a different potential person located within the environment. - View Dependent Claims (20)
-
Specification