Face Annotation In Streaming Video

• US 20080235724A1
• Filed: 09/19/2006
• Published: 09/25/2008
• Est. Priority Date: 09/30/2005
• Status: Abandoned Application

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Benien, Christian, Kneser, Reinhard, Sassenscheidt, Frank
• Application Number:
  US12/088,001
• Publication Number:
  US 20080235724A1
• Time in Patent Office: Days
• US Class Current: 725/32
• CPC Class Codes: G06F 16/784 the detected or recognised objects being people

Abstract

The invention relates to a system (5, 15) and a method for detecting and annotating faces on-the-fly in video data. The annotation (29) is performed by modifying the pixel content of the video and is thereby independent of file types, protocols and standards. The invention can also perform real-time face-recognition by comparing detected faces with known faces from storage, so that the annotation can contain personal information (38) relating to the face. The invention can be applied at either end of a transmission channel and is particularly applicable in videoconferences, Internet classrooms, etc.

Claims

1. A system (5,15) for real-time face-annotating of streaming video, the system comprising:

a streaming video source (1);

a face-detection component (10) operable connected to receive streaming video (4) from the streaming video source and being configured to perform a real-time detection of regions holding candidate faces in the streaming video;

an annotator (14) being operable connected to receive;

the streaming video;

locations of candidate face regions from the face-detection component;

the annotator being configured to modify pixel content in the streaming video related to at least one candidate face region;

an output (17) being operable connected to receive the face-annotated streaming video (18) from the annotator.

2. The system according to claim 1, wherein:

the streaming video source (1) is configured to provide un-compressed streaming video comprising image frames; and

the face-detection component (10) is further configured to perform detection only on selected image frames of the streaming video.

3. The system according to claim 1, further comprising

a storage (23, 24) holding data identifying one or more faces and related annotation information;

and

a face-recognition component (12) operable connected to receive candidate face regions from the face-detection component (10) and access the storage, and being configured to perform a real-time identification of candidate faces in the storage,

and wherein

the annotator (14) is further operable connected to receive

information that a candidate face has been identified, and

annotation information for any identified candidate faces from either of the face-recognition component or the storage; and

the annotator is further configured to include annotation information in relation to identified candidate faces in the modulation of pixel content in the streaming video.

4. The system according to claim 1, wherein the streaming video source (1) comprises a digital video camera (28) for recording a digital video and generating the streaming video.

5. The system according to claim 1, wherein the output (17) comprises an encoder and a transmitter for encoding and transmitting the face-annotated streaming video.

6. The system according to claim 1, wherein the output (17) comprises a display (36) operable connected to receive the face-annotated streaming video from the output terminal and display it to an end user.

7. The system according to claim 1, wherein the streaming video source (1) comprises a receiver and a decoder for receiving and decoding a streaming video.

8. A method for making face-annotation of streaming video, the method comprising the steps of:

receiving streaming video;

performing a real-time face-detection procedure to detect regions holding candidate faces in the streaming video; and

annotating the streaming video by modifying pixel content in the streaming video related to at least one candidate face region.

9. The method of claim 8, further comprising the steps of

providing data identifying one or more faces;

performing a real-time face-recognition procedure to perform a real-time identification of candidate faces in the data; and

including annotation information related to identified candidate faces in the modulation of pixel content in the streaming video.

10. The method of claim 8, wherein the streaming video comprises un-compressed streaming video consisting of image frames, and wherein the face-detection procedure is performed only on selected image frames of the streaming video.