System and method for applying digital make-up in video conferencing

• US 20060268101A1
• Filed: 05/25/2005
• Published: 11/30/2006
• Est. Priority Date: 05/25/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Rui, Yong, Cohen, Michael, He, Li-wei, Manaka, Shinichi
• Granted Patent: US 7,612,794 B2
• Time in Patent Office: Days
• US Class Current: 348/14.120
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q

Abstract

A method of digitally adding the appearance of makeup to a videoconferencing participant. The system and method for applying digital make-up operates in a loop processing sequential video frames. For each input frame, there are typically three general steps: 1) Locating the face and eye and mouth regions; 2) Applying digital make-up to the face, preferably with the exception of the eye and open mouth areas; and 3) Blending the make-up region with the rest of the face. In one embodiment of the invention, the background in the frame containing a video conferencing participant can also be modified so that other video conferencing participants cannot clearly see the background behind the participant in the image frame. In one such embodiment of the invention, the video conferencing participant tries to make his or her own image look comical or altered. In another embodiment of the invention, a particular remote participant tries to make another participant look funny to the other participants.

Claims

1. A process for applying digital make-up to an image frame used for face to face video conferencing, comprising the process actions of:

inputting a video frame;

processing said video frame to locate a face; and

applying digital make-up to said face to obtain a made-up image of said face.

2. The process of claim 1 wherein processing said video frame to locate a face, further comprises the process actions of:

separating the face from the background of the frame; and

modifying said background.

3. The process of claim 2 wherein modifying said background comprises one of:

blurring the background;

modifying the background so that it is no longer visible;

deleting the background;

replacing the background with a different background; and

making the background all one pixel color.

4. The process of claim 1 further comprising the process actions of:

encoding said made-up image of said face; and

transmitting said made-up image of said face to other video conferencing participants.

5. The process of claim 1 wherein the face is located by using at least one of:

a face detector,

edge detection, and

motion detection.

6. The process of claim 1 further comprising the process action of finding the features of the face prior to applying the digital make-up.

7. The process of claim 6 wherein the features of the face are the eyes and mouth and wherein these features are found by using at least one of:

knowledge of face proportions;

template matching;

edge detection; and

corner detection.

8. The process of claim 6 wherein the process action of applying digital make-up comprises the process actions of:

applying a low pass filter to the face with the exception of the eyes and any open mouth that is shown.

9. The process of claim 1 wherein applying digital make-up comprises applying a low pass filter to the face.

10. The process of claim 1 wherein the process action of applying digital make-up comprises the process actions of:

applying a bilateral filter to the face.

11. The process of claim 1 wherein applying digital make-up comprises applying a contrast reduction filter to the face.

12. The process of claim 1 wherein the process action of applying digital make-up comprises the process actions of:

changing the color of some pixels of given areas of the face to enhance their color.

13. The process of claim 1 wherein the process actions are performed by a server who receives said input video frame from a video conferencing participant.

14. The process of claim 13 further comprising the process actions of:

the server encoding said made-up image of said face; and

the server transmitting said made-up image of said face to other video conferencing participants who display it.

15. The process of claim 1 wherein the input video frame is received at a meeting participant from another meeting participant.

16. A computer-readable medium having computer-executable instructions for performing the process recited in claim 1.

17. A system for transforming a face in an image frame used for face-to-face video conferencing, comprising

a general purpose computing device;

and

a computer program comprising program modules executable by the computing device, wherein the computing device is directed by the program modules of the computer program to,

input a video frame;

process said video frame to identify a face, the area outside of the face comprising the background;

find the features of the face to identify areas that are not to be transformed;

apply a transformation to the features of the face that are not in the areas that are not to be transformed; and

blend the areas that are transformed with the areas that are not transformed.

18. The system of claim 17 wherein the module for applying the transformation applies digital make-up.

19. The system of claim 17 wherein the module for applying the transformation adds objects to the face.

20. A method for transforming an image of a video conference participant, comprising the process actions of:

inputting a video frame at a first video conferencing participant;

said first video conferencing participant locating a face and its features within the video frame;

said first video conferencing participant determining a transformation to be applied to said face;

said first video conferencing participant sending information as to the location of the face and the features as well as the transformation to be applied to one or more video conference participants; and

said one or more video conference participants applying said transformation to said video frame.