SYSTEMS AND METHODS FOR PROVIDING PERSONAL VIDEO SERVICES

• US 20100073458A1
• Filed: 01/04/2008
• Published: 03/25/2010
• Est. Priority Date: 01/23/2007
• Status: Active Grant

• Current Assignee: Euclid Discoveries LLC
• Original Assignee: Euclid Discoveries LLC
• Inventors: Pace, Charles P.
• Granted Patent: US 8,243,118 B2
• Time in Patent Office: Days
• US Class Current: 348/14.160
• CPC Class Codes: G06V 10/7557 based on appearance, e.g. active appearance models [AAM]
• CPC Class Codes: G06V 40/167 using comparisons between temporally consecutive images
• CPC Class Codes: H04N 21/23412 for generating or manipulating the scene composition of objects, e.g. MPEG-4 objects
• CPC Class Codes: H04N 21/4223 Cameras H04N23/00 takes precedence
• CPC Class Codes: H04N 21/44012 involving rendering scenes according to scene graphs, e.g. MPEG-4 scene graphs
• CPC Class Codes: H04N 21/4415 using biometric characteristics of the user, e.g. by voice recognition or fingerprint scanning cryptography using biological data H04L9/3231; authentication in networks using biometric H04L63/0861
• CPC Class Codes: H04N 21/4532 involving end-user characteristics, e.g. viewer profile, preferences monitoring of user activities for profile generation for accessing a video database G06F16/739; user profiles in network data switching protocols H04L67/306; processing of user preferences or user profiles in wireless networks H04W8/18
• CPC Class Codes: H04N 21/4788 communicating with other users, e.g. chatting arrangements for providing for computer conferences, e.g. chat rooms, to substation in data switching networks H04L12/1813; distributed application using peer-to-peer [P2P] networks H04L67/104
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/157 defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827

Abstract

Systems and methods for processing video are provided. Video compression schemes are provided to reduce the number of bits required to store and transmit digital media in video conferencing or videoblogging applications. A photorealistic avatar representation of a video conference participant is created. The avatar representation can be based on portions of a video stream that depict the conference participant. A face detector is used to identify, track and classify the face. Object models including density, structure, deformation, appearance and illumination models are created based on the detected face. An object based video compression algorithm, which uses machine learning face detection techniques, creates the photorealistic avatar representation from parameters derived from the density, structure, deformation, appearance and illumination models.

Claims

1. A method of video conferencing, the method comprising the steps of:

detecting a human face of a video conference participant depicted in portions of a video stream;

creating one or more object models to model the face of the video conference participant; and

using the object models, creating a photorealistic avatar representation of the video conference participant.

2. A method for providing video conferencing as in claim 1 wherein the face of the video conference participant is detected and tracked using a Viola/Jones face detection algorithm.

3. A method for providing video conferencing as in claim 1 wherein the photorealistic avatar representation object models are created as an implicit representation of the face of the video conference participant.

4. A method for providing video conferencing as in claim 3 wherein the implicit representation of the video conference participant is a simulated representation of the face of the video conference participant.

5. A method for providing video conferencing as in claim 3 wherein the detecting and tracking comprise using a Viola/Jones face detection algorithm further includes the steps of:

identifying corresponding elements of at least one object associated with the face in two or more video frames from the video stream; and

tracking and classifying the corresponding elements to identify relationships between the corresponding elements based on previously calibrated and modeled faces.

6. A method for providing video conferencing as in claim 1 wherein the object models include object models for structure, deformation, pose, motion, illumination, and appearance.

7. A video conferencing system comprising:

a face detector detecting a face of a video conference participant in a video stream;

a calibrator generating a calibration model calibrating the face of the video conference participant;

object models, in combination with the calibrator and face detector, the object models modeling portions of the video stream depicting the face of the video conference participant based on the calibration model; and

a photorealistic avatar representation of the video conference participant, the photorealistic avatar representation generated from the face detector, calibrator and object models.

8. A system for video conferencing comprising:

means for providing object models to model portions of a video stream depicting at least one participant of a video conference; and

means for using the object models to create a photorealistic avatar representation of the video conference participant.