Method of Maintaining Eye Contact in Video Conferencing Using View Morphing

• US 20070159523A1
• Filed: 11/30/2006
• Published: 07/12/2007
• Est. Priority Date: 11/29/2000
• Status: Active Grant

• Current Assignee: Applied Invention, LLC
• Original Assignee: Applied Minds, Inc.
• Inventors: Ferren, Bran, Howe, Russel, Hillis, W.
• Granted Patent: US 7,773,108 B2
• Time in Patent Office: Days
• US Class Current: 348/14.80
• CPC Class Codes: H04N 7/144 camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

A view morphing algorithm is applied to synchronous collections of video images from at least two video imaging devices, and interpolating between the images, creates a composite image view of the local participant. This composite image approximates what might be seen from a point between the video imaging devices, presenting the image to other video session participants.

Claims

1. A method of providing eye contact with a first video conferencing participant comprising the steps of:

receiving at least two source images of said first participant;

generating a composite image of said first participant; and

providing said composite image to a second video conferencing participant;

wherein each of said source images is comprised of an array of source image pixels; and

wherein said step of generating said composite image comprises the steps of

calculating a dense correspondence for at least one pair of images among said source images, said dense correspondence specifying a pixel displacement for each of said source image pixels within each image within said pair of images,

generating at least two interpolated images from said source images based upon said pixel displacement, and

combining at least two of said interpolated images using an averaging scheme to create said composite image.

2. The method of claim 1, said step of generating at least two interpolated images comprising displacing each of said source image pixels within at least two of said source images by a fraction of said pixel displacement.

3. The method of claim 2, wherein said fraction is one half.

4. The method of claim 2, wherein said source images are acquired from at least two imaging devices, and said fraction is based upon the positions of said image devices relative to a location of said second participant on a display within view of said first participant.

5. The method of claim 1,

wherein said composite image comprises an array of composite image pixels;

wherein each of said interpolated images comprises an array of interpolated image pixels; and

wherein said averaging scheme comprises calculating values for aid composite image pixels based on a weighted average of corresponding pixels among said interpolated image pixels.

6. The method of claim 5, wherein said weighted average is based upon the location of said corresponding pixels within said interpolated images.

7. The method of claim 1, said step of generating a composite image additionally comprising the step of, before said calculating a dense correspondence step:

applying a rectifying transformation to at least one of said source images.

8. The method of claim 1,

wherein said source images are acquired from a time series of source images, and

said step of generating a composite image additionally comprising the step of, before said calculating a dense correspondence step:

applying a warping transformation to at least one of said source images to displace each of said source image pixels by a fraction of a prior pixel displacement determined from a prior dense correspondence computed for source images prior to said source images within said time series of source images.

9. The method of claim 8, wherein said fraction is less than one.

10. The method of claim 1, wherein said pixels are characterized by color values within a perceptually uniform color space.

11. The method of claim 10, wherein said color space is L*a*b* CIE.