Method and apparatus maintaining eye contact in video delivery systems using view morphing

• US 20040196360A1
• Filed: 10/22/2003
• Published: 10/07/2004
• Est. Priority Date: 11/29/2000
• Status: Active Grant

• Current Assignee: Applied Invention, LLC
• Original Assignee: Applied Minds, Inc.
• Inventors: Hillis, W. Daniel, Ferren, Bran, Howe, Russel
• Granted Patent: US 7,202,887 B2
• Time in Patent Office: Days
• US Class Current: 348/14.16
• CPC Class Codes: H04N 7/144 camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact
• CPC Class Codes: H04N 7/15 Conference systems

Abstract

A view morphing algorithm is applied to synchronous collections of video images from at least two video imaging devices, and interpolating between the images, creates a composite image view of the local participant. This composite image approximates what might be seen from a point between the video imaging devices, presenting the image to other video session participants.

Claims

1. An apparatus for presenting a composite image conveying eye contact with a conference participant, comprising:

a composite image generator communicatively coupled with at least two members of an imaging device collection, for repeatedly receiving an image from each image device in said collection at approximately a same time to provide a synchronized image collection of said images based upon observations of said participant by said at least two members of said imaging device collection;

wherein said composite image generator is communicatively coupled to a motion video portal for providing a succession of said composite images based upon at least said synchronized image collection for presentation to a video delivery system;

wherein said video delivery system presents a second participant with a motion video stream conveying eye contact generated by said motion video portal based upon said composite image succession.

2. An apparatus that conveys eye contact with a conference participant, comprising:

means for generating a composite image of said participant receiving at least two images of said participant;

wherein each image is comprised of a two-dimensional array of pixels;

wherein said means for generating said composite image is comprised of;

means for calculating at least one dense correspondence to determine a displacement in at least a first dimension for each of said pixels in at least one first of said images;

means for generating an interpolated image based upon said at least one dense correspondence for each of said at least two images.

3. The apparatus of claim 2, further comprising:

means for combining at least two of said interpolated images by using an averaging scheme to create said composite image.

4. The apparatus of claim 2, wherein said means for generating said interpolated image further comprises:

means for displacing each of said pixels in said image pixel arrays by a partial displacement in at least a first dimension.

5. The apparatus of claim 3, wherein said means for combining said interpolated images, is further comprised of:

means for combining corresponding pixels in said at least a second dimension to create said composite image pixel;

wherein a sum of said partial displacements of said images is approximately equal to said displacement.

6. The apparatus of claim 2, further comprising:

means for applying a rectifying transformation to said image.

7. The apparatus of claim 2, further comprising:

means for warping said image by said partial displacement to modify said image.