Maintaining eye-contact in teleconferencing using structured light

• US 6,714,234 B1
• Filed: 04/11/2002
• Issued: 03/30/2004
• Est. Priority Date: 04/11/2001
• Status: Active Grant

• Current Assignee: Applied Invention, LLC
• Original Assignee: Applied Minds, Inc.
• Inventors: Hillis, W. Daniel, Ferren, Bran
• Primary Examiner(s): CHAN, WING F
• Application Number:
  US10/121,562
• Time in Patent Office: 719 Days
• Field of Search: 348/14.01, 348/14.07, 348/14.08, 348/14-2-
• US Class Current: 348/14.16
• CPC Class Codes: H04N 7/144 camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact

Abstract

The invention comprises a structured light projector, a video camera, and an image processor, for achieving perspective corrected images that enhance eye-contact during teleconferencing. A structured light projector is offset in one direction from the monitor center, and illuminates a local participant with a structured light pattern. The image of the local participant, illuminated by both ambient and structured light, is captured by the video camera, also offset from the monitor center, preferably in the direction opposite the structured light projector. By considering the distortion of the structured light observed from the position of the video camera and the position of the structured light projector and video camera relative to the monitor center, an image processor creates an image of the local participant as viewed from a perspective that, when viewed by the remote participant, provides a sense of eye contact with the local participant.

Claims

1. A method for providing eye contact in a video conference, comprising the steps of:

illuminating a local participant with structured light from a first position;

obtaining an image of said local participant from a second position;

determining from distortion of said structured light illumination a requisite warping value for producing a head on view of said local participant; and

applying said requisite warping to said image to yield a final image;

wherein said final image provides a head on view of said local participant when presented to a remote participant.

2. The method of claim 1, wherein said structured light illumination is in the infrared spectrum.

3. The method of claim 1, wherein said structured light illumination is only active during a vertical blanking interval of a video camera used in obtaining said image of said local participant.

4. The method of claim 1, wherein said determining step further comprises the step of:

applying a high pass convolution filter.

5. The method of claim 1, wherein said structured light comprises a series of substantially parallel lines.

6. The method of claim 5, wherein said determining step further comprises the step of:

applying at least one directional filter.

7. The method of claim 6, wherein said determining step further comprises the step of:

integrating a series of directional values along a line substantially parallel to said lines of structured light.

8. The method of claim 7, wherein limits of said integrating step are defined by a mask surrounding said local participant.

9. The method of claim 1, wherein said requisite warping value is applied using an image warping routine.

10. An apparatus for providing eye contact in a video conference, comprising:

a source of structured light for illuminating a local participant with structured light from a first position;

an imaging device for obtaining an image of said local participant from a second position;

a processor for determining from distortion of said structured light illumination a requisite warping value for producing a head on view of said local participant; and

for applying said requisite warping to said image to yield a final image;

wherein said final image, provides a head on view of said local participant when presented to a remote participant.

11. The apparatus of claim 10, wherein said structured light illumination is in the infrared spectrum.

12. The apparatus of claim 10, wherein said structured light illumination is only active during a vertical blanking interval of a video camera used in obtaining said image of said local participant.

13. The apparatus of claim 10, wherein said processor further comprises:

a high pass convolution filter.

14. The apparatus of claim 10, wherein said structured light comprises a series of substantially parallel lines.

15. The method of claim 14, wherein said processor further comprises:

at least one directional filter.

16. The apparatus of claim 15, wherein said processor further comprises:

means for integrating a series of directional values along a line substantially parallel to said lines of structured light.

17. The apparatus of claim 16, wherein limits of said integrating means are defined by a mask surrounding said local participant.

18. The apparatus of claim 10, wherein said requisite warping value is applied using an image warping routine.