Maintaining eye-contact in teleconferencing using structured light

• US 6,906,742 B1
• Filed: 12/09/2003
• Issued: 06/14/2005
• Est. Priority Date: 04/11/2001
• Status: Active Grant

• Current Assignee: Applied Invention, LLC
• Original Assignee: Applied Minds, Inc.
• Inventors: Hillis, W. Daniel, Ferren, Bran
• Primary Examiner(s): CHAN, WING F
• Application Number:
  US10/731,919
• Time in Patent Office: 553 Days
• Field of Search: 348/14.01, 348/14.07, 348/14.08, 348 1412- 1416
• US Class Current: 348/14.16
• CPC Class Codes: H04N 7/144 camera and display on the same optical axis, e.g. optically multiplexing the camera and display for eye to eye contact

Abstract

The invention comprises a structured light projector, a video camera, and an image processor, for achieving perspective corrected images that enhance eye-contact during teleconferencing. A structured light projector is offset in one direction from the monitor center, and illuminates a local participant with a structured light pattern. The image of the local participant, illuminated by both ambient and structured light, is captured by the video camera, also offset from the monitor center, preferably in the direction opposite the structured light projector. By considering the distortion of the structured light observed from the position of the video camera and the position of the structured light projector and video camera relative to the monitor center, an image processor creates an image of the local participant as viewed from a perspective that, when viewed by the remote participant, provides a sense of eye contact with the local participant.

Claims

1. A method for synthesizing a final image of an object from a desired perspective, comprising the steps of:

illuminating said object with structured light from a first position;

obtaining an image of said object from a second position;

determining a requisite warping based on a distortion of said structured light as observed from said second position; and

applying said requisite warping to said image to yield said final image.

2. The method of claim 1, wherein said requisite warping is based on a restorative warping, said restorative warping returning said distortion of said structured light to an undistorted configuration.

3. The method of claim 2, wherein said requisite warping is a fraction of said restorative warping.

4. The method of claim 3, wherein said fraction is a quotient of a distance from said second position to a point characterizing said desired perspective and a distance from said second position to said first position.

5. The method of claim 1, wherein said structured light illumination is in the infrared spectrum.

6. The method of claim 1, wherein said structured light illumination is only active during a vertical blanking interval of a video camera used in obtaining said image of said object.

7. The method of claim 1, wherein said determining step further comprises the step of:

applying a high pass convolution filter.

8. The method of claim 1, wherein said structured light comprises a series of substantially parallel lines.

9. The method of claim 8, wherein said determining step further comprises the step of:

applying at least one directional filter.

10. The method of claim 9, wherein said determining step further comprises the step of:

integrating a series of directional values along a line substantially parallel to said lines of structured light.

11. The method of claim 10, wherein limits of said integrating step are defined by a mask surrounding said object.

12. The method of claim 1, wherein said requisite warping is applied using an image warping routine.

13. An apparatus for synthesizing a final image of an object from a desired perspective, comprising:

means for illuminating said object with structured light from a first position;

means for obtaining an image of said object from a second position;

means for determining a requisite warping based on a distortion of said structured light as observed from said second position; and

means for applying said requisite warping to said image to yield said final image.

14. The apparatus of claim 13, wherein said structured light illumination is in the infrared spectrum.

15. The apparatus of claim 13, wherein said structured light illumination is only active during a vertical blanking interval of a video camera used in obtaining said image of said object.

16. The apparatus of claim 13, wherein said means for determining a requisite warping further comprises:

means for applying a high pass convolution filter.

17. The apparatus of claim 13, wherein said structured light comprises a series of substantially parallel lines.

18. The apparatus of claim 17, wherein said means for determining a requisite warping further comprises:

means for applying at least one directional filter.

19. The apparatus of claim 18, wherein said means for determining a requisite warping further comprises:

means for integrating a series of directional values along a line substantially parallel to said lines of structured light.

20. The apparatus of claim 19, wherein limits of said integrating means are defined by a mask surrounding said object.

21. The apparatus of claim 13, wherein said requisite warping is applied using an image warping routine.