Holographic video capture and telepresence system

• US 10,084,990 B2
• Filed: 11/06/2017
• Issued: 09/25/2018
• Est. Priority Date: 01/20/2016
• Status: Active Grant

• Current Assignee: Gerard Dirk Smits
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Smits, Gerard Dirk
• Primary Examiner(s): Patel, Hemant
• Application Number:
  US15/804,909
• Publication Number:
  US 20180063484A1
• Time in Patent Office: 323 Days
• Field of Search: 348 1401- 1416
• CPC Class Codes: G01S 3/00 Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received position-fixing by co-ordinating a plurality of determinations of direction or position lines G01S5/00
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G02B 27/0103 comprising holographic elements
• CPC Class Codes: G02B 30/56 by projecting aerial or floating images
• CPC Class Codes: G03H 1/0005 Adaptation of holography to specific applications holographic optical element G02B5/32; holographic scanner G02B26/106; recognition using holographic mask G06V10/88; holographic memories G11B7/0065, G11C13/042
• CPC Class Codes: G03H 2001/0088 for video-holography, i.e. integrating hologram acquisition, transmission and display
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 13/257 Colour aspects
• CPC Class Codes: H04N 13/275 from 3D object models, e.g. computer-generated stereoscopic image signals
• CPC Class Codes: H04N 13/324 Colour aspects
• CPC Class Codes: H04N 13/344 with head-mounted left-right displays
• CPC Class Codes: H04N 13/363 using image projection screens volumetric display H04N13/388
• CPC Class Codes: H04N 13/366 using viewer tracking
• CPC Class Codes: H04N 13/383 for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
• CPC Class Codes: H04N 2213/001 Constructional or mechanical details
• CPC Class Codes: H04N 7/147 Communication arrangements, e.g. identifying the communication as a video-communication, intermediate storage of the signals selecting H04Q
• CPC Class Codes: H04N 7/157 defining a virtual conference space and using avatars or agents computer conference optimisation or adaptation H04L12/1827

Abstract

The invention is directed to recording, transmitting, and displaying a three-dimensional image of a face of a user in a video stream. Reflected light from a curved or geometrically shaped screen is employed to provide multiple perspective views of the user'"'"'s face that are transformed into the image, which is communicated to remotely located other users. A head mounted projection display system is employed to capture the reflective light. The system includes a frame, that when worn by a user, wraps around and grips the user'"'"'s head. Also, at least two separate image capture modules are included on the frame and generally positioned relatively adjacent to the left and right eyes of a user when the system is worn. Each module includes one or more sensor components, such as cameras, that are arranged to detect at least reflected non-visible light from a screen positioned in front of the user.

Claims

1. A system for displaying an image to one or more users, comprising:

a display surface that is non-flat;

an apparatus that includes;

a chassis that is adapted to attach to a portion of a head of a user; and

two or more modules that are attached to the chassis and separately located adjacent to one of a left eye or a right eye of the user, wherein each module includes one or more sensors that are arranged to detect non-visible light that is reflected from the display surface; and

one or more processors that execute instructions to perform actions, including;

employing detected reflections of the non-visible light to provide a plurality of real time perspective views of at least a face of the user; and

generating the image of at least the face of the user based on the plurality of perspective views, wherein the image is provided for display by one or more remotely located users.

2. The system of claim 1, further comprising one or more light scanners that are attached to the chassis and arranged to direct non-visible light onto the display surface.

3. The system of claim 1, further comprising one or more light scanners that are attached to the chassis and arranged to direct visible light onto the display surface.

4. The system of claim 1, further comprising one or more other sensors that are attached to the chassis and arranged to detect visible light reflected from the display surface.

5. The system of claim 1, wherein the non-flat display surface further comprises a shape that is concave, folded, parabolic, triptych, cylindrical, curved, elliptical, toroid, or elliptical parabolic.

6. The system of claim 1, wherein the display surface further comprises one of:

a transparent screen positioned for total internal reflection at a back surface exposed to air;

a translucent screen arranged to reflect near infrared light with a retro-reflective coating;

or

a window of a vehicle.

7. The system of claim 1, wherein the image further comprises a real time three-dimensional (3D) video image.

8. An apparatus for capturing images of a user, that includes:

a chassis that is adapted to attach to a portion of a head of a user; and

two or more modules that are attached to the chassis and separately located adjacent to one of a left eye or a right eye of the user, wherein each module includes one or more sensors that are arranged to detect non-visible light that is reflected from a display surface; and

one or more processors that execute instructions to perform actions, including;

employing detected reflections of the non-visible light to provide a plurality of real time perspective views of at least a face of the user; and

generating an image of at least the face of the user based on the plurality of perspective views, wherein the image is provided for display by one or more remotely located users.

9. The apparatus of claim 8, further comprising one or more light scanners that are attached to the chassis and arranged to direct non-visible light onto the display surface.

10. The apparatus of claim 8, further comprising one or more light scanners that are attached to the chassis and arranged to direct visible light onto the display surface.

11. The apparatus of claim 8, further comprising one or more other sensors that are attached to the chassis and arranged to detect visible light reflected from the display surface.

12. The apparatus of claim 8, wherein the non-flat display surface further comprises a shape that is concave, folded, parabolic, triptych, cylindrical, curved, elliptical, toroid, or elliptical parabolic.

13. The apparatus of claim 8, wherein the display surface further comprises one of:

a transparent screen positioned for total internal reflection at a back surface exposed to air;

a translucent screen arranged to reflect near infrared light with a retro-reflective coating;

or

a window of a vehicle.

14. The apparatus of claim 8, wherein the image further comprises a real time three-dimensional (3D) video image.

15. A method for capturing images, comprising:

employing an apparatus to capture the images, wherein the apparatus includes

a chassis that is adapted to attach to a portion of a head of a user; and

two or more modules that are attached to the chassis and separately located adjacent to one of a left eye or a right eye of the user, wherein each module includes one or more sensors that are arranged to detect non-visible light that is reflected from the display surface; and

employing detected reflections of the non-visible light to provide a plurality of real time perspective views of at least a face of the user; and

generating an image of at least the face of the user based on the plurality of perspective views, wherein the image is provided for display by one or more remotely located users.

16. The method of claim 15, further comprising one or more light scanners that are attached to the chassis and arranged to direct non-visible light onto the display surface.

17. The method of claim 15, further comprising one or more light scanners that are attached to the chassis and arranged to direct visible light onto the display surface.

18. The method of claim 15, further comprising one or more other sensors that are attached to the chassis and arranged to detect visible light reflected from the display surface.

19. The method of claim 15, wherein the non-flat display surface further comprises a shape that is concave, folded, parabolic, triptych, cylindrical, curved, elliptical, toroid, or elliptical parabolic.

20. The method of claim 15, wherein the display surface further comprises one of:

a transparent screen positioned for total internal reflection at a back surface exposed to air;

a translucent screen arranged to reflect near infrared light with a retro-reflective coating;

or

a window of a vehicle.