Time-of-flight based imaging system using a display as illumination source

• US 8,810,647 B2
• Filed: 10/08/2010
• Issued: 08/19/2014
• Est. Priority Date: 04/11/2008
• Status: Expired due to Fees

• Current Assignee: Ecole polytechnique fãdãrale de lausanne epfl (Schweizerische Eidgenossenschaft)
• Original Assignee: Ecole polytechnique fãdãrale de lausanne epfl (Schweizerische Eidgenossenschaft)
• Inventors: Niclass, Cristiano, Charbon, Edoardo, Nolan, Julian
• Primary Examiner(s): TORRENTE, RICHARD T
• Application Number:
  US12/900,798
• Publication Number:
  US 20110037849A1
• Time in Patent Office: 1,411 Days
• Field of Search: 348/135
• US Class Current: 348/135
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 7/4814 of transmitters alone
• CPC Class Codes: G01S 7/497 Means for monitoring or calibrating
• CPC Class Codes: G06F 3/0421 by interrupting or reflecting a light beam, e.g. optical touch-screen

Abstract

The present invention concerns a time-of-flight based imaging system having a photon emitter used as an illumination source, a photon sensor, and an electronic system for delivering a signal depending on the reception time of photons by said photon detector. The electronic display is used as the photonic source.

Claims

1. A time-of-flight based imaging system comprising:

a photon emitter used as an illumination source for illuminating a scene;

a photon sensor to sense photons reflected by an object in the scene;

an electronic system for delivering a signal depending on the reception time of photons reflected by said scene onto said photon sensor,

a backlit display comprising a backlight assembly and an electronic display placed in front of the backlight assembly,

said backlight assembly comprises LEDs which are used for illuminating the electronic display and which are also used as said illumination source by superposing on the LEDs intensity normally used for lighting the display, a modulated light;

the photon sensor being configured to record the modulated light reflected by an object and,

the electronic system being configured to demodulate the modulated light reflected by an object and recorded by the photon sensor, to determine the position of said object, and the distance to said object,

wherein the system further comprises a backlighting driver for successively illuminating said photon sensor from different regions of said display, and computing means for computing the position of a light reflecting object based on light received from the different regions and a triangulation method.

2. The imaging system of claim 1, wherein said backlight assembly comprises infrared LEDs, said photon sensor being associated with an infrared filter for improving sensitivity to the infrared light emitted by said infrared LEDs.

3. The imaging system of claim 1, wherein said backlight assembly uses white LEDs, said photon detector being associated with a filter for improving sensitivity to the blue light emitted by said white LEDs.

4. The imaging system of claim 1, comprising different LEDs of different colors illuminated and/or modulated at different times.

5. The imaging system of claim 1, comprising a backlighting driver for successively emitting light used by said photon sensor from successive regions of said display.

6. The imaging system of claim 5, said backlighting driver being arranged for successively illuminating successive regions of said display.

7. The imaging system of claim 5, where the distances between each region successively lit and object are successively computed.

8. The imaging system of claim 5, said electronic display being arranged for modifying the video content displayed by said currently backlighted regions so as to increase the amount of light sent from said regions and received by said photon sensor.

9. The imaging system of claim 1, wherein a video image is used as said illumination source.

10. The imaging system of claim 1, wherein said photon sensor is mounted on different device than said electronic display, so that the position between said display and said photon sensor is not fixed, the imaging system also comprising a clock recovery system for recovering the clock signal from said display in said different device.

11. The imaging system of claim 1, as part of a game console.

12. The imaging system of claim 1, as part of a personal digital assistant or mobile phone.

13. The imaging system of claim 1, which photon sensor comprises an array of single photon avalanche photodiodes elements.

14. Illumination method for an imaging system comprising at least a backlit display comprising a backlight assembly and an electronic display placed in front of the backlight assembly, the backlight assembly comprising LEDs which are used for illuminating the electronic display and which are also used as an illumination source by superposing on the LEDs intensity normally used for lighting the display, a modulated light;

a photon sensor; and

an electronic system for delivering a signal by said photon sensor;

comprising the steps of;

generating a modulated illumination from successively different regions of an electronic display using a backlighting driver by superposing on the LEDs intensity normally used for lighting the display, a modulated light;

computing the distances between the center of each region successively lit and the photon sensor;

recording the reception time of the modulated illumination reflected by an object or the phase delay between the emitted illumination and the reflected illumination using the photon sensor;

demodulating the modulated light reflected by an object and recorded by the photon sensor, using the electronic system, to determine the position of said object, and the distance to said object,

wherein the position of a light reflecting object is computed based on light received from the different regions and a triangulation method.

15. Illumination method according to claim 14, where display video content is modified in synchronization with the successive illumination.

16. Illumination method according to claim 14, where the regions successively illuminated are moved across the display following a random pattern.

17. A portable device comprising:

an electronic display;

a led-based backlighting system for backlighting said electronic display;

a photon detector;

an electronic system for delivering a signal depending on the time of flight of photons sent by said backlighting system, reflected by an object and received by said photon detector;

wherein LEDs of the led-based backlighting system are used for illuminating the electronic display and are also used as an illumination source to illuminate an object by superposing a modulated light on the LEDs intensity normally used for lighting the display;

the photon detector being configured to record the modulated light reflected by an object and,

the electronic system being configured to demodulate the modulated light reflected by an object and recorded by the photon sensor, to determine the position of said object, and the distance to said object,

wherein the system further comprises a backlighting driver for successively illuminating said photon sensor from different regions of said display, and computing means for computing the position of a light reflecting object based on light received from the different regions and a triangulation method.

18. The portable device of claim 17, further comprising computing means programmed for computing a 3D representation of said object based on said signal.

19. Computer program product containing a non-transitory computer program which is capable of being executed by processing means in order to carry out an illumination method for an imaging system comprising at least a backlit display comprising a backlight assembly and an electronic display placed in front of the backlight assembly, the backlight assembly comprising LEDs which are used for illuminating the electronic display and which are also used as an illumination source by superposing on the LEDs intensity normally used for lighting the display, a modulated light;

a photon sensor; and

an electronic system for delivering a signal by said photon sensor;

the illumination method comprising the steps of;

generating a modulated illumination from successively different regions of an electronic display using a backlighting driver by superposing on the LEDs intensity normally used for lighting the display, a modulated light;

computing the distances between the center of each region successively lit and the photon sensor;

recording the reception time of the modulated illumination reflected by an object or the phase delay between the emitted illumination and the reflected illumination using the photon sensor;

demodulating the modulated light reflected by an object and recorded by the photon sensor, using the electronic system, when said program is executed by said processing means in order to carry out the driving of the illumination source of a time-of- flight based imaging system comprising;

a photon emitter used as an illumination source for illuminating a scene;

a photon sensor to sense photons reflected by an object in the scene;

an electronic system for delivering a signal depending on the reception time of photons reflected by said scene onto said photon sensor;

a backlit display comprising a backlight assembly and an electronic display placed in front of the backlight assembly;

said backlight assembly comprises LEDs which are used for illuminating the electronic display and which are also used as said illumination source by superposing on the LEDs intensity normally used for lighting the display, a modulated light;

the photon sensor being configured to record the modulated light reflected by an object and,

the electronic system being configured to demodulate the modulated light reflected by an object and recorded by the photon sensor, to determine the position of said object, and the distance to said object,

wherein the system further comprises a backlighting driver for successively illuminating said photon sensor from different regions of said display, and computing means for computing the position of a light reflecting object based on light received from the different regions and a triangulation method.