3-dimensional hybrid camera and production system

• US 8,743,176 B2
• Filed: 05/19/2010
• Issued: 06/03/2014
• Est. Priority Date: 05/20/2009
• Status: Active Grant

• Current Assignee: Continental Autonomous Mobility US LLC (Continental AG)
• Original Assignee: Advanced Scientific Concepts, Inc.
• Inventors: Stettner, Roger, Short, Brad, Gilliland, Patrick, Heughebaert, Laurent, Laux, Tom
• Primary Examiner(s): Dastouri, Mehrdad
• Assistant Examiner(s): ANDERSON II, JAMES M
• Application Number:
  US12/782,845
• Publication Number:
  US 20100309288A1
• Time in Patent Office: 1,476 Days
• Field of Search: None
• US Class Current: 348/43
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: H04N 13/189 Recording image signals; Reproducing recorded image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/25 using two or more image sensors with different characteristics other than in their location or field of view, e.g. having different resolutions or colour pickup characteristics; using image signals from one sensor to control the characteristics of another sensor
• CPC Class Codes: H04N 13/254 in combination with electromagnetic radiation sources for illuminating objects
• CPC Class Codes: H04N 23/54 Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
• CPC Class Codes: H04N 23/55 Optical parts specially adapted for electronic image sensors; Mounting thereof
• CPC Class Codes: H04N 25/41 Extracting pixel data from a plurality of image sensors simultaneously picking up an image, e.g. for increasing the field of view by combining the outputs of a plurality of sensors
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A hybrid three dimensional imaging camera comprises a 2-D video camera utilizing a focal plane array visible light detector and a 3-D flash laser radar utilizing an infrared focal plane array detector. The device is capable of capturing a complete 3-D scene from a single point of view. A production system combining multiple hybrid 3-D cameras around a subject provides 3-D solid models of an object or scene in the common field of view.

Claims

1. A hybrid three dimensional camera system comprising:

a pulsed infrared laser transmitter providing an output laser pulse upon command from a system controller;

an optical sampler adapted to sample the output of the laser transmitter and provide the optical sample via optical transmission path to a laser power detector, said laser power detector having a laser power detector electrical signal indicating the time of transmission of the laser pulse;

a two dimensional visible light camera, having a field of view, the two dimensional visible light camera incorporating an array of visible light optical detectors positioned at the focal plane of a motorized zoom lens and a two dimensional image output, said zoom lens adapted to optimize overlap of the two dimensional visible light camera field of view and the output laser pulse;

a pixilated infrared optical receiver incorporating an array of infrared optical detectors positioned at the focal plane of a lens system and each optical detector of said array producing an electrical output pulse at a terminal of said optical detector in response to a reflected laser pulse, and each optical detector having said terminal electrically connected to an input of a unit cell electrical circuit of a readout integrated circuit;

said readout integrated circuit having an array of unit cell electrical circuits, and each unit cell electrical circuit having an input adapted to receive an electrical output pulse from an optical detector of said array of infrared optical detectors, and each unit cell electrical circuit having a timing circuit initiated by said laser power detector electrical signal, and said timing circuit terminating upon receipt of an electrical output pulse from an optical detector of said array of infrared optical detectors; and

said readout integrated circuit having an output providing range and intensity data for each optical detector of said array of infrared optical detectors receiving an incident reflected light portion of said output laser pulse;

an image processor which merges the output of said two dimensional visible light camera with the range and intensity data output of said infrared optical receiver to produce a three dimensional visible picture output;

a system controller which initiates the pulsing of said pulsed laser transmitter and which controls the operation of said two dimensional visible light camera zoom lens, as well as the timing of the output of said infrared optical receiver and the timing of said image processor three dimensional visible picture output.

2. The hybrid three dimensional camera system of claim 1 wherein the infrared optical receiver and two dimensional visible light camera are separately housed and said infrared optical receiver provides at least two adjustable mounts for establishing coaxial mechanical alignment of said infrared optical receiver and the two dimensional visible light camera.

3. The hybrid three dimensional camera system of claim 1 wherein ambient visible light and reflected infrared light enter a lens tube of a camera housing coaxially, and are angularly separated by a dichroic optical element, and a focal plane array of the infrared optical receiver is mounted within the camera housing to receive said reflected infrared light, and a focal plane array of the two dimensional visible light camera is mounted within the camera housing to receive said ambient visible light.

4. The hybrid three dimensional camera system of claim 1 further comprising a fused fiber breakout having an ordered array of light guiding fibers, said array of light guiding fibers, with an input face and at least one output face, positioned with the input face at the focal plane of the lens system, said input face receiving a reflected portion of the transmitted laser pulse as a light signal image and transmitting said light signal image through a length of an optical wave guiding fiber bundle in an ordered fashion to preserve a light signal image, to at least one array of infrared optical detectors.

5. The hybrid three dimensional camera system of claim 4 wherein the fused fiber breakout is fused at the input face and at least one output face.

6. The hybrid three dimensional camera system of claim 5 wherein said length of an optical wave guiding fiber bundle is contoured in at least one axis, and provides a translation of the light signal image incident upon an input face on at least one lateral axis at an output face.

7. The hybrid three dimensional camera system of claim 4 wherein the input face of the fused fiber breakout incorporates a precision substrate perforated by a two dimensional array of holes, each hole centered on a rectangular grid pattern and passing through said substrate, said holes adapted to accept and retain an optical fiber of an optical fiber bundle.

8. The hybrid three dimensional camera system of claim 4 wherein the input face of the fused fiber breakout is overmolded by an array of lenses, each lens being centered over a light guiding fiber.

9. The hybrid three dimensional camera system of claim 4 wherein a fiber of the fused fiber breakout has a rectangular cross section.

10. The hybrid three dimensional camera system of claim 4 wherein an output face of the fused fiber breakout is held in optical alignment to a two dimensional array of infrared optical detectors.

11. The hybrid three dimensional camera system of claim 1 wherein the array of infrared optical detectors is situated atop an integrated circuit substrate with an array of electrically conductive bumps, each bump making electrical contact with an electrical contact of an element of the two dimensional array of infrared detectors.

12. The hybrid three dimensional camera system of claim 1 wherein the array of infrared optical detectors is a two dimensional array of avalanche photodiodes.

13. The hybrid three dimensional camera system of claim 1 wherein the array of infrared optical detectors is a two dimensional array of PIN photodiodes.

14. The hybrid three dimensional camera system of claim 4 wherein an output face of the fused fiber breakout is held in optical alignment to a proximity focused image tube.

15. The hybrid three dimensional camera system of claim 1 wherein said array of infrared optical detectors has a rectangular aspect ratio of sixteen to nine.

16. The hybrid three dimensional camera system of claim 1 wherein the array of infrared optical detectors is a two dimensional array, and is situated atop a readout integrated circuit with a corresponding two dimensional array of electrically conductive bumps, each bump making electrical contact with an electrical contact of an element of the two dimensional array of infrared optical detectors and each bump further making electrical connection to a unit cell of said readout integrated circuit,

wherein a unit cell of said readout integrated circuit includes a flash laser radar receiving and pulse detection and timing circuit adapted to produce range data for any reflected portion of a transmitted laser pulse incident upon any element of the two dimensional array of infrared optical detectors.

17. A 3D imagery production system comprising:

a plurality of hybrid 3D cameras, each hybrid 3D camera of said plurality of hybrid 3D cameras having a field of view, and adapted to develop a 3D visual image by combining the output of a 2D visible light focal plane array detector with a 3D camera;

a zoom lens adapted to optimize overlap of the 2D visible light focal plan array detector and the field of view of a 3D camera;

each 3D camera including a scene illuminating pulsed infrared laser transmitter; and

each laser transmitter having an optical sampler adapted to sample the output of the laser transmitter and provide the optical sample via optical transmission path to a laser pulse detector;

said laser pulse detector having a laser pulse detector output indicating the time of transmission of a laser pulse; and

each 3D camera further including a plurality of optical detector electrical outputs from a pixellated infrared focal plane array detector, and each of said optical detector electrical outputs adapted to detect a reflected laser pulse; and

each of said optical detector electrical outputs connected to an amplification and range determining circuit of a companion unit cell electrical circuit of a mating readout integrated circuit;

wherein the range determining circuit within each unit cell electrical circuit comprises a timer which is initiated by said laser pulse detector output, and said timer terminated upon arrival of a reflected laser pulse, the range measuring circuit thus adapted to measure the range to a point in a scene of interest;

wherein each hybrid 3D camera further includes a viewing axis directed towards a common object or scene of interest, each said viewing axis separated by a solid angle from the viewing axis of any other hybrid 3D camera of said 3D imagery production system;

wherein each hybrid 3D camera further includes a communications link to a central processor, the central processor adapted to merge the outputs of said hybrid 3D cameras into a 3D solid model of an object within the common field of view of the hybrid 3D cameras.

18. A hybrid three dimensional camera system comprising:

a pulsed infrared laser transmitter providing an output laser pulse upon command from a system controller;

an optical sampler adapted to sample the output of the laser transmitter and provide the optical sample via optical transmission path to a laser power detector;

said laser power detector having a laser power detector electrical signal indicating the time of transmission of the laser pulse;

a two dimensional visible light camera incorporating an array of visible light optical detectors positioned at the focal plane of a visible light transmissive lens and with a two dimensional image output;

a pixilated infrared optical receiver incorporating an array of infrared optical detectors positioned at the focal plane of an infrared light transmissive lens, and each optical detector of said array producing an electrical output pulse at a terminal of said optical detector in response to a reflected laser pulse, and each optical detector having said terminal electrically connected to an input of a unit cell electrical circuit of a readout integrated circuit;

said readout integrated circuit having an array of unit cell electrical circuits, and each unit cell electrical circuit having an input adapted to receive an electrical output pulse from an optical detector of said array of infrared optical detectors, and each unit cell electrical circuit having a timing circuit initiated by said laser power detector electrical signal, and said timing circuit terminated upon receipt of an electrical output pulse from an optical detector of said array of infrared optical detectors; and

said readout integrated circuit providing, through an output, range and intensity data for any pixel element of said array of infrared optical detectors receiving an incident reflected light portion of said output laser pulse;

an image processor which merges the output of said two dimensional visible light camera with the range and intensity data output of said infrared optical receiver to produce a three dimensional visible picture electrical output;

a system controller which initiates the pulsing of said pulsed laser transmitter and which controls the operation of said two dimensional visible light camera and the infrared optical receiver, and further controls the timing of the output of said infrared optical receiver and the timing of said image processor three dimensional visible picture electrical output.

19. The hybrid three dimensional camera system of claim 18 wherein the infrared optical receiver and two dimensional visible light camera are separately housed and said infrared optical receiver provides at least two adjustable mounts for establishing coaxial mechanical alignment of said infrared optical receiver and the two dimensional visible light camera.

20. The hybrid three dimensional camera system of claim 18 wherein said two dimensional array of infrared optical detectors is situated atop an integrated circuit substrate having an array of electrically conductive bumps, each bump making electrical contact with an electrical contact of an element of the said two dimensional array of infrared detectors.