Image Acquisition Unit, Acquisition Method, and Associated Control Unit

• US 20120081544A1
• Filed: 09/29/2011
• Published: 04/05/2012
• Est. Priority Date: 10/01/2010
• Status: Abandoned Application

• Current Assignee: Jay Young Wee
• Original Assignee: Jay Young Wee
• Inventors: Wee, Jay Young
• Application Number:
  US13/248,053
• Publication Number:
  US 20120081544A1
• Time in Patent Office: Days
• US Class Current: 348/140
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G01S 17/931 of land vehicles
• CPC Class Codes: G01S 7/4813 Housing arrangements
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects

Abstract

Video arrayed data acquired via at least one video camera, can be co-registered with lidar arrayed data acquired from a lidar receiver data into a combined arrayed data. The co-registration and data acquisition can be done within a common housing having an combined arrayed data output which can be connected to a control module. The control module can have a video signal acquirer, a video processor for processing the acquired video signal, a threat analyzer capable of detecting a threat from the processed video signal or from another source, and a memory storage device.

Claims

1. An image acquisition unit comprising:

a housing,

a video camera system including at least one video camera and a video output for video arrayed data acquired via the at least one video camera,

a lidar system including at least one lidar emitter and a lidar receiver, and a lidar output for lidar arrayed data acquired from the lidar receiver,

a fusion integrator connected to both the video output and the lidar output for receiving both the video arrayed data and the lidar arrayed data, the fusion integrator having a co-registering function to co-register the video arrayed data and the lidar arrayed data into a combined arrayed data, and

an output for the combined arrayed data leading out of the housing.

2. The image acquisition unit of claim 1 wherein the video camera system includes a CMOS and a camera controller.

3. The image acquisition unit of claim 1 wherein the lidar system includes a pulse generator, a range/intensity acquirer, and an intensity/depth array constructor.

4. The image acquisition unit of claim 3 wherein the pulse generator includes a coder, further comprising a pulse code validator acting as a gate between the range/intensity acquirer and the intensity/depth constructor for rejecting acquired range/intensity data if the pulse code is not validated.

5. The image acquisition unit of claim 1 wherein the video camera is one of a wide angle camera and a telephoto/zoom camera, and the video camera system further includes the other one of a wide angle camera and a telephoto/zoom camera.

6. The image acquisition unit of claim 5 wherein the telephoto/zoom camera is orientable.

7. The image acquisition unit of claim 1 wherein the video camera is movable.

8. The image acquisition unit of claim 7 wherein the video camera is mounted on an extendible arm and is movable by extension of the extendible arm.

9. The image acquisition unit of claim 1 wherein the lidar system includes at least two laser emitters coupled by co-alignment optics.

10. The image acquisition unit of claim 1 wherein all of the video camera system, the lidar system, and the fusion integrator have electronics part of a common FPGA.

11. The image acquisition unit of claim 1 wherein the video camera system, the lidar system, and the fusion integrator are mounted in the housing.

12. A method comprising:

acquiring video arrayed data from at least one video camera;

acquiring lidar arrayed data from the reflected lidar signal received; and

co-registering the video arrayed data with the lidar arrayed data into a combined arrayed data signal.

13. The method of claim 12, wherein the video arrayed data has a 2D array of a given number of video pixels, each video pixel having red (R), green (G) and blue (B) data;

the lidar arrayed data has a 2D array of a given number of lidar pixels, each lidar pixel having intensity (I) and depth (D) data; and

the combined arrayed data has a number of combined pixels, each combined pixel having red (R), green (G), blue (B), intensity (I) and depth (D) data.

14. The method of claim 13 wherein the number of video pixel is greater than the number of lidar pixels, wherein said co-registering includes associating the intensity (I) and depth (D) data of each lidar pixel to the red (R), green (G) and blue (B) data of more than one of said video pixels.

15. The method of claim 12 wherein said acquiring lidar arrayed data includes emitting a lidar signal and receiving a reflected lidar signal.

16. The method of claim 15 wherein said emitting a lidar signal includes obtaining a given pattern and emitting a lidar signal based on the given pattern in a repetitive manner;

wherein said receiving further comprises comparing the reflected lidar signal to said given pattern and rejecting said reflected lidar signal if the reflected lidar signal does not match the given pattern.

17. The method of claim 16 further comprising monitoring a number of successive rejected reflected lidar signals, and changing the given pattern to another pattern upon determining that the number of successive rejected reflected lidar signals has reached a predetermined threshold.

18. The method of claim 16 wherein the given pattern is selected from a given number of patterns.

19. The method of claim 12 further comprising providing the combined arrayed data signal to control modules of an automotive vehicle for analysis.

20. A control module comprising a video signal acquirer, a video processor for processing the acquired video signal, a threat analyzer capable of detecting a threat from the processed video signal or from another source, and a memory storage device.

21. The control module of claim 20, further comprising a common housing in which each of the video signal acquirer, the video processor, the threat analyser and the memory storage device are mounted.

22. The control module of claim 20 further comprising storing a predetermined amount of recent history data from the video signal acquirer;

wherein the recent history data is stored into the memory storage device upon detection of a threat.

23. The control module of claim 20 further comprising a sound recorder system including a microphone, an audio output for audio data acquired via the microphone, and an audio memory storing a predetermined amount of recent history data from the audio data;

wherein the recent history data is stored into the memory storage device upon detection of a threat.

24. The control module of claim 20 wherein the video signal includes combined arrayed data having a number of combined pixels, wherein each combined pixel has at least red (R), green (G), blue (B), and depth (D) data.

25. The control module of claim 20 having at least one of relative vehicle position, velocity data, and time of impact data which can be automatically stored into the memory storage device upon detection of a threat.