Foveal camera systems and methods

• US 7,965,314 B1
• Filed: 03/23/2006
• Issued: 06/21/2011
• Est. Priority Date: 02/09/2005
• Status: Active Grant

• Current Assignee: Teledyne FLIR LLC (Teledyne Technologies Incorporated)
• Original Assignee: FLIR Systems, Inc. (Teledyne Technologies Incorporated)
• Inventors: Teich, Andrew C., Frank, Jeff, Miller, John L.
• Primary Examiner(s): Philippe; Gims S
• Application Number:
  US11/388,696
• Time in Patent Office: 1,916 Days
• Field of Search: 348/164, 348/333.12, 356/5.04, 345/698
• US Class Current: 348/164
• CPC Class Codes: G08B 13/19643 wherein the cameras play different roles, e.g. different resolution, different camera type, master-slave camera
• CPC Class Codes: G08B 13/19667 Details realated to data compression, encryption or encoding, e.g. resolution modes for reducing data volume to lower transmission bandwidth or memory requirements
• CPC Class Codes: H04N 1/3876 Recombination of partial images to recreate the original image
• CPC Class Codes: H04N 21/2365 Multiplexing of several video streams
• CPC Class Codes: H04N 21/2662 Controlling the complexity of the video stream, e.g. by scaling the resolution or bitrate of the video stream based on the client capabilities
• CPC Class Codes: H04N 21/43072 of multiple content streams on the same device
• CPC Class Codes: H04N 21/64322 IP
• CPC Class Codes: H04N 21/8451 using Advanced Video Coding [AVC]
• CPC Class Codes: H04N 23/11 for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 23/23 from thermal infrared radiation
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

Systems and methods are disclosed for providing foveal images. For example, high and low resolution images can be combined to provide foveal images. Parameters other than resolution can vary between the larger and smaller images of the foveal image. Real, artificial, 2D and 3D images can be used in any desired combination to form foveal images. The use of such foveal images can substantially enhance the ability of viewers to accurately and reliably interpret the content of images and provide reduce bandwidth for image transmission.

Claims

1. A camera system comprising:

a camera housing;

a first camera, secured with the camera housing, adapted to provide first image data from a first portion of an electromagnetic spectrum, wherein the first camera is adapted to provide a wide field of view;

a second camera, secured with the camera housing, adapted to provide second image data from a second portion of the electromagnetic spectrum, wherein the second camera is configured to provide a narrow field of view disposed within the wide field of view of the first camera, and wherein at least one parameter associated with generating the first image data of the first camera is different than the at least one parameter associated with generating the second image data of the second camera;

a third camera, secured within the camera housing, adapted to provide third image data covering a visible portion of the electromagnetic spectrum, wherein the third camera is configured to provide a field of view disposed within the wide field of view of the first camera;

a first processor, secured within the camera housing, adapted to receive the first image data from the first camera, the second image data from the second camera, and the third image data from the third camera to provide the first image data, the second image data, and the third image data as one or more network data streams to a network, with at least one of the network data streams providing a foveal view composite image which has been scaled and encoded with different scale factors; and

wherein the camera housing encloses the first camera, the second camera, the third camera, and the first processor and forms a first camera system, and wherein a remote processor coupled to the network is able to receive the network data streams from one or more of the first camera systems to select from among the network data streams to store and/or display.

2. The camera system as recited in claim 1, further comprising means for moving the camera housing, wherein the first camera and the second camera are infrared cameras, and wherein the at least one parameter comprises illumination or polarization.

3. The camera system as recited in claim 1, further comprising means for moving the camera housing, wherein the network data streams provide a first network data stream corresponding to the third image data and at least a second network data stream corresponding to the first image data and the second image data to provide a spectrally fused image for the foveal view composite image.

4. The camera system as recited in claim 1, further comprising a display adapted to display images from the network data streams, wherein the display is further adapted to display at least one of radar data and lidar data along with the selected network data streams, and wherein the first processor is adapted to provide an artificial image to display along with at least one of the first image data, the second image data, the third image data, or the foveal view composite image.

5. The camera system as recited in claim 4, wherein the at least one of radar data and lidar data provides spectrally fused imagery.

6. The camera system as recited in claim 1, wherein the field of view of the third camera is centered within the narrow field of view of the second camera which is centered within the wide field of view of the first camera relative to a certain distance from the camera housing.

7. The camera system as recited in claim 1, wherein the at least one parameter comprises a color balance, a compression type, and/or a compression rate.

8. The camera system as recited in claim 1, wherein the at least one parameter comprises a contrast, a frame rate, and/or an intensity.

9. The camera system as recited in claim 1, wherein the first processor is adapted to provide image processing, image merging, encoding, and formatting of the first, second, and third image data.

10. The camera system as recited in claim 1, wherein the at least one parameter comprises encoding, focus, and/or frame rate.

11. The camera system as recited in claim 1, further comprising an interface adapted to provide a wireless interface to the network.

12. The camera system as recited in claim 1, further comprising means for moving the camera housing, wherein the first camera and the second camera are infrared cameras, and wherein the first portion of the electromagnetic spectrum differs from the second portion.

13. The camera system as recited in claim 1, wherein the first processor is adapted to provide image stabilization, edge enhancement, and/or warping for the first, second, and/or third image data.

14. The camera system as recited in claim 1, wherein the at least one parameter comprises a variable portion of the electromagnetic spectrum to be imaged by the first and second cameras.

15. The camera system as recited in claim 1, wherein the at least one parameter comprises a frame rate, an edge enhancement, and/or a polarization, which are variable.

16. The camera system as recited in claim 1, wherein:

the first camera is adapted to provide the first image data comprising infrared image data having a first polarization, a first spectral fusion, and based on an active illumination;

the second camera is adapted to provide the second image data comprising infrared image data having a second polarization, a second spectral fusion, and not based on an active illumination; and

a display system adapted to show the first image data and the second image data as the foveal view composite image.

17. The camera system as recited in claim 1, further comprising:

at least one of a LIDAR transceiver and a radar transceiver; and

a display system adapted to show image data from the first camera system and the at least one of a LIDAR transceiver and a radar transceiver as a foveal image.

18. The camera system as recited in claim 1, further comprising:

the remote processor adapted to receive fourth image data from the network, wherein the fourth data image data is provided from a radar system or a lidar system;

a memory adapted to store the first, second, third, and fourth image data, wherein the first image data has a lower resolution than the second image data; and

a display system adapted to show the first, second, third, and fourth image data in various combinations as a foveal image.

19. The camera system as recited in claim 18, wherein the first image data provides a real image and the fourth image data provides a synthetic image.

20. The camera system as recited in claim 18, wherein the first image data and the second image data provides infrared image data and the fourth image data provides a real image, and wherein the first processor is adapted to provide an artificial image to display along with at least one of the first image data, the second image data, the third image data, or the foveal view composite image.

21. The camera system as recited in claim 18, wherein at least one of the first and second image data provides an infrared image and the fourth image data provides a radar image.

22. The camera system as recited in claim 18, wherein at least one of the first and second image data provides an infrared image and along with the fourth image data provides a fused three-dimensional image.

23. The camera system as recited in claim 18, wherein the first and second image data provide infrared images covering different portions of the electromagnetic spectrum and the fourth image data provides radar images.

24. The camera system as recited in claim 18, further comprising a pan/tilt mechanism adapted to move the camera housing, and wherein the first and second image data provide infrared images covering different portions of the electromagnetic spectrum and the fourth image data provides lidar images.

25. The camera system as recited in claim 18, wherein the field of view of the third camera is centered within the narrow field of view of the second camera which is centered within the wide field of view of the first camera relative to a certain distance from the camera housing.

26. The camera system as recited in claim 18, wherein the at least one parameter comprises a color balance, a compression type, and/or a compression rate.

27. The camera system as recited in claim 18, wherein the at least one parameter comprises a contrast, a frame rate, and/or an intensity.

28. The camera system as recited in claim 18, wherein the first processor is adapted to provide image processing, image merging, encoding, and formatting of the first, second, and third image data.

29. The camera system as recited in claim 18, further comprising an interface, disposed at least partially within the camera housing, adapted to provide a wireless interface from the camera housing to the network.

30. A method for providing a foveal image, the method comprising:

generating first thermal image data, from within a first camera housing, covering a narrow field of view;

generating second thermal image data, from within the first camera housing, covering a wide field of view, wherein the narrow field of view of the first thermal image data is disposed in a first area of the wide field of view of the second thermal image data, and wherein at least one parameter associated with the generating of the first thermal image data differs from the at least one parameter associated with the generating of the second thermal image data;

generating first visible image data having a first field of view from within the first camera housing, wherein the first field of view of the first visible image data is disposed in a second area of the wide field of view of the second thermal image data;

providing the first visible image data, the first thermal image data, and the second thermal image data from the first camera housing as network data streams to a network, with at least one of the network data streams providing a foveal view composite image which has been scaled and encoded with different scale factors; and

wherein the network data streams are provided to the network such that a computer coupled to the network is able to receive the network data streams to select from among the network data streams to store and/or display the corresponding image data.

31. The method as recited in claim 30, further comprising selecting, by each of the computers coupled to the network, from the network data streams provided on the network from the first camera housings, and wherein the network data streams correspond to the first thermal image data, the second thermal image data, the first visible image data, and the foveal view composite image.

32. The method as recited in claim 30, further comprising moving the first camera housing to point in a desired direction, and wherein the narrow field of view and the first field of view are centered within the wide field of view for a certain viewing distance from the first camera housing.

33. The method as recited in claim 30, wherein the providing comprises image processing, image merging, encoding, and formatting for at least the first and second thermal image data, and wherein the at least one parameter comprises polarization, phase, frame rate, and/or portion of the electromagnetic spectrum.

34. The method as recited in claim 30, wherein the at least one parameter comprises illumination, polarization, portion of the electromagnetic spectrum imaged, compression type, and/or compression rate.

35. The method as recited in claim 30, wherein the at least one parameter comprises contrast, color balance, edge enhancement, image stabilization, and/or intensity.

36. The method as recited in claim 30, wherein the providing to the network comprises a wireless transmission.

37. The method as recited in claim 30, further comprising providing one or more artificial images to display within a designated area of an image along with at least one of the first visible image data, the first thermal image data, or the second thermal image data.