Multiple-view processing in wide-angle video camera

• US 7,450,165 B2
• Filed: 04/30/2004
• Issued: 11/11/2008
• Est. Priority Date: 05/02/2003
• Status: Active Grant

• Current Assignee: Grandeye Limited (Oncam Global Group AG)
• Original Assignee: Grandeye Limited (Oncam Global Group AG)
• Inventors: Ahiska, Yavuz
• Primary Examiner(s): Ye; Lin
• Assistant Examiner(s): Hsu; Amy
• Application Number:
  US10/837,325
• Publication Number:
  US 20050007478A1
• Time in Patent Office: 1,656 Days
• Field of Search: 348/143, 348/211.9, 348/36, 348/39, 382/313, 382/312, 345/419, 345/418, 345/420, 345/422, 345/423, 345/424
• US Class Current: 348/240.2
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

A motion video camera that removes the need for mechanical pan, tilt, and zoom apparatus is disclosed. The video camera includes the following. A wide-angle optical system is configured to receive an optical image. An image sensor is coupled to the optical system and is configured to convert the optical image to an electronic image. An image processing circuit is coupled to the image sensor and is configured to receive the electronic image and to execute movement and zoom operations by correcting distortions in the electronic image introduced by the wide-angle optical system and image sensor.

Claims

1. A video camera comprising:

a wide-angle optical system configured to receive an optical image covering at least an approximate hemispherical visual field;

a high-resolution image sensor coupled to the optical system and configured to convert the optical image to an electronic image;

an image processing circuit coupled to the image sensor and configured to receive and process the electronic image, one of its tasks being to correct optical distortion through reference to a table of parameters specific to a combination of the optical system and the image sensor on one or more regions of interest of the captured said electronic image, wherein said table also maps a sensed individual pixel to a weighted average of a group of pixels to be used in said correction;

a control circuit coupled to said image processing circuit and configured to receive movement and zoom commands and to transmit a signal to said image processing circuit relating the movement and zoom commands specific to one or more selected regions of interest in said electronic image with each covering a portion of the approximate hemispherical image data, wherein the selected regions of interest include a portion of said electronic image, the image processing circuit responding to the commands to generate an electronic image with the executed commanded movement and zoom to the selected regions of interest by correcting optical distortion to simulate the function of a pan-and-tilt camera with zoom capability, and capable of selecting any region as the region of interest throughout the approximate hemisphere of image data while maintaining visibility within the entire approximate hemispherical visual field; and

an output circuit coupled to said image processing circuit capable of outputting in a plurality of electronic signal formats.

2. The video camera of claim 1, wherein the output circuit is configured to compress and format the region of interest in a manner suitable for transmission to a receiving circuit.

3. The video camera of claim 2 in which compression may include changes to frame rate and/or resolution, wavelet or DCT and compression may be varied to fit within the bandwidth of the link between the camera and receiver.

4. The video camera of claim 2 in which compression may be performed by said image processing circuit.

5. The video camera of claim 2 in which image processing and compression are performed by said controlling circuit.

6. The video camera of claim 1 in which said image processing circuit may perform image analysis on the portions or the totality of said electronic image or on the geometrically corrected portions or the totality of the electronic image to determine event detection and act upon said detection.

7. The video camera of claim 1 in which a magnetic storage device is used to store previously captured electronic image.

8. The video camera of claim 6 in which regions of interest are extracted from said stored electronic image.

9. The video camera of claim 1 wherein the wide-angle optical system includes a fisheye lens capturing image data for the approximate hemisphere.

10. The video camera of claim 1 wherein said image processing circuit can execute said movement and zoom commands specific to said one or more regions of interest in said electronic image while performing image analysis on the portions or the totality of said electronic image or on the geometrically corrected portions or the totality of the electronic image to determine event detection and act upon said detection.

11. The video camera of claim 1 wherein said image processing circuit is a digital signal processor.

12. The video camera of claim 11 wherein said digital signal processor is comprised of an arithmetic and logic unit, an instruction sequencer, an instruction cache, a memory module, a data cache, image capture circuitry, and a register storage.

13. The video camera of claim 11 wherein said digital signal processor is used to execute algorithms upon an image selected from the group consisting of:

sharpening, contrast adjustment, color adjustment, resizing, motion detection, smoke detection, and abandoned package detection.

14. The video camera of claim 1 wherein said image processing circuit is a programmable graphics processor.

15. The video camera of claim 14 wherein said programmable graphics processor is comprised of a rasterizer, a pixel shader, a memory module, a data cache, a pixel filter, a pixel write unit, and image capture circuitry.

16. The video camera of claim 1 wherein said image processing circuit is a combination of a digital signal processor and a graphics processor.

17. The video camera of claim 1 wherein said control circuit is a central processing unit.

18. The video camera of claim 1 wherein said camera can combine a plurality of images into a single image.

19. The video camera of claim 1 wherein said camera can provide said region of interest and said electronic image in overlapping views.

20. The video camera of claim 1 wherein said camera can scale said region of interest and said electronic image at different resolutions.

21. A video camera comprising:

a wide-angle optical system configured to receive an optically distorted optical image of an approximate visual hemisphere;

an image sensor coupled to the optical system that converts the optical image into an electronic image;

an image processing circuit coupled to the image sensor that receives and processes the electronic image data to mathematically correct the optical distortion and capable of magnifying and displaying at least one view of a region of interest within the electronic image while maintaining and displaying a view covering the entire approximate hemispherical visual field of the electronic image;

a control circuit coupled to said image processing circuit that receives movement and zoom commands to transmit to the image processing circuit relating to one or more selected regions of interest in the electronic image, the image processing circuit responding to the commands to generate an electronic image correlating with the executed commanded movement by correcting optical distortion to simulate the movement and zoom commands; and

a compression and formatting circuit coupled to said image processing circuit capable of outputting processed electronic image data in a plurality of signal formats.

22. The video camera of claim 21, further comprising:

at least one analysis algorithm operating in the image processing circuit.

23. The video camera of claim 22, wherein an algorithm detects at least one of the following:

motion detection;

smoke detection; and

object detection.

24. The video camera of claim 21, further comprising:

a table of parameters used to correct the distorted optical image that is specific to a combination of the optical system and the image sensor on one or more regions of interest of the captured electronic image, wherein the table also maps a sensed individual pixel to a weighted average of a group of pixels to be used in the correction of the distorted optical image.

25. The video camera of claim 21, wherein the image processing circuit comprises an arithmetic and logic unit, an instruction sequencer, an instruction cache, a memory module, a data cache, image capture circuitry, and a register storage.

26. The video camera of claim 21, wherein the image processing circuit can execute movement and zoom commands specific to one or more regions of interest while simultaneously performing image analysis on the portions or the totality of the electronic image, on the geometrically corrected portions, or the totality of the electronic image to detect an event.

27. The video camera of claim 21 in which said image processing circuit may perform image analysis on the portions or the totality of said electronic image or on the geometrically corrected portions or the totality of the electronic image to determine event detection and act upon said detection.