Multi-sensor panoramic network camera

• US 7,680,192 B2
• Filed: 07/14/2004
• Issued: 03/16/2010
• Est. Priority Date: 07/14/2003
• Status: Active Grant

• Current Assignee: Arecont Vision Costar, LLC (Costar Technologies, Inc.)
• Original Assignee: Arecont Vision LLC (Costar Technologies, Inc.)
• Inventors: Kaplinsky, Michael
• Primary Examiner(s): Vo; Tung
• Assistant Examiner(s): HOLDER, ANNER N
• Application Number:
  US10/890,912
• Publication Number:
  US 20050141607A1
• Time in Patent Office: 2,071 Days
• Field of Search: 375/240.2, 375/240.12, 375/240.19, 348/36
• US Class Current: 375/240.2
• CPC Class Codes: H04N 23/63 by using electronic viewfinders
• CPC Class Codes: H04N 23/661 Transmitting camera control signals through networks, e.g. control via the Internet
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/843 Demosaicing, e.g. interpolating colour pixel values
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 7/181 for receiving images from a plurality of remote sources

Abstract

A multi-sensor network camera providing up to 360 degrees angle of view. The includes multiple image sensors with individual optics, one or more image processors, compression units and network interfaces mounted in the single housing. The image sensors are positioned in non-parallel planes, cumulatively providing panoramic field of view and image streams originating from all sensors share the same image compression and network interface hardware, providing for low cost implementation. The images from all sensors are transmitted over the network simultaneously via packet interleaving, with appropriate bandwidth reduction achieved by image decimation. Simultaneously with transmission of decimated images from all sensors, full resolution window or entire image of one or more sensors may also transmitted, where the selection of contents is based on motion detection or user setting.

Claims

1. A multi-sensor network camera comprising:

a plurality of image sensors, wherein focal planes of said image sensors are positioned at an angle to each other;

a plurality of image pre-processors each electrically coupled to a respective one of the plurality of sensors, wherein a number of images pre-processors is the same as a number of sensors, wherein each image pre-processor comprises a plurality of first pipelined hardware stages each including a Bayer interpolation hardware module, a color correction matrix transform hardware module, a white balancing color measurement hardware module, an automatic exposure luminance measurement hardware module and a plurality of hardware buffers, wherein flow of video stream pixels is operated on sequentially by said first pipeline hardware stages;

a multiplexor and decimation module for decimating high resolution video streams of a scene from the image sensors to generate low resolution video streams of said scene, and multiplexing said high resolution and said decimated low resolution video streams of said scene;

an image post-processor comprising a plurality of second pipelined hardware stages including a compression hardware module for compressing the high resolution video streams generated by the image sensors and the low resolution video streams from the multiplexor and decimation module, wherein flow of multiplexed video stream pixels is operated on sequentially by said second pipeline hardware stages;

a microprocessor coupled to the plurality of sensors, the plurality of image pre-processors and the image post-processor, wherein said microprocessor is configured to control the operation of the pipeline hardware stages, and to perform white balancing and auto exposure using data from said white balancing color measurement hardware module and said automatic exposure luminance measurement hardware module; and

a network interface for transmitting said compressed high resolution and said compressed low resolution video streams compressed by said compression hardware module via a computer network.

2. Multi-sensor network camera of claim 1, wherein each of said image sensors has individual optics or lens positioned in front of it.

3. Multi-sensor network camera of claim 1, wherein said image sensors are positioned in non-parallel planes as to cumulatively provide 360 degrees field of view along at least one circumference centered at the camera location.

4. Multi-sensor network camera of claim 1, wherein said image sensors are positioned as to provide cumulative field of view covering at least 180 degrees hemisphere centered at camera location.

5. Multi-sensor network camera of claim 1, wherein said network interface is Ethernet network interface.

6. Multi-sensor network camera of claim 1, wherein said network interface supports one or more of a group consisting of TCP, UDP, TFTP, and IP network protocols.

7. Multi-sensor network camera of claim 1, wherein said multiplexor and decimation module is a time-interleaved multiplexing means, wherein transmission of some portion of data stream from one of the said image sensors is followed by the transmission of some portion of data stream from another of said image sensors.

8. Multi-sensor network camera of claim 1, wherein said multiplexor and decimation module is configured to allow selection of the image data streams to be transmitted off camera to be controlled from an external input.

9. Multi-sensor network camera of claim 1, wherein said image sensors are CMOS image sensors with resolution in excess of 1 million pixels.

10. Multi-sensor network camera of claim 1, wherein said multiplexor and decimation module further generates high resolution windows of video streams of portions of said scene, and wherein the network interface transmits said compressed high resolution and said compressed low resolution video streams and said high resolution windows of video streams via the computer network at substantially a video rate.

11. Multi-sensor network camera of claim 1, wherein said multiplexor and decimation module are configured to deliver to said network interface said full resolution video streams if moving objects have been detected in a field of view of the image sensors and-said low resolution video streams if moving objects have not been detected in the field of view of the image sensors.

12. The multi-sensor network camera system of claim 1, further comprising a computer coupled to the computer network for receiving the images and configured to assemble received video streams into one composite video stream and display the composite video stream, wherein the computer is further configured to zoom in the received high resolution video streams.