Image forming and processing device and method for use with no moving parts camera

• US 5,691,765 A
• Filed: 07/27/1995
• Issued: 11/25/1997
• Est. Priority Date: 07/27/1995
• Status: Expired due to Term

• Current Assignee: Sensormatic Electronics LLC (Johnson Controls International plc)
• Original Assignee: Sensormatic Electronics Corporation (Johnson Controls International plc)
• Inventors: Schieltz, Steve W., Glatt, Terry Laurence, Kupersmit, Carl
• Primary Examiner(s): Faile, Andrew
• Assistant Examiner(s): Ho, Tuan V.
• Application Number:
  US08/508,115
• Time in Patent Office: 852 Days
• Field of Search: 348/207, 348/335, 348/340, 348/344, 348/218, 348/143, 348/169, 348/36, 348/39, 382/276, 382/285, 382/293, 395/125, 395/130, 395/136
• US Class Current: 348/335
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G08B 13/19619 Details of casing
• CPC Class Codes: G08B 13/19628 of wide angled cameras and camera groups, e.g. omni-directional cameras, fish eye, single units having multiple cameras achieving a wide angle view
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation

Abstract

A video surveillance system has a camera equipped with a fisheye lens having a substantially hemispheric field of view. The system implements operations equivalent to the panning, tilting and zooming of a conventional camera without the use of moving parts. The lens is mounted vertically above a plane under surveillance. The camera produces a fisheye image made up of a plurality of pixels. The fisheye image is distorted due to the properties of the fisheye lens. The system corrects the distortion by mapping the pixels of the fisheye image to coordinates produced by selecting a particular part of the fisheye image to be viewed. This allows an operator to select parts of the field of view of the fisheye lens and view them as if they had been produced by a camera having a conventional lens being panned tilted or zoomed. The fisheye image formed by the camera is split into four separate image components carried by four bundles of optical fibers. Each bundle has a CCD and associated image processing circuitry which forms an electronic representation of the image component carried by that bundle.

Claims

1. A method of video surveillance comprising the steps of:

providing a video camera having an image plane and a fisheye lens, the fisheye lens having a lens constant indicative of distortion caused by the fisheye lens, a primary axis and a substantially hemispheric field of view and the camera having an image plane, the image plane having a center point through which the primary axis passes;

mounting the camera at a mounting point vertically above a surveillance plane such that the primary axis of the fisheye lens lies substantially perpendicular to the surveillance plane;

causing the video camera to form a fisheye image, the fisheye image being distorted by the fisheye lens and comprising a plurality of pixels, each pixel having unique fisheye coordinates;

generating a corrected image of sub-area in the field of view by the steps of;

providing angular coordinates of an object in the field of view by simulating a hypothetical camera having a hypothetical axis and such that the hypothetical axis passes through the center point and the object in the field of view at specific pan and tilt angles;

selecting an area of the fisheye image, wherein the area comprises a subset of the plurality of pixels, each pixel of the subset being displaced from the specific pan angle by an angle "a";

determining the unique fisheye coordinates of each pixel of the subset of the plurality of pixels by means of a formula comprising the lens constant, the specific tilt angle "b" and the angle "a", said formula being given by

space="preserve" listing-type="equation">X.sub.d =f.cos.sup.-1 (cos(a).cos(b)).cos(tan.sup.-1 (tan(a)/sin(b)))

space="preserve" listing-type="equation">Y.sub.d =f.cos.sup.-1 (cos(a).cos(b)).sin(tan.sup.-1 (tan(a)/sin(b))),

where "f" is the lens constant and X_d and Y_d are the unique fisheye coordinates; and

mapping the pixels in the subset of the plurality of pixels to normal coordinates, thereby forming the corrected image of the sub-area.

2. The method of claim 1 wherein the specific tilt angle is taken relative to the primary axis of the fisheye lens.

3. The method of claim 1 wherein the specific pan angle is taken relative to a prior pan angle about the primary axis.

4. The method of claim 1 wherein the step of mapping the pixels in the subset to the normal coordinates comprises the steps of:

providing a lookup table containing sets of pre-calculated coordinates calculated in accordance with the formula such that for each pixel in the subset of the plurality of pixels there is a corresponding set of angular coordinates, and such that for each set of angular coordinates, there is a set of pre-calculated pixel coordinates;

providing the angular coordinates to the lookup table and in response, causing the lookup table to output the pre-calculated pixel coordinates.

5. The method of claim 4 further comprising the steps of:

storing the plurality of pixels in an image memory, at .addresses corresponding to the pre-calculated coordinates, providing the pre-calculated coordinates to the image memory and receiving as an output from the image memory, the pixels in the subset of the plurality of pixels.

6. The method of claim 1 wherein the step of providing the angular coordinates of the object comprises the step of receiving the specific pan and tilt angles via a user input.

7. The method of claim 1 wherein the angular coordinates are polar coordinates and further comprising the step of converting the polar coordinates into the specific pan and tilt angles.

8. The method of claim 1 wherein the step of providing the angular coordinates of the object comprises the step of selecting an area on a video display and calculating the specific pan and tilt angles of the object based on cartesian coordinates of the sub-area.

9. The method of claim 1 further comprising the step of displaying the corrected image of the sub-area.

10. The method of claim 1 further comprising the step of zooming the corrected image.

11. The method of claim 10 wherein the step of zooming comprises incrementing angle "a".