Method for directly scanning a rectilinear imaging element using a non-linear scan

• US 6,243,131 B1
• Filed: 01/17/1995
• Issued: 06/05/2001
• Est. Priority Date: 05/13/1991
• Status: Expired due to Term

• Current Assignee: Sony Semiconductor Solutions Corporation (Sony Group Corp.)
• Original Assignee: Interactive Pictures Corp. (IPIX Corporation)
• Inventors: Martin, H. Lee
• Primary Examiner(s): LEE, MICHAEL
• Application Number:
  US08/373,446
• Time in Patent Office: 2,331 Days
• Field of Search: H04/N.530, H04/N.700, H04/N.5225, H04/N.118, 348/36, 348/43, 348/44, 348/39, 348/46, 348/147, 348/143, 348/315, 348/335, 348/340, 348/294, 348/311, 348/342, 348/91, 382/293, 382/295, 382/296, 382/297, 382/298, 395/137-139, 257/231
• US Class Current: 348/36
• CPC Class Codes: A61B 2090/0813 Accessories designed for easy sterilising, i.e. re-usable
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G08B 13/19628 of wide angled cameras and camera groups, e.g. omni-directional cameras, fish eye, single units having multiple cameras achieving a wide angle view
• CPC Class Codes: G08B 13/1963 Arrangements allowing camera rotation to change view, e.g. pivoting camera, pan-tilt and zoom [PTZ]
• CPC Class Codes: G08B 13/19656 Network used to communicate with a camera, e.g. WAN, LAN, Internet
• CPC Class Codes: G08B 13/19658 Telephone systems used to communicate with a camera, e.g. PSTN, GSM, POTS
• CPC Class Codes: G08B 13/19676 Temporary storage, e.g. cyclic memory, buffer storage on pre-alarm
• CPC Class Codes: G08B 13/19689 Remote control of cameras, e.g. remote orientation or image zooming control for a PTZ camera
• CPC Class Codes: H04N 1/2158 using a detachable storage unit
• CPC Class Codes: H04N 1/217 Interfaces allowing access to a single user
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 25/00 Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 7/002 Special television systems not provided for by H04N7/007 - H04N7/18 still pictures via a television channel H04N1/00098
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A method for capturing and directly scanning a rectilinear imaging element using a non-linear scan is incorporated into a single chip comprising at least a sensor array and an MSD. The method directly addresses each picture element of an analog image captured with an imaging device having either a partial spherical field of view or a conventional two-dimensional field of view. An image transform processor is used to process the captured image depending upon the particular portion of interest of the image. In the case of a non-linear scan, the image transform processor is provided with the capability of geometrically filtering the portion of interest of the captured image such that a two-dimensional, undistorted image is displayed at the monitor. A CMOS active pixel image sensor (APS) or Charge Injection Diode (CID) camera array are used to capture the image to be scanned. The image transform processor of the present invention is a Mixed-signal Semiconductor Device (MSD). The image transform processor corrects any predetermined distortion introduced by the image sensor array.

Claims

1. An image processor for directly de-warping data collected using a directly addressable non-linear scan of a rectilinear imaging element comprising:

an imaging system comprising an array of a plurality of picture element sensors for capturing a light image and

a digital image transform processor, coupled to the imaging system, comprising a digital application specific integrated circuit for implementing a direct scan of said array and directly addressing selected pixels determined by a de-warping logic circuit of the digital image transform processor to select an image portion, the digital image transform processor eliminating predetermined distortion introduced by the imaging system and outputting a real-time distortion corrected image to vertical line averaging circuitry followed by post-processing circuitry.

2. A semiconductor device as recited in claim 1 wherein each said picture element sensor comprises a CMOS active pixel sensor.

3. A semiconductor device as recited in claim 1 wherein at least one picture element sensor receives a coating to improve light energy receptivity.

4. An imaging system comprising at least two semiconductor devices according to claim 1, the devices being placed a predetermined interdevice distance from one another, each device outputing two dimensional image data, the data being adaptable for conversion to three dimensional data.

5. A method for providing a distortion-free image for use in an image processing system comprising the steps of:

capturing, by a picture element sensor array, picture elements for a hemispherical/two-dimensional light image; and

using a digital image transform processor for directly addressing the picture elements in a non-linear sequence determined by a de-warping logic circuit of the digital image transform processor in real-time to eliminate predetermined distortion introduced by the array and sending output data to vertical line averaging circuitry followed by post-processing circuitry.

6. The method for providing a distortion-free image according to claim 5, further including the step of:

non-linearly scanning said array to eliminate said any distortion from the received image.

7. A processing terminal comprising:

an imaging system comprising a two-dimensional array of a plurality of picture element sensors for capturing a hemispherical/two-dimensional light image; and

a digital image transform processor for scanning said array in a non-linear sequence determined by a de-warping logic circuit of the digital image transform processor for selecting an image portion, the digital image transform processor eliminating predetermined distortion introduced by the imaging system and outputting a real-time distortion corrected image and sending output data to vertical line averaging circuitry followed by post-processing circuitry, wherein intermediate storage of data in random access memory after scanning is avoided.

8. A method for geometrically filtering a portion of a captured distorted image to output a selected undistorted portion of the distorted image according to predetermined viewing angles and a magnification, the method comprising the steps of:

capturing the distorted image;

receiving the viewing angles and the magnification;

using a digital image transform processor for converting the predetermined viewing angles and the magnification to a nonlinear sequence of scanning addresses determined by a de-warping logic circuit of the digital image transform processor for the portion of the captured distorted image; and

scanning the nonlinear sequence of scanning addresses to output the selected undistorted portion in real-time and sending output data to vertical line averaging circuitry followed by post-processing circuitry, wherein intermediate storage of data in random access memory after scanning is avoided.

9. The method of claim 8, wherein the distorted image is distorted by a wide angle lens.

10. The method of claim 8, wherein the distorted image is distorted by a fisheye lens.

11. Apparatus for geometrically filtering a portion of a captured distorted image to output, without storing in memory, a selected undistorted portion of the distorted image according to predetermined viewing angles and a magnification, the apparatus comprising:

an input for receiving the predetermined viewing angles and the magnification;

a digital image transform processor responsive to the input and configured to convert the predetermined viewing angles and the magnification to a nonlinear sequence of scanning addresses determined by a de-warping logic circuit of the digital image transform processor;

an array of picture element sensors coupled to the digital image transform processor and configured to capture the distorted image; and

a scanning circuit configured to receive the nonlinear sequence of scanning addresses from the digital image transform processor and to scan the nonlinear sequence of scanning addresses determined by a de-warping logic circuit of the digital image transform processor to output the selected undistorted portion in real-time and send output data to vertical line averaging circuitry followed by post-processing circuitry, wherein intermediate storage of data in random access memory after scanning is avoided.

12. The apparatus of claim 11, further including a camera, the camera including the array.

13. The apparatus of claim 12, wherein the camera includes a wide angle lens in optical communication with the array, the distorted image being distorted by the wide angle lens.

14. The apparatus of claim 12, wherein the camera includes a fisheye lens in optical communication with the array, the distorted image being distorted by the fisheye lens.

15. The apparatus of claim 12, wherein the image processor comprises a geometric filter.

16. The apparatus of claim 11, wherein each of the picture elements sensors comprises a CMOS active pixel sensor.

17. An image processing device for directly de-warping data collected by a plurality of image sensors using a directly addressable non-linear scan of a rectilinear imaging element comprising:

the plurality of the image sensors, arranged to capture a light image to provide an array of data;

a converter engine having a video input unit for providing camera synchronization, camera control and host control and timing and address generation and a video processing unit with a digital image transform processor, where the video input unit and the video processing unit are coupled to a random access memory multiplexer which receives the array from the image sensors, where the video processing unit includes a digital application specific integrated circuit for implementing a direct scan of said array by directly addressing selected pixels determined by a de-warping logic circuit of the digital image transform processor to select an image portion and use vertical line averaging circuitry to eliminate distortion of the light image and output a real-time distortion corrected image.