Motionless camera orientation system distortion correcting sensing element

• US 6,201,574 B1
• Filed: 02/05/1997
• Issued: 03/13/2001
• Est. Priority Date: 05/13/1991
• Status: Expired due to Fees

• Current Assignee: Acra Products, LLC, Sony Corporation (Sony Group Corp.)
• Original Assignee: Interactive Pictures Corp. (IPIX Corporation)
• Inventors: Martin, H. Lee
• Primary Examiner(s): Lee, Michael
• Application Number:
  US08/795,005
• Time in Patent Office: 1,497 Days
• Field of Search: 348/36, 348/43, 348/44, 348/39, 348/46, 348/147, 348/143, 348/332, 348/340, 348/315, 348/335, 348/342, 348/294, 348/311, 348/91, 382/293-298, 395/137-139, 250/208.1, 250/206.2, 702/3, 73/170.16
• US Class Current: 348/315
• CPC Class Codes: A61B 2090/0813 Accessories designed for easy sterilising, i.e. re-usable
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G08B 13/19628 of wide angled cameras and camera groups, e.g. omni-directional cameras, fish eye, single units having multiple cameras achieving a wide angle view
• CPC Class Codes: H04N 1/2158 using a detachable storage unit
• CPC Class Codes: H04N 1/217 Interfaces allowing access to a single user
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 25/00 Circuitry of solid-state image sensors [SSIS]; Control thereof
• CPC Class Codes: H04N 5/2628 Alteration of picture size, shape, position or orientation, e.g. zooming, rotation, rolling, perspective, translation
• CPC Class Codes: H04N 7/002 Special television systems not provided for by H04N7/007 - H04N7/18 still pictures via a television channel H04N1/00098
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

A device for omnidirectional image viewing providing pan-and-tilt orientation, rotation, and magnification within a hemispherical field-of-view that utilizes no moving parts. The imaging device is based on the effect that the image from a fisheye lens, which produces a circular image of an entire hemispherical field-of-view, can be mathematically corrected using high-speed electronic circuitry. More specifically, an incoming fisheye image is captured on a Charge Coupled Diode array in such a way as to remove the distortion caused by the lens by locating the picture pickup elements in a nonlinear manner described herein. As a result, this device can accomplish the functions of pan, tilt, rotation, and zoom throughout a hemispherical field-of-view without the need for any mechanical mechanisms. The preferred embodiment of the image capture device can provide corrected images at standard video frame rates, compatible with standard video equipment. The device can be used for aiming the video device in any application where a conventional pan-and-tilt or orientation mechanism might be considered including inspection, monitoring, surveillance, video teleconferencing, and target acquisition.

Claims

1. An electronic imaging system comprising:

a wide-angle lens for producing a wide-angle image, the wide-angle lens producing a perspective distortion in the wide-angle image; and

an imaging sensor, having a surface in optical communication with the wide-angle lens, for generating an output signal representing at least a portion of a corrected wide-angle image, the imaging sensor comprising a plurality of imaging elements, the plurality of imaging elements having a distribution on the surface representable by a nonlinear function,

wherein the plurality of imaging elements are distributed such that the perspective distortion in the wide-angle image is substantially corrected to produce the corrected wide-angle image, at least three of the plurality of imaging elements being distributed to define a first curved row of imaging elements, each of the imaging elements defining the first curved row corresponding to a different point on a first straight line along the corrected wide-angle image,

thereby allowing for direct mapping between the at least three sensor elements and the points on the first straight line in the corrected wide-angle image.

2. The system of claim 1 wherein the plurality of imaging elements are distributed as a two-dimensional array.

3. The system of claim 2 wherein each of the plurality of imaging elements is electronically addressable by a two-dimensional address.

4. The system of claim 1 wherein each of the plurality of imaging elements includes a pixel of a charge-coupled device.

5. The system of claim 1 wherein the output signal is a video signal.

6. The system of claim 1 wherein the output signal includes a plurality of image data.

7. The system of claim 1 further comprising a display for displaying at least a portion of the corrected wide-angle image responsive to the output signal.

8. The system of claim 1 further comprising an image processing unit for selectively magnifying a first region of the corrected wide-angle image responsive to the output signal and a user command signal.

9. The system of claim 8 further comprising an image processing unit for selectively panning from a second region of the corrected wide-angle image to a third region of the corrected wide-angle image responsive to the output signal and a user command signal.

10. The system of claim 8 further comprising a user interface device coupled to the image processing unit for generating the user command signal.

19. The electronic imaging system of claim 1, further including a second curved row of imaging elements at least partially different from the first curved row of imaging elements, the second curved row of imaging elements being defined by at least three of the plurality of imaging elements, each of the imaging elements defining the second curved row corresponding to a different point on a second straight line along the corrected wide-angle image different from the first straight line.

11. A sensor used in an electronic imaging system having a wide-angle lens for producing a wide-angle image having a perspective distortion, the sensor comprising:

a surface in optical communication with the wide-angle lens; and

a plurality of imaging elements, coupled to the surface, for generating an output signal representing at least a portion of a corrected wide-angle image, the plurality of imaging elements having a distribution on the surface representable by a nonlinear function,

wherein the plurality of imaging elements are distributed such that the perspective distortion in the wide-angle image is substantially corrected to produce the corrected wide-angle image, at least three of the plurality of imaging elements being distributed to define a first curved row of imaging elements, each of the imaging elements defining the first curved row corresponding to a different point on a first straight line along the corrected wide-angle image.

thereby allowing for direct mapping between the at least three sensor elements and the points on the first straight line in the corrected wide-angle image.

12. The sensor of claim 11 wherein the plurality of imaging elements are distributed as a two-dimensional array.

13. The sensor of claim 12 wherein each of the plurality of imaging elements is electronically addressable by a two-dimensional address.

14. The sensor of claim 11 wherein each of the plurality of imaging elements includes a pixel of a charge-coupled device.

15. The sensor of claim 11 wherein the output signal is a video signal.

16. The sensor of claim 11 wherein the output signal includes a plurality of image data.

20. The sensor of claim 11, further including a second curved row of imaging elements at least partially different from the first curved row of imaging elements, the second curved row of imaging elements being defined by at least three of the plurality of imaging elements, each of the imaging elements defining the second curved row corresponding to a different point on a second straight line along the corrected wide-angle image different from the first straight line.

17. A processing terminal comprising:

a wide-angle lens for producing a wide-angle image, the wide-angle lens producing a perspective distortion in the wide-angle image; and

an imaging sensor, having a surface in optical communication with the wide-angle lens, for generating an output signal representing at least a portion of a corrected wide-angle image, the imaging sensor comprising a plurality of imaging elements, the plurality of imaging elements having a distribution on the surface representable by a nonlinear function,

wherein the plurality of imaging elements are distributed such that the perspective distortion in the wide-angle image is substantially corrected to produce the corrected wide-angle image, at least three of the plurality of imaging elements being distributed to define a first curved row of imaging elements, each of the imaging elements defining the first curved row corresponding to a different point on a first straight line along the corrected wide-angle image;

thereby allowing for direct mapping between the at least three sensor elements and the points on the first straight line in the corrected wide-angle image.

21. The processing terminal of claim 17, further including a second curved row of imaging elements at least partially different from the first curved row of imaging elements, the second curved row of imaging elements being defined by at least three of the plurality of imaging elements, each of the imagging elements defining the second curved row corresponding to a different point on a second straight line along the corrected wide-angle image different from the first straight line.

18. A processing terminal configured to be used with a wide-angle lens, the processing terminal comprising:

a surface, in optical communication with the wide-angle lens, configured to receive a wide-angle image having a perspective distortion associated with the wide-angle lens; and

a plurality of imaging elements, coupled to the surface, for generating an output signal representing at least a portion of a corrected wide-angle image, the plurality of imaging elements having a distribution on the surface representable by a nonlinear function,

wherein the plurality of imaging elements are distributed such that the perspective distortion in the wide-angle image is substantially corrected to produce the corrected wide-angle image, at least three of the plurality of imaging elements being distributed to define a first curved row of imaging elements, each of the imaging elements defining the first curved row corresponding to a different point on a first straight line along the corrected wide-angle image.

thereby allowing for direct mapping between the at least three sensor elements and the points on the first straight line in the corrected wide-angle image.

22. The processing terminal of claim 18, further including a second curved row of imaging elements at least partially different from the first curved row of imaging elements, the second curved row of imaging elements being defined by at least three of the plurality of imaging elements, each of the imaging elements defining the second curved row corresponding to a different point on a second straight line along the corrected wide-angle image different from the first straight line.

23. A sensor configured to optically receive a wide-angle image, the wide-angle image having a perspective distortion associated with a wide-angle lens, the sensor comprising a plurality of imaging elements nonlinearity distributed in space, at least three of the plurality of imaging elements being distributed in space so as to define a first curved row of imaging elements, each of the at least three imaging elements defining the first curved row corresponding to a different point on a first straight line along a corrected wide-angle image, the corrected wide-angle image being the wide-angle image substantially without the perspective distortion,

thereby allowing for direct mapping between the at least three sensor elements and the points on the first straight line in the corrected wide-angle image.

24. The sensor of claim 23, further including a second curved row of imaging elements at least partially different from the first curved row of imaging elements, the second curved row of imaging elements being defined by at least three of the plurality of imaging elements, each of the imaging elements defining the second curved row corresponding to a different point on a second straight line along the corrected wide-angle image different from the first straight line.

25. The sensor of claim 24, wherein a set of the image elements defining the first curved row and a set of the imaging elements defining the second curved row comprise two different imaging elements in common.

26. The sensor of claim 25, wherein the two different imaging elements are diametrically opposed.

27. The sensor of claim 23, wherein the plurality of imaging elements are distributed as a two-dimensional array.

28. The sensor of claim 23, wherein the sensor is configured to generate a signal representing at least a portion of the corrected wide-angle image.

29. The sensor of claim 23, wherein sensor is configured to receive a plurality of wide-angle images over time each having the distortion, the sensor being configured to generate a signal representing at least a portion of each of a plurality of corrected wide-angle images over time, each of the corrected wide-angle images comprising one of the plurality of wide-angle images substantially without the distortion.

30. The sensor of claim 29, wherein the signal represents the at least a portion of each of the plurality of corrected wide-angle images at a real-time video rate.

31. The sensor of claim 30, wherein the real-time video rate is about 30 images per second.

32. The sensor of claim 23, wherein the sensor comprises a charge-coupled device, and wherein each of the plurality of imaging elements comprises a picture element of the charge-couple device.

33. A method comprising:

optically receiving at a sensor a wide-angle image, the wide-angle image having a perspective distortion associated with a wide-angle lens, the sensor comprising a plurality of imaging elements nonlinearity distributed in space, at least three of the plurality of imaging elements being distributed in space so as to define a first curved row of imaging elements, each of the imaging elements defining the first curved row corresponding to a different point on a first straight line along a corrected wide-angle image, the corrected wide-angle image being the wide-angle image substantially without the perspective distortion; and

generating a signal representing at least a portion of the corrected wide-angle image by directly mapping between the at least three sensor elements and the points on the first straight line in the corrected wide-angle image.

34. The method of claim 33, wherein the step of generating the signal comprises the steps of:

scanning at least a portion of the first curved row of imaging elements; and

scanning at least a portion of a second curved row of imaging elements at least partially different from the first curved row of imaging elements, the second curved row of imaging elements being defined by at least three of the plurality of imaging elements, each of the imaging elements defining the second curved row corresponding to a different point on a second straight line along the corrected wide-angle image different from the first straight line.

35. The method of claim 34, wherein a set of the imaging elements defining the first curved row and a set of the imaging elements defining the second curved row comprise two different imaging elements in common.

36. The method of claim 35, wherein the two different imaging elements are diametrically opposed.