Omniview motionless camera orientation system

• US 7,382,399 B1
• Filed: 05/21/1999
• Issued: 06/03/2008
• Est. Priority Date: 05/13/1991
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Martin, H. Lee, McCall, Danny A.
• Primary Examiner(s): LEE, MICHAEL
• Application Number:
  US09/315,962
• Time in Patent Office: 3,301 Days
• Field of Search: 348/207, 348/222, 348/239, 348/36, 348/39, 348/47, 348/48, 348/143, 348/147, 348/37, 348/38, 348/576, 382/268, 382/282, 382/284, 382/293, 382/285, 359/708, 359/719, 359/712, 359/725, 359/728, 359/364, 345/425, 345/427
• US Class Current: 348/207.99
• CPC Class Codes: H04N 23/58 Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A method and apparatus for capture of a spherical image is disclosed. The present invention includes at least one camera having a lens with at least a 180° field-of-view for capturing a hemispherical image. In a first embodiment, a second hemispherical image is created corresponding to a mirror image of the hemispherical image captured by the camera. In a second embodiment, two back-to-back cameras capture first and second hemispherical images, respectively. In both embodiments, a converter combines the two images along their outside edges to form a single, spherical image. Finally, the converter stores the complete spherical image for later retrieval and perspective corrected viewing.

Claims

1. A system for providing perspective corrected views of a selected portion of a received optical image captured using a wide angle lens, the received optical image being distorted, the system comprising:

image capture means for receiving signals corresponding to said received optical image and for digitising said signal;

input image memory means for receiving said digitised signal;

input means for selecting a non-predetermined portion of said received image to view;

image transform processor means for processing said digitised signals to produce an output signal corresponding to a perspective corrected image of said selected portion of said received image;

output image memory means for receiving said output signal from said image transform processor means; and

output means connected to said output image memory means for recording or displaying said perspective corrected image of said selected portion;

characterised in that said image transform processor means comprises transform parameter calculation means for calculating transform parameters in response to the selection of said selected portion of said image and processes said digitised signal based on said calculated transform parameters to generate said output signal.

2. A system according to claim 1, comprising a camera imaging system for receiving said optical image and for producing said signals corresponding to said received optical image for output to said image capture means.

3. A system according to claim 2, comprising wide angle lens means mounted on said camera imaging system for producing said optical image for optical conveyance to said camera imaging system.

4. A system according to claim 3, wherein said lens means is one or more fish-eye lenses.

5. A system according to claim 1, wherein said input means provides for input to said image transform processor means of one or more of:

a direction of view;

tilting of a viewing angle;

rotation of a viewing angle;

pan of said viewing angle;

focus of said image and magnification of the selected portion of the image.

6. A system according to claim 5, wherein tilting of said viewing angle through at least 180 degrees is provided for.

7. A system according to claim 4, wherein rotation of said viewing angle through 360 degrees is provided for.

8. A system according to any one of claims 5, wherein pan of said viewing angle through at least 180 degrees is provided for.

9. A system according to claim 8, wherein pan of said viewing angle through 360 degrees is provided for.

10. A system according to claim 1, wherein said input means is a user-operated manipulator switch means.

11. A system according to any one of claims 1, wherein said input means is a signal from a computer input means.

12. A system according to claim 1, wherein said image transform processing means is programmed to implement the following two equations:

$x=\frac{R<br><br>\left\{\mathrm{uA}-\mathrm{vB}+\mathrm{mR}<br><br>\text{\hspace{1em}<br><br>}<br><br>\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{\beta <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{\delta <br><br>}\right\}}{\sqrt{u^2+v^2+m^2<br><br>\text{\hspace{1em}<br><br>}<br><br>R^2}}$$y=\frac{R<br><br>\left\{\mathrm{uC}-\mathrm{vD}+\mathrm{mR}<br><br>\text{\hspace{1em}<br><br>}<br><br>\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{\beta <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{\delta <br><br>}\right\}}{\sqrt{u^2+v^2+m^2<br><br>\text{\hspace{1em}<br><br>}<br><br>R^2}}$

where;

A=(cos φ

cos δ

−

sin φ

sin δ

cos β

)

B=(sin φ

cos δ

+cos φ

sin δ

cos β

)

C=(cos φ

sin δ

+sin φ

cos δ

cos β

)

D=(sin φ

sin δ

+cos φ

cos δ

cos β

)

and where;

R=radius of the image circle

β

=zenith angle

δ

=Azimuth angle in image plane

φ

=Object plane rotation angle

m=Magnification

u,v=object plane coordinates

x,y=image plane coordinates.

13. A method for providing perspective corrected views of a selected portion of an optical image captured with a wide angle lens, the received optical image being distorted, the method comprising:

providing a digitised signal corresponding to said optical image;

selecting a non-predetermined portion of said optical image;

transforming said digitised signal to produce an output signal corresponding to a perspective corrected image of said selected portion of said received image; and

displaying or recording said perspective corrected image of said selected portion;

characterised in that said step of transforming said digitised signal comprises calculating transform parameters in response to the selection of said selected portion of said image, said calculated transform parameters being used to control said transformation of the digitised signal to generate said output signal.

14. A method according to claim 13, comprising first receiving said optical image, producing signals corresponding to said received optical image and digitizing said signals.

15. A method according to claim 13, comprising capturing said optical image with one or more fish-eye lenses.

16. A method according to any one of claims 13, wherein said step of selecting the portion of the image to view comprises selecting one or more of:

a direction of view;

tilting of a viewing angle;

rotation of a viewing angle;

pan of said viewing angle;

focus of said image and magnification of the selected portion of the image.

17. A method according to claim 16, wherein tilting of said viewing angle through at least 180 degrees is provided for.

18. A method according to claim 16, wherein rotation of said viewing angle through 360 degrees is provided for.

19. A method according to any one of claims 16, wherein pan of said viewing angle through at least 180 degrees is provided for.

20. A method according to claim 19, wherein pan of said viewing angle through 360 degrees is provided for.

21. A method according to any one of claims 13, wherein selection of said portion of the image to view is achieved using a user-operated manipulator switch means.

22. A method according to any one of claims 13, wherein selection of said portion of the image to view is controlled by a signal from a computer input means.

23. A method according to any one of claims 13, wherein said image transformation implements the following two equations:

$x=\frac{R<br><br>\left\{\mathrm{uA}-\mathrm{vB}+\mathrm{mR}<br><br>\text{\hspace{1em}<br><br>}<br><br>\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{\beta <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{\delta <br><br>}\right\}}{\sqrt{u^2+v^2+m^2<br><br>\text{\hspace{1em}<br><br>}<br><br>R^2}}$$y=\frac{R<br><br>\left\{\mathrm{uC}-\mathrm{vD}+\mathrm{mR}<br><br>\text{\hspace{1em}<br><br>}<br><br>\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{\beta <br><br>}<br><br>\text{\hspace{1em}<br><br>}<br><br>\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>\mathrm{\delta <br><br>}\right\}}{\sqrt{u^2+v^2+m^2<br><br>\text{\hspace{1em}<br><br>}<br><br>R^2}}$

where;

A=(cos φ

cos δ

−

sin φ

sin δ

cos β

)

B=(sin φ

cos δ

+cos φ

sin δ

cos β

)

C=(cos φ

sin δ

+sin φ

cos δ

cos β

)

D=(sin φ

sin δ

+cos φ

cos δ

cos β

)

and where;

R=radius of the image circle

β

=zenith angle

δ

Azimuth angle in image plane

φ

=Object plane rotation angle

m=Magnification

u,v=object plane coordinates

x,y=image plane coordinates.

24. A method according to any one of claims 13, wherein a plurality of portions of said image are selected for viewing and are displayed either simultaneously or consecutively.

25. A method according to any one of claims 13, wherein the image is viewed interactively by repeating the steps of selecting, transforming and displaying said portion of the image.

26. A method according to claim 13, wherein said step of transforming the image is based on lens characteristics of the wide angle lens.

27. A method according to claim 26, wherein the step of transformation is based on azimuth angle invariability and equidistant projection.

28. A method according to claim 13, wherein the step of transforming the image is performed at real time video rates.

29. A method for providing perspective corrected views of a selected portion of a spherical image comprising two images captured with a fisheye lens, the received spherical image being distorted, the method comprising:

providing a digitised signal corresponding to said spherical image;

selecting a portion of said spherical image;

transforming said digitised signal to produce an output signal corresponding to a perspective corrected image of said selected portion of said spherical image; and

displaying or recording said perspective corrected image of said selected portion;

characterised in that said step of transforming said digitised signal comprises calculating transform parameters for said selected portion of said image, said calculated transform parameters being used to control said transformation of the digitised signal to generate said output signal.

30. A system for providing perspective corrected views of a selected poriton of a received optical image captured using a wide angle lens, the received optical image being distorted, the system comprising:

image capture means for receiving signals corresponding to said received optical image and for digitizing said signal;

input image memory means for receiving said digitized signal;

input means for selecting a portion of said received image to view;

image transform processor means for processing said digitized signals to produce an output signal corresponding to a perspective corrected image of said selected portion of said received image;

output image memory means for receiving said output signal from said image transform processor means; and

output means connected to said output image memory means for recording or displaying said perspective corrected image of said selected portion;

characterized in that the input means is adapted to input a pan, tilt and magnification for the selected portion; and

said image transform processor means comprises transform parameter calculation means for calculating transform parameters in response to the input pan, tilt and magnification for the selected portion of said image and processes said digitized signal based on said calculated transform paraeters to generate said output signal.

31. A method for providing perspective corrected views of a selected portion of an optical image captured with a wide angle lens, the received optical image being distorted, the method comprising:

providing a digitized signal corresponding to said optical image;

selecting a portion of said optical image;

transforming said digitized signal to produce an output signal corresponding to a perspective corrected image of said selected portion of said received image; and

displaying or recording said perspective corrected of said selected portion;

characterized in that the step of selecting the portion inputting a pan, tilt and magnification; and

said step of transforming said digitized signal comprises calculating transform parameters in response to the input pan, tilt and magnification for the selected portion of said image, said calculated transform parameters being used to control said transformation of the digitized signal to generate said output signal.

32. A method of processing images, comprising:

(a) capturing a wide-angle image from a motionless lens, the wide-angle image being distorted relative to human perception;

(b) creating a digitized image corresponding to the captured wide-angle image;

(c) receiving incremental inputs corresponding to pan, tilt, and magnificatio;

(d) defining a portion of the digitized image with motionless pan, tilt, and magnification;

(e) transforming the portion of the digitized wide-angle image to remove the distortion in real-time relative to the received incremental inputs corresponding to pan, tilt, and magnification; and

(f) displaying the transformed portion.

33. The method of claim 32, wherein the step of displaying is performed at video rates.

34. The method of claim 32, wherein the wide-angle image is captured with a fisheye lens.

35. The method of claim 32, wherein the act of removing the distortion comprises direct mapping from a 2-D image plane to a 2-D object plane.

36. The method of claim 32, further comprising displaying multiple undistorted portions with different orientations simultaneously.

37. The method of claim 32, wherein the wide-angle image is hemispherical.

38. The method of claim 32, wherein the wide-angle image is sphereical.