Panoramic video system with real-time distortion-free imaging

• US 7,336,299 B2
• Filed: 01/15/2004
• Issued: 02/26/2008
• Est. Priority Date: 07/03/2003
• Status: Active Grant

• Current Assignee: Physical Optics Corporation
• Original Assignee: Physical Optics Corporation
• Inventors: Ro, Sookwang, Bennahmias, Mark, Agurok, Il'ya, Kostrzewski, Andrew A
• Primary Examiner(s): HO, TUAN V
• Application Number:
  US10/758,829
• Publication Number:
  US 20060023105A1
• Time in Patent Office: 1,503 Days
• Field of Search: 348/335, 348/207.99, 348/143, 348/333.06, 348/333.07, 359/727
• US Class Current: 348/207.99
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A panoramic annular lens system (PAL), a unitary video camera and a PC-based software system that unwraps a 360° video image into a seamless, distortion free horizontal image image in real time. The PAL system of the preferred embodiment has a 360° horizontal field of view and a 90° vertical field of view in a 40 mm diameter compact package. The invention is not limited to any particular type of lens system. In fact, there are numerous lens systems for providing a 360° panoramic view. The video camera may be a CCD or CMOS based device having a pixel resolution of either 1280×1024 (high resolution) or 720×480 (NTSC). The unwrapping system is a radiometric ray tracing program carried out using a computer'"'"'s graphics card capabilities to produce highly efficient regional transformation while minimizing software overhead. The result is real time, high resolution 30 fps conversion from a spherical distorted image to a flat panoramic image in Cartesian coordinates.

Claims

1. A method of providing a real-time panoramic video image in a rectangular format;

the method comprising the steps of;

a) providing a panoramic annular lens system to capture a 360°

viewed annular image;

b) focusing said 360°

viewed annular image on a video camera image plane;

c) transferring a data signal output of said camera image plane to a personal computer;

d) utilizing said personal computer to unwrap said annular image into a substantially distortion free rectangular image at a rate of at least 30 fps; and

e) presenting said rectangular image on a visual display; and

wherein step d) comprises the steps of utilizing radiometric ray tracing to first convert said annular image to a distorted unwrapped image and then to convert said distorted unwrapped image to an undistorted unwrapped image.

2. A method of providing a real-time panoramic video image in a rectangular format;

the method comprising the steps of;

a) providing a panoramic annular lens system to capture a 360°

viewed annular image;

b) focusing said 360°

viewed annular image on a video camera image plane;

c) transferring a data signal output of said camera image plane to a personal computer;

d) utilizing said personal computer to unwrap said annular image into a substantially distortion free rectangular image at a rate of at least 30 fps; and

e) presenting said rectangular image on a visual display; and

wherein step d) comprises the step of employing a vertex-based transformation using graphics processing units of said personal computer.

3. The method recited in claim 2 wherein in step a) providing said panoramic annular lens system comprises the step of providing a hyperboloidal lens and ellipsoidal mirror.

4. The method recited in claim 2 wherein in step b) providing said video camera comprises the step of providing a CCD image plane.

5. The method recited in claim 2 wherein in step b) providing said video camera comprises the step of providing CMOS image plane.

6. The method recited in claim 2 wherein step d) comprises the step of using at least one graphics card of said personal computer to unwrap said annular image.

7. An apparatus for providing a real-time panoramic video image in a rectangular format;

the apparatus comprising;

a panoramic annular lens system configured for capturing 360°

viewed annular image;

a video camera having an image plane for receiving said annular image and generating a corresponding data signal output;

a computer receiving said data signal output;

a graphics card and at least one software module in said computer for unwrapping said data signal output from an annular image into a substantially undistorted rectangular image at a rate of at least 30 fps; and

a visual display for displaying said rectangular image;

wherein said software module has a program for radiometric ray tracing to first convert said annular image to a distorted unwrapped image and then to convert said distorted unwrapped image to an undistorted unwrappen image.

8. An aparatus for providing a real-time panoramic video image in a rectangular format;

the apparatus comprising;

a panoramic anular lens system configured for capturing 360°

viewed annular image;

a video camera having an image plane for receiving said annular image and generating a corresponding data signal output;

a computer receiving said data signal output;

a graphics card and at least one software module in said computer for unwrapping said data signal output from an annular image into a substantially undistorted rectangular image at a rate of at least 30 fps; and

a visual display for displaying said rectangular image; and

wherein said software module has a program for vertex-based transformation for unwrapping said annular image.

9. The apparatus recited in claim 8 further comprising means for capturing said data signal output;

means for converting said video image from said data signal output;

means for manipulating said converted video image; and

means for rendering said image in a Cartesian format.

10. The apparatus recited in claim 8 wherein said panoramic annular lens system has a hyperboloidal lens and an ellipsoidal mirror.

11. The apparatus recited in claim 8 wherein said video camera has a COD imaging plane.

12. The apparatus recited in claim 8 wherein said vide camera has a CMOS imaging plane.

13. A panoramic video system having real-time distortion-free imaging;

the system comprising;

a panoramic optical system having at least one optical element for viewing a 360°

field of view and focusing a corresponding image on an image plane;

a video camera having a sensing element at said image plane for converting said image into a corresponding video signal;

a computer receiving said video signal and having at least one graphics processing unit employing a vertex-based transformation program for configuring a substantially distortion-free rectangular display of said image at a rate of at least 30 fps; and

a monitor for presenting said display.

14. The panoramic video system of claim 13 wherein said optical system optical element comprises an annular element and said corresponding image is an annular image of said 360°

field of view.

15. The panoramic video system of claim 13 wherein said video camera comprises a CCD sensing element.

16. The panoramic video system of claim 13 wherein said video camera comprises a CMOS sensing element.

17. The panoramic video system of claim 13 wherein said video camera sensing element has a pixel resolution of at least 1280×

1024.

18. The panoramic video system of claim 13 wherein said video camera sensing element has a pixel resolution of at least 720×

480.

19. The panoramic video system of claim 13, said computer comprising at least one graphics card for configuring said rectangular display.