METHOD AND APPARATUS FOR OBTAINING PANORAMIC AND RECTILINEAR IMAGES USING ROTATIONALLY SYMMETRIC WIDE-ANGLE LENS

• US 20150062292A1
• Filed: 09/04/2013
• Published: 03/05/2015
• Est. Priority Date: 09/04/2013
• Status: Active Grant

• Current Assignee: Gyeongil Kweon
• Original Assignee: Gyeongil Kweon
• Inventors: KWEON, Gyeongil
• Granted Patent: US 9,762,795 B2
• Time in Patent Office: Days
• US Class Current: 348/37
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture
• CPC Class Codes: H04N 23/81 for suppressing or minimising disturbance in the image signal generation

Abstract

The present invention provides mathematically accurate image processing algorithms for extracting natural looking panoramic images and distortion-free rectilinear images from images acquired using a camera equipped with a wide-angle lens which is rotationally symmetric about an optical axis and devices implementing such algorithms. Imaging systems using this method can be used not only in security•surveillance applications for indoor and outdoor environments, but also in diverse areas such as video phone for apartment entrance door, rear view camera for vehicles, visual sensor for unmanned aerial vehicles and robots, camera phone, PC camera, and broadcasting camera. Also, it can be used to obtain panoramic or rectilinear photographs using a digital camera.

Claims

1. An imaging system comprising:

an image acquisition means for acquiring an uncorrected image plane using a wide-angle lens rotationally symmetric about an optical axis,

an image processing means for extracting a panoramic image plane from the uncorrected image plane; and

an image display means for displaying the panoramic image plane on a rectangular screen,

wherein,

if a coordinate of an image point on the rectangular screen corresponding to an object point having a coordinate (X, Y, Z) in a world coordinate system, which has a nodal point of the wide-angle lens as an origin and a vertical line passing through the origin as an Y-axis and an intersection line between a reference plane, which is a plane containing both the Y-axis and the optical axis of the wide-angle lens, and a horizontal plane perpendicular to the Y-axis as a Z-axis, is given as (x″

, y″

),

the lateral coordinate x″

of the image point is given as x″

=cψ

,

and the longitudinal coordinate y″

of the image point is given as y″

=cF(δ

),

wherein,

c is a proportionality constant,

ψ

is a horizontal incidence angle an incident ray originating from the object point makes with an Y-Z plane and is given as

2. The imaging system of claim 1,

wherein the function F(d) of the vertical incidence angle d of the incident ray is given by any one among F(δ

)=tan δ

, F(δ

)=δ and

3. An imaging system comprising:

an image acquisition means for acquiring an uncorrected image plane using a fisheye lens rotationally symmetric about an optical axis,

an image processing means for extracting a panoramic image plane from the uncorrected image plane; and

an image display means for displaying the panoramic image plane on a rectangular screen,

wherein,

if a coordinate of an image point on the rectangular screen corresponding to an object point having a coordinate (X, Y, Z) in a world coordinate system, which has a nodal point of the fisheye lens as an origin and a vertical line passing through the origin as an Y-axis and an intersection line between a reference plane, which is a plane containing both the Y-axis and the optical axis of the fisheye lens, and a horizontal plane perpendicular to the Y-axis as a Z-axis, is given as (x″

, y″

),

the lateral coordinate x″

of the image point is proportional to a horizontal incidence angle ψ

,

herein, the horizontal incidence angle ψ

is an angle an incident ray originating from the object point makes with an Y-Z plane and is given as

4. The imaging system of claim 3 further comprising an image selection means.

5. The imaging system of claim 4,

wherein the image selection means comprise a situation awareness module.

6. The imaging system of claim 4,

wherein the image selection means comprise a direction sensing means.

7. An imaging system comprising:

an image acquisition means for acquiring an uncorrected image plane using a fisheye lens rotationally symmetric about an optical axis,

an image processing means for extracting a panoramic image plane from the uncorrected image plane; and

an image display means for displaying the panoramic image plane on a rectangular screen,

wherein,

if the optical axis of the fisheye lens is aligned parallel to a horizontal plane,

and a coordinate of an image point on the rectangular screen corresponding to an object point having a coordinate (X, Y, Z) in a world coordinate system, which has a nodal point of the fisheye lens as an origin and a vertical line passing through the origin as an Y-axis and the optical axis of the fisheye lens as a Z-axis, is given as (x″

, y″

),

the lateral coordinate x″

of the image point is proportional to a horizontal incidence angle ψ

,

herein, the horizontal incidence angle w is an angle an incident ray originating from the object point makes with an Y-Z plane and is given as

8. The imaging system of claim 7,

wherein the function F(d) of the vertical incidence angle d of the incident ray is given by any one among F(δ

)=tan δ

, F(δ

)=δ and

9. An imaging system comprising:

an image acquisition means for acquiring an uncorrected image plane using a fisheye lens rotationally symmetric about an optical axis,

an image processing means for extracting a panoramic image plane from the uncorrected image plane; and

an image display means for displaying the panoramic image plane on a rectangular screen,

wherein,

if the optical axis of the fisheye lens is aligned parallel to a horizontal plane,

and a coordinate of an image point on the rectangular screen corresponding to an object point having a coordinate (X, Y, Z) in a world coordinate system, which has a nodal point of the fisheye lens as an origin and a vertical line passing through the origin as an Y-axis and the optical axis of the fisheye lens as a Z-axis, is given as (x″

, y″

),

the lateral coordinate x″

of the image point is proportional to a horizontal incidence angle ψ

,

herein, the horizontal incidence angle w is an angle an incident ray originating from the object point makes with an Y-Z plane and is given as

10. An imaging system comprising:

an image acquisition means for acquiring an uncorrected image plane using a wide-angle lens rotationally symmetric about an optical axis,

an image processing means for extracting a rectilinear image plane from the uncorrected image plane; and

an image display means for displaying the rectilinear image plane on a rectangular screen,

wherein,

if a coordinate of an image point on the rectangular screen corresponding to an object point having a coordinate (X, Y, Z) in a world coordinate system, which has a nodal point of the wide-angle lens as an origin and a vertical line passing through the origin as an Y-axis, is given as (x″

, y″

),

arbitrary straight lines in the world coordinate system appear as straight lines on the screen.

11. The imaging system of claim 10,

wherein a straight line parallel to the Y-axis in the world coordinate system and intersecting with a principal direction of vision for the rectilinear image plane appears as a straight line parallel to an y″

-axis on the screen.

12. The imaging system of claim 10 further comprising an image selection means for determining parameters of the rectilinear image plane.

13. The imaging system of claim 12,

wherein the parameters of the rectilinear image plane comprise a size of the processed image plane (I_max, J_max), a horizontal field of view Δ

ψ

before any of the slide, pan and tilt operations, an amount of lateral slide operation Δ

J, an amount of longitudinal slide operation Δ

I, a pan angle, and a tilt angle.

14. An imaging system comprising:

an image acquisition means for acquiring an uncorrected image plane using a fisheye lens rotationally symmetric about an optical axis,

an image processing means for extracting a rectilinear image plane from the uncorrected image plane; and

an image display means for displaying the rectilinear image plane on a rectangular screen,

wherein,

if a coordinate of an image point on the rectangular screen corresponding to an object point having a coordinate (X, Y, Z) in a world coordinate system, which has a nodal point of the fisheye lens as an origin and a vertical line passing through the origin as an Y-axis, is given as (x″

, y″

),

arbitrary straight lines in the world coordinate system appear as straight lines on the screen,

a principal direction of vision for the rectilinear image plane is different from the optical axis direction of the fisheye lens,

and the principal direction of vision for the rectilinear image plane does not pass through the center of the rectilinear image plane.

15. An imaging system comprising:

an image acquisition means for acquiring an uncorrected image plane using a wide-angle lens rotationally symmetric about an optical axis,

an image processing means for extracting a rectilinear image plane from the uncorrected image plane; and

an image display means for displaying the rectilinear image plane on a rectangular screen,

wherein,

if a coordinate of an image point on the rectangular screen corresponding to an object point having a coordinate (X, Y, Z) in a world coordinate system, which has a nodal point of the wide-angle lens as an origin and a vertical line passing through the origin as an Y-axis, is given as (x″

, y″

),

the rectilinear image plane is obtained from the uncorrected image plane using digital slide operation and at least one digital pan or tilt operation,

and arbitrary straight lines in the world coordinate system appear as straight lines on the screen.

16. An imaging system comprising:

an image acquisition means for acquiring an uncorrected image plane using a wide-angle lens rotationally symmetric about an optical axis,

an image processing means for extracting a multiple viewpoint panoramic image plane from the uncorrected image plane; and

an image display means for displaying the multiple viewpoint panoramic image plane on a rectangular screen,

wherein,

the multiple viewpoint panoramic image plane includes the 1^st through the n^th sub rectilinear image planes laid out horizontally on the screen where n is a natural number larger than 2,

an arbitrary straight line in a world coordinate system, which has a nodal point of the wide-angle lens as an origin, appears as a straight line on any of the 1^st through the n^th sub rectilinear image planes,

and any straight line in the world coordinate system appearing on more than two adjacent sub rectilinear image planes appears as connected line segments.

17. The imaging system of claim 16,

wherein the 1^st through the n^th sub rectilinear image planes have identical magnification ratios,

and the optical axis of the wide-angle lens penetrates one sub rectilinear image plane among the n sub rectilinear image planes.

18. An imaging system comprising:

an image acquisition means for acquiring an uncorrected image plane using a wide-angle lens rotationally symmetric about an optical axis,

an image processing means for extracting a processed image plane from the uncorrected image plane; and

an image display means for displaying the processed image plane on a rectangular screen,

wherein,

the processed image plane includes the 1^st through the n^th sub rectilinear image planes where n is a natural number larger than 2,

an arbitrary straight line in a world coordinate system having a nodal point of the wide-angle lens as an origin appears as a straight line on any of the 1^st through the n^th sub rectilinear image planes.

19. Imaging system of claim 18,

wherein a straight line parallel to the Y-axis in the world coordinate system and intersecting with a principal direction of vision for any sub rectilinear image plane appears as a straight line parallel to an y″

-axis on the screen.

20. The imaging system in claim 18 further comprising an image selection means for determining parameters of the sub rectilinear image planes.

21. The imaging system of claim 18,

wherein the 1^st through the n^th sub rectilinear image planes have identical magnification ratios.

22. An imaging system comprising:

an image acquisition means for acquiring an uncorrected image plane using a wide-angle lens rotationally symmetric about an optical axis,

an image processing means for extracting a processed image plane from the uncorrected image plane,

an image selection means for determining projection schemes and parameters of the processed image plane; and

an image display means for displaying the processed image plane on a rectangular screen,

wherein,

the projection schemes of the processed image plane include a panoramic projection scheme and a rectilinear projection scheme,

if a coordinate of an image point on the rectangular screen corresponding to an object point having a coordinate (X, Y, Z) in a world coordinate system, which has a nodal point of the wide-angle lens as an origin and a vertical line passing through the origin as an Y-axis, is given as (x″

, y″

),

the panoramic projection scheme is a scheme wherein the lateral coordinate x″

of the image point is proportional to a horizontal incidence angle ψ

,

herein, the horizontal incidence angle ψ

is an angle an incident ray originating from the object point makes with an Y-Z plane and is given as

23. The imaging system of claim 22,

wherein the wide-angle lens is a fisheye lens.

24. The imaging system of claim 22,

wherein the image selection means comprise a situation awareness module.

25. The imaging system of claim 22,

wherein the image selection means comprise a direction sensing means.

26. The imaging system of claim 22,

wherein a straight line parallel to the Y-axis in the world coordinate system and intersecting with a principal direction of vision for a processed image plane having a rectilinear projection scheme appears as a straight line parallel to the y″

-axis on the screen.

27. The imaging system of claim 22,

wherein the image selection means comprise a direction sensing means,

and the image acquisition means is a video camera installed on an automobile.

28. The imaging system of claim 22,

wherein the image acquisition means is a camera module installed on a camera phone,

the image processing means is embedded software running in the camera phone,

and the image display means is a display screen of the camera phone.

29. The imaging system of claim 22,

wherein the image acquisition means is a camera module installed on a camera phone,

the image processing means is software running on a computer,

and the image display means is a display monitor of the computer.

30. The imaging system of claim 22,

wherein the image acquisition means is a PC camera,

the image processing means is software running on a computer,

and the image display means is a display monitor of the computer.

31. The imaging system of claim 22,

wherein the image acquisition means is a digital camera,

the image processing means is software running on a computer,

and the image display means is a display monitor of the computer.

32. The imaging system of claim 22,

wherein the image acquisition means is a camera module in a network camera,

and the image processing means is embedded software running in the network camera.

33. The imaging system of claim 22,

wherein the image acquisition means is a camera module on a network camera,

the image processing means is software running on a computer connected to the same network as the network camera,

and the image display means is a display monitor of the computer.

34. The imaging system of claim 22,

wherein the image acquisition means is a camera module on a video doorphone,

and the image processing means is embedded software running in the video doorphone,

and the image display means is a wallpad monitor in the house.

35. The imaging system of claim 22,

wherein the image acquisition means is a camera module on a video doorphone,

and the image processing means is embedded software running in the video doorphone,

and the image display means is display monitor of a smartphone connected to the video doorphone using a wireless Internet.

36. The imaging system of claim 22,

wherein the image acquisition means is a camera module on a broadcasting camera,

and the image processing means is embedded software running in the broadcasting camera.