Adjustable imaging system with wide angle capability

• US 6,226,035 B1
• Filed: 03/04/1998
• Issued: 05/01/2001
• Est. Priority Date: 03/04/1998
• Status: Expired due to Term

• Current Assignee: RemoteReality Corporation
• Original Assignee: Cyclovision Technologies Incorporated
• Inventors: Yaseen, L. Clayton II, Nayar, Shree K., Korein, James, Peri, Venkata N.
• Primary Examiner(s): Garber, Wendy R.
• Assistant Examiner(s): MOE, AUNG SOE
• Application Number:
  US09/034,208
• Time in Patent Office: 1,154 Days
• Field of Search: 348/335, 348/211, 348/36, 348/37, 348/38, 348/39, 348/207, 348/373, 348/143, 353/30, 353/98, 353/99, 352/69, 352/70
• US Class Current: 348/335
• CPC Class Codes: G02B 13/06 Panoramic objectives; So-called "sky lenses" including panoramic objectives having reflecting surfaces
• CPC Class Codes: G02B 17/06 using mirrors only , i.e. having only one curved mirror used in non-imaging applications G02B19/00
• CPC Class Codes: G03B 15/12 using mirrors
• CPC Class Codes: G03B 17/17 with reflectors arranged in beam forming the photographic image, e.g. for reducing dimensions of camera
• CPC Class Codes: G03B 37/00 Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe
• CPC Class Codes: G08B 13/19626 optical details, e.g. lenses, mirrors or multiple lenses
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

Optical systems are disclosed that enable an image sensor to generate an image of either a wide-angle view of an area of interest, or a direct view of an area of interest. In an embodiment, a wide-angle optical system is mounted to reflect radiation reflected from the area of interest along an optical axis. Radiation on the optical axis is directed to a zoom lens of the image sensor. Optionally, a planar mirror redirects the radiation, so that the optical axis is angled with respect to a central axis of the image sensor. In the preferred embodiment, the image sensor is a pan-tilt-zoom camera that can move from the wide-angle view to a direct view of a target or region of interest, using pan, tilt, and zoom controls, under either remote or automatic control. The disclosure also provides a method for maintaining registration of the image sensor when it is moved from a wide-angle view of a target to a direct view of a target.

Claims

1. An optical system that provides a wide field of view and a direct field of view of an area of interest, said optical system comprising:

a wide-angle optical system that reflects electromagnetic radiation from the wide field of view in the area of interest;

an image sensor that can sense the radiation and generate a signal representing a visible image from the radiation;

means for redirecting radiation; and

means for moving the image sensor to a first position in which the image sensor receives the redirected radiation and forms a wide field of view image of the area of interest, and for moving the image sensor to a second position facing away from the wide-angle optical system and the means for redirecting radiation and forms a direct field of view image.

2. The optical system in accordance with claim 1, in which the wide-angle optical system provides a substantially hemispherical field of view of the area of interest.

3. The optical system in accordance with claim 1, in which the wide-angle optical system comprises a curved mirror.

4. The optical system in accordance with claim 3, in which the curved mirror is formed with a partial quadric surface.

5. The optical system in accordance with claim 3, in which the curved mirror is a spherical mirror.

6. The optical system in accordance with claim 3, in which the curved mirror is a parabolic mirror.

7. The optical system in accordance with claim 3, in which the curved mirror is a hyperbolic mirror.

8. The optical system in accordance with claim 3, in which the curved mirror is an elliptical mirror.

9. The optical system in accordance with claim 1 in which the wide-angle optical system comprises a plurality of planar mirrors.

10. The optical system in accordance with claim 1, in which said wide-angle optical system comprises a faceted surface, and in which each facet of the faceted surface is a mirror.

11. The optical system in accordance with claim 1, in which the wide-angle optical system comprises a faceted surface, and in which each facet of the faceted surface is a curved mirror.

12. The optical system in accordance with claim 1, in which the wide-angle optical system comprises a curved mirror and a second mirror aligned to receive radiation reflected from the curved mirror and to direct the reflected radiation to the image sensor.

13. The optical system in accordance with claim 12, wherein the curved mirror is a paraboloid mirror having a curved outer surface that substantially obeys the equation z=(h²−

r²)/2h, where z is an axis of rotation, r is a radial coordinate, and h is twice the focal length of the paraboloid mirror.

14. The optical system in accordance with claim 1, in which the wide-angle optical system comprises a curved mirror, and in which the means for redirecting radiation comprises a planar mirror aligned to receive radiation reflected from the curved mirror and to direct the reflected radiation to the image sensor.

15. The optical system in accordance with claim 1, in which the means for redirecting radiation comprises one or more reflecting surfaces.

16. The optical system in accordance with claim 1, in which the means for redirecting radiation comprises one or more refracting surfaces.

17. The optical system in accordance with claim 1, in which the means for redirecting radiation comprises one or more optical fibers.

18. The optical system recited in claim 1, wherein the image sensor and the means for moving the image sensor comprises a pan-tilt-zoom (PTZ) camera.

19. The optical system in accordance with claim 1, in which the means for redirecting radiation comprises a relay lens axially aligned with a zoom lens.

20. The optical system recited in claim 1, in which the image sensor is aligned, when in the first position, to receive the radiation along an imaging axis that is substantially coincident with an optical axis of the wide-angle optical system.

21. The optical system recited in claim 1, further comprising:

means for selecting a target area in the wide field of view image of the area of interest; and

means for causing the image sensor to obtain a direct view that includes the target area.

22. The optical system recited in claim 1, further comprising:

a selection device selecting a target area in the wide field of view image of the area of interest; and

an image control mechanism controlling the image sensor to obtain a direct view including the target area.

23. An optical system that provides a wide field of view and a direct field of view of an area of interest, comprising:

a wide-angle optical system that reflects electromagnetic radiation from the wide field of view in the area of interest;

a camera that can sense the radiation and generate a signal representing a visible image from the radiation, in which the camera is movable to pan, tilt, and zoom from a first position in which the camera receives the radiation reflected from the wide-angle optical system and forms a wide field of view image of the area of interest, and a second position directed away from the wide-angle optical system in which the image sensor receives radiation directly from the area of interest and forms a direct field of view image.

24. The optical system of claim 23, in which the wide-angle optical system comprises a parabolic reflector.

25. The optical system of claim 24, in which the wide-angle optical system further comprises a relay lens for directing the reflected radiation from the parabolic reflector to the camera.

26. The optical system of claim 25, in which the camera further comprises a zoom lens that cooperates with the relay lens to cause the reflected radiation to be telecentric when the camera is in the first position.

27. An optical system that provides a wide field of view and a direct field of view of an area of interest, comprising:

a wide-angle optical system that reflects at least one image from the wide field of view in the area of interest;

a camera that receives the at least one image from said wide-angle optical system and generates a signal responsive thereto, wherein said camera is movable to pan, tilt, and zoom from a first position in which the camera receives the at least one image reflected from the wide-angle optical system and forms a wide field of view image of the area of interest, and a second position directed away from the wide-angle optical system in which said camera receives at least one other image directly from the area of interest and forms a direct field of view image therefrom.

28. An optical system that provides a wide field of view and a direct field of view of an area of interest, comprising:

a wide-angle optical or mirror system that reflects at least one image from the wide field of view in the area of interest;

an image acquisition device that receives the at least one image from said wide-angle optical or mirror system and generates a signal responsive thereto, wherein said image acquisition device is movable to pan, tilt, and zoom from a first position in which the image acquisition device receives the at least one image reflected from the wide-angle optical or acquisition system and forms a wide field of view image of the area of interest, and a second position directed away from the wide-angle optical or mirror system in which said image acquisition device receives at least one other image directly from the area of interest and forms a direct field of view image therefrom.

29. A method of providing a wide field of view and a direct field of view of an area of interest using an optical system, comprising the steps of:

(a) reflecting at least one image from the wide field of view in the area of interest using a wide-angle optical or mirror system; and

(b) controlling the alternate performance of;

(1) receiving the at least one image and generating a signal responsive thereto using an image acquisition device movable to pan, tilt, and zoom, forming a wide field of view image of the area of interest, and

(2) receiving at least one other image directly from the area of interest using the image acquisition device without the use of the wide-angle optical or mirror system, forming a direct field of view image therefrom.

30. A method according to claim 29, wherein said controlling step (b) is perfomed remotely from the image acquisition device.

31. A method according to claim 29, wherein said controlling step (b) is perfomed remotely from the image acquisition device via at least one of a local area network or a public network.

32. A method according to claim 29, wherein said controlling step (b) is perfomed remotely from the image acquisition device via at least one of a local area network or a public network including the Internet.

33. An optical system that provides a wide field of view and a direct field of view of an area of interest, said optical system comprising:

a wide-angle optical system that reflects electromagnetic radiation from the wide field of view in the area of interest;

an image sensor that can sense the radiation and generate a signal representing a visible image from the radiation;

means for moving the image sensor to a first position in which the image sensor receives the radiation reflected from the wide-angle optical system and forms a wide field of view image of the area of interest, and for moving the image sensor to a second position facing away from the wide-angle optical system and forms a direct field of view images

one or more processors; and

a memory coupled to the one or more processors, the memory having stored therein sequences of instructions which, when executed by the one or more processors, cause the one or more processors to move the image sensor from the first position to the second position by causing the processor to perform the steps of;

(A) determining first coordinates of a first viewpoint associated with the wide field of view image, second coordinates of a second viewpoint associated with the direct field of view image, and direction of a target in a first coordinate system associated with the first viewpoint;

(B) converting the direction into a ray extending from the first viewpoint to the target;

(C) computing an intersection of the ray with values describing a reference plane on which the target lies to obtain third coordinates of the target with respect to a first coordinate system that is associated with the first viewpoint;

(D) translating the third coordinates into a second coordinate system that is associated with the second viewpoint, using the intersection computed in said step (C) to obtain fourth coordinates; and

(E) converting the fourth coordinates into a pan angle value, a tilt angle value, and a focal distance value representing the second position.

34. The optical system recited in claim 33, in which the sequences of instructions further cause the one or more processors to carry out the step of (F) commanding the image sensor to move from the first position to the second position according to the pan angle value, tilt angle value, and focal distance value.

35. An optical system that provides a wide field of view and a direct field of view of an area of interest, comprising:

a wide-angle optical system that reflects electromagnetic radiation from the wide field of view in the area of interest;

an image sensor that can sense the radiation and generate a signal representing a visible image from the radiation; and

means for moving the image sensor to a first position in which the image sensor receives the radiation reflected from the wide-angle optical system and forms a wide field of view image of the area of interest, and for moving the image sensor to a second position facing away from the wide-angle optical system in which the image sensor receives radiation from the area of interest and forms a direct field of view image,

one or more processors; and

a memory coupled to the one or more processors, the memory having stored therein sequences of instructions which, when executed by the one or more processors, cause the one or more processors to move the image sensor from the first position to the second position by causing the processor to perform the steps of;

(A) receiving a target height value z₂ and a vector S directed from a viewpoint associated with the direct field of view image to a second viewpoint associated with the wide field of view image, wherein the first viewpoint and the second viewpoint is respectively associated with a first coordinate system and a second coordinate system;

(B) computing a pan angle θ

₂ and a tilt angle φ

₂;

(C) computing a vectors T₂ of target coordinates (x₂, y₂, z₂) in the second coordinate system associated with the second viewpoint by computing the equations

36. An optical system that provides a wide field of view and a direct field of view of an area of interest, said optical system comprising:

a wide-angle optical system that reflects electromagnetic radiation from the wide field of view in the area of interest;

an image sensor that can sense the radiation and generate a signal representing a visible image from the radiation;

means for moving the image sensor to a first position in which the image sensor receives the radiation reflected from the wide-angle optical system and forms a wide field of view image of the area of interest, and for moving the image sensor to a second position facing away from the wide-angle optical system and forms a direct field of view images one or more processors;

a memory coupled to the one or more processors, the memory having stored therein;

a plurality of presets, each of the presets comprising information describing a pre-defined position of the image sensor that provides a direct view of the area of interest; and

sequences of instructions which, when executed by the one or more processors, cause the one or more processors to move the image sensor from the first position to the second position by causing the processor to perform the steps of;

(A) determining first coordinates of a first viewpoint associated with the wide field of view image, second coordinates of a second viewpoint associated with the direct field of view image, and direction of a target in a first coordinate system associated with the first viewpoint;

(B) converting the direction into a ray extending from the first viewpoint to the target;

(C) computing an intersection of the ray with values describing a reference plane on which the target lies to obtain third coordinates of the target with respect to a first coordinate system that is associated with the first viewpoint;

(D) translating the third coordinates into a second coordinate system that is associated with the second viewpoint, using the intersection computed in said step (C) to obtain fourth coordinates; and

(E) selecting one of the plurality of presets that provides a direct view of a region of the area of interest that includes the fourth coordinates.

37. The optical system recited in claim 36, wherein the sequences of instructions further comprises instructions that cause the processor carry out the step of (F) commanding the image sensor to move to the preset select in step (E).

38. The optical system recited in claim 36, wherein the plurality of presets collectively defines direct views of the entire area of interest.

39. The optical system recited in claim 36, wherein each of the presets comprises values representing a pan position, tilt position, and zoom position of the image sensor.

40. An optical system that provides a wide field of view and a direct field of view of an area of interest, said optical system comprising:

a wide-angle optical system that reflects electromagnetic radiation from the wide field of view in the area of interest;

an image sensor that can sense the radiation and generate a signal representing a visible image from the radiation;

means for moving the image sensor to a first position in which the image sensor receives the radiation reflected from the wide-angle optical system and forms a wide field of view image of the area of interest, and for moving the image sensor to a second position facing away from the wide-angle optical system and forms a direct field of view image; and

a computer-readable medium carrying sequences of instructions which, when executed by one or more processors, cause the one or more processors to move the image sensor from the first position to the second position by causing the processor to perform the steps off;

(A) determining first coordinates of a first viewpoint associated with the wide field of view image, second coordinates of a second viewpoint associated with the direct field of view image, and direction of a target in a first coordinate system associated with the first viewpoint;

(B) converting the direction into a ray extending from the first viewpoint to the target;

(C) computing an intersection of the ray with values describing a reference plane on which the target lies to obtain third coordinates of the target with respect to a first coordinate system that is associated with the first viewpoint;

(D) translating the third coordinates into a second coordinate system that is associated with the second viewpoint, using the intersection computed in said step (C) to obtain fourth coordinates; and

(E) converting the fourth coordinates into a pan angle value, a tilt angle value, and a focal distance value representing the second position.

41. The optical system recited in claim 40, in which the sequences of instructions further cause the one or more processors to carry out the step of (F) commanding the image sensor to move from the first position to the second position according to the pan angle value, tilt angle value, and focal distance value.

42. An optical system that provides a wide field of view and a direct field of view of an area of interest, said optical system comprising:

a wide-angle optical system that reflects electromagnetic radiation from the wide field of view in the area of interest;

an image sensor that can sense the radiation and generate a signal representing a visible image from the radiation;

means for moving the image sensor to a first position in which the image sensor receives the radiation reflected from the wide-angle optical system and forms a wide field of view image of the area of interest, and for moving the image sensor to a second position facing away from the wide-angle optical system and forms a direct field of view image;

a computer-readable medium carrying sequences of instructions which, when executed by one or more processors, cause the one or more processors to move the image sensor from the first position to the second position by causing the processor to perform the steps of;

(A) determining first coordinates of a first viewpoint associated with the wide field of view image, second coordinates of a second viewpoint associated with the direct field of view image, and direction of a target in a first coordinate system associated with the first viewpoint;

(B) converting the direction into a ray extending from the first viewpoint to the target;

(C) computing an intersection of the ray with values describing a reference plane on which the target lies to obtain third coordinates of the target with respect to a first coordinate system that is associated with the first viewpoint;

(D) translating the third coordinates into a second coordinate system that is associated with the second viewpoint, using the intersection computed in said step (C) to obtain fourth coordinates; and

(E) converting the fourth coordinates into a pan angle value, a tilt angle value, and a focal distance value representing the second position.

43. The optical system recited in claim 42, wherein the sequences of instructions further comprise instructions that cause the processor to carry out the step of (F) commanding the image sensor to move to the preset selected in step (E).

44. The optical system recited in claim 42, wherein the plurality of presets collectively defines direct views of the entire area of interest.

45. The optical system recited in claim 42, wherein each of the presets comprises values representing a pan position, tilt position, and zoom position of the image sensor.

46. An optical system that provides a wide field of view and a direct field of view of an area of interest, said optical system comprising:

a wide-angle optical system that reflects electromagnetic radiation from the wide field of view in the area of interest;

a camera that can sense the radiation and generate a signal representing a visible image from the radiation, in which the camera is movable to pan, tilt, and zoom from a first position in which the camera receives the radiation reflected from the wide-angle optical system and forms a wide field of view image of the area of interest, and a second position directed away from the wide-angle optical system and forms a direct field of view image; and

a computer-readable medium carrying sequences of instructions which, when executed by one or more processors, cause the one or more processors to move the image sensor from the first position to the second position by causing the processor to perform the steps of;

(A) determining first coordinates of a first viewpoint associated with the wide field of view image, second coordinates of a second viewpoint associated with the direct field of view image, and direction of a target in a first coordinate system associated with the first viewpoint;

(B) converting the direction into a ray extending from the first viewpoint to the target;

(C) computing an intersection of the ray with values describing a reference plane on which the target lies to obtain third coordinates of the target with respect to a first coordinate system that is associated with the first viewpoint;

(D) translating the third coordinates into a second coordinate system that is associated with the second viewpoint, using the intersection computed in said step (C) to obtain fourth coordinates; and

(E) converting the fourth coordinates into a pan angle value, a tilt angle value, and a focal distance value representing the second position.

47. The optical system recited in claim 46, in which the sequences of instructions further cause the one or more processors to carry out the step of (F) commanding the camera to move from the first position to the second position according to the pan angle value, tilt angle value, and focal distance value.

48. An optical system that provides a wide field of view and a direct field of view of an area of interest, said optical system comprising:

a wide-angle optical system that reflects electromagnetic radiation from the wide field of view in the area of interest;

a camera that can sense the radiation and generate a signal representing a visible image from the radiation, in which the camera is movable to pan, tilt, and zoom from a first position in which the camera receives the radiation reflected from the wide-angle optical system and forms a wide field of view image of the area of interest, and a second position directed away from the wide-angle optical system and forms a direct field of view image;

a computer-readable medium having stored therein;

a plurality of presets, each of the presets comprising information describing a pre-defined position of the camera that provides a direct view of the area of interest; and

sequences of instructions which, when executed by one or more processors, cause the one or more processors to move the camera from the first position to the second position by causing the processor to perform the steps of;

(A) determining first coordinates of a first viewpoint associated with the wide field of view image, second coordinates of a second viewpoint associated with the direct field of view image, and direction of a target in a first coordinate system associated with the first viewpoint;

(B) converting the direction into a ray extending from the first viewpoint to the target;

(C) computing an intersection of the ray with values describing a reference plane on which the target lies to obtain third coordinates of the target with respect to a respect to a first coordinate system that is associated with the first viewpoint;

(D) translating the third coordinates into a second coordinate system that is associated with the second viewpoint, using the intersection computed in said step (C) to obtain fourth coordinates; and

(E) selecting one of the plurality of presets that provides a direct view of a region of the area of interest that includes the fourth coordinates.

49. The optical system recited in claim 48, wherein the sequences of instructions further comprise instructions that cause the processor to carry out the step of (F) commanding the camera to move to the preset selected in step (E).

50. The optical system recited in claim 48, wherein the plurality of presets collectively defines direct views of the entire area of interest.

51. The optical system recited in claim 48, wherein each of the presets comprises values representing a pan position, tilt position, and zoom position of the camera.

52. An optical system that provides a wide field of view and a direct field of view of an area of interest, comprising:

a wide-angle optical system that reflects electromagnetic radiation from the wide field of view in the area of interest;

an image sensor that can sense the radiation and generate a signal representing a visible image from the radiation; and

a mount having a first and a second position for moving the image sensor to a first position in which the image sensor receives the radiation reflected from the wide-angle optical system and forms a wide field of view image of the area of interest, and for moving the image sensor to a second position facing away from the wide-angle optical system in which the image sensor receives radiation from the area of interest and forms a direct field of view image.