Apparatus for generating and interactively viewing spherical image data and memory thereof

• US 6,323,862 B1
• Filed: 07/08/1999
• Issued: 11/27/2001
• Est. Priority Date: 12/14/1992
• Status: Expired due to Term

• Current Assignee: Grandeye Limited (Oncam Global Group AG)
• Original Assignee: Ford Oxaal
• Inventors: Oxaal, Ford
• Primary Examiner(s): Zimmerman, Mark
• Assistant Examiner(s): Sealey, Lance W.
• Application Number:
  US09/349,241
• Time in Patent Office: 873 Days
• Field of Search: 345/427, 345/430, 348/36
• US Class Current: 345/581
• CPC Class Codes: G03B 37/02 with scanning movement of lens or cameras
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

An apparatus for modeling the visible world using a spherical data set includes a first device means for generating the spherical data set, the first device having a wide angle lens which is pivotable around a predetermined focal point, a second device for selecting a viewpoint within a p-surface, a third device for texture mapping selected data of the spherical image data onto the p-surface such that the resultant texture map is substantially equivalent to projecting the selected data onto the p-surface from the viewpoint to thereby generate a texture mapped p-sphere, and a fourth device for displaying a predetermined portion of the texture mapped p-sphere. A memory which stores a seamless spherical image data set formed from N circular image data sets captured by a single lens, where the circular image data sets are taken from a single focal point of the lens, and where N is a positive integer greater than 1 is also described.

Claims

1. A memory which stores a substantially seamless spherical image data set formed from N circular image data sets captured by a single lens, wherein the N circular image data sets are taken from a single point of the lens, and wherein N is a positive integer greater than 1.

2. The memory as recited in claim 1, wherein:

N equals 2; and

each of the N circular image data sets corresponds to a hemispherical projection onto a plane.

3. The memory as recited in claim 1, wherein:

N is greater than 2; and

each of the N circular image data sets corresponds to circular projection onto a plane.

4. The memory as recited in claim 1, wherein all of the N circular image data sets are disposed within the memory so as to permit selection and subsequent display of a rectangular image generated from combined data extractable from at least two of the N circular image data sets.

5. A system for acquisition of a substantially seamless spherical data set and subsequent display of a selected portion thereof on a display, comprising:

means for capturing at least two circular images from a single point and different directions of view;

means for transferring the circular images to a memory of a computer;

means for combining the circular images into a substantially seamless spherical data set; and

means for processing a selected portion of the substantially seamless spherical data set for presentation on the display of the computer.

6. The system as set forth in claim 5, further comprising means for digitizing each ofthe circular images, wherein the digitizing means is disposed within the capturing means.

7. The system as set forth in claim 5, wherein the capturing means comprises a camera optically coupled to a single wide-angle lens.

8. The system as set forth in claim 5, wherein the capturing means permits the taking of at least two respective pictures with a single camera with each respective picture being registered with respect to the previously-taken picture.

9. The system as recited in claim 5, wherein the selected portion of the substantially seamless spherical data set includes data from two adjacent circular images.

10. The system as recited in claim 5, wherein the processing means comprises:

a first software module for reading the selected portion of the substantially seamless spherical data set to thereby generate a read spherical data; and

a second software module for processing the read spherical data to thereby generate a rectangular image suitable for output on the computer display.

11. The system as recited in claim 10, wherein the processing means is controlled by an input device operated by a system user.

12. The system as recited in claim 5, wherein:

the processing means and the computer display are disposed within the computer; and

the processing means is controlled responsive to operation of an input device coupled to the computer.

13. The system as recited in claim 5, wherein:

the at least two circular images comprise two hemispherical images; and

the combining means comprises software for registering data corresponding to a first of the two hemispherical images with additional data corresponding to a second of the two hemispherical images.

14. The system as recited in claim 5, further comprising means for selecting a magnification level at which the selected potion of the substantially seamless spherical data set is to be displayed on the display.

15. The system as recited in claim 5, further comprising:

means for selecting a magnification level at which the selected potion of the substantially seamless spherical data set is to be displayed on the computer display;

wherein the processing means comprises;

a first software module for reading the selected portionof the substantially seamless spherical data set to thereby generate a read spherical data set; and

a second software module for processing the read spherical data set to thereby generate a rectangular image suitable for output on the computer display.

16. The system as recited in claim 15, wherein the selecting means and the first and second software modules respond to an input device operated by a system user.

17. The system as recited in claim 15, wherein the processing means and the selecting means comprise software.

18. The system as recited in claim 15, wherein:

the processing means, the selecting means, and the display are disposed within a computer; and

the processing means and the selecting means respond to an input device operated by a user of the computer.

19. The system as recited in claim 15, wherein:

the at least two circular images comprise two hemispherical images; and

the combining means comprises software for registering data corresponding to a first of the two hemispherical images with additional data corresponding to a second of the two hemispherical images.

20. The system as recited in claim 5, further comprising an input device operatively connected to the processing means for selecting a magnification level at which the selected potion of the substantially seamless spherical data set is displayed on the computer display.

21. The system as recited in claim 5, wherein the combining means comprises software which digitally stitches together the at least two respective digital images.

22. A system which acquires image content corresponding to all pixels visible from a predetermined point using a camera generating circular images, which stores the content as a substantially seamless spherical data set, and which displays a selected portion of the substantially seamless spherical data set, comprising:

a camera equipped with a lens which acquires N circular images from the predetermined point and different directions of view;

a memory which stores the N circular images as the substantially seamless spherical data set; and

a processor which generates a rectangular image from the selected portion ofthe substantially seamless spherical data set suitable for output on a computer display,

where N is an integer greater than or equal to two.

23. The system as set forth in claim 22, wherein:

the system further comprises the display; and

the memory, the processor, and the computer display are disposed within a computer.

24. The system as set forth in claim 22, wherein the memory is disposed within the camera.

25. The system as set forth in claim 22, wherein the camera acquires at least two digital images, and the memory stores the digital images comprising the substantially seamless spherical data set as registered digital images.

26. The system as recited in claim 22, wherein the selected portion of the substantially seamless spherical data set includes data from two adjacent circular images.

27. The system as recited in claim 22, further comprising an input device permitting selection of the selected portion of the substantially seamless spherical data set.

28. The system as recited in claim 22, further comprising an input device permitting selection of the selected portion ofthe substantially seamless spherical data set and the magnification level of the resultant rectangular image.

29. The system as recited in claim 22, wherein the lens comprises a fisheye lens.

30. The system as recited in claim 22, further comprising:

a tripod; and

a camera mount coupling the camera to the tripod which permits the camera to pivot about the predetermined point to thereby acquire the N circular images from the predetermined point and different directions of view.

31. The system as recited in claim 22, further comprising:

a stationary support; and

a mounting support member coupling the camera to the stationary support, which mounting support member is rotatable about a rotation axis which is coplanar with a plane that is perpendicular to the optical axis of the lens of the camera.

32. The system as recited in claim 31, wherein the stationary support comprises a tripod.

33. The system as recited in claim 22, further comprising:

a stationary support; and

a mounting support member coupling the camera to the stationary support, which mounting support member is rotatable about a rotation axis which is coplanar with a plane that is perpendicular to the optical axis of the lens of the camera.

34. The system as recited in claim 22, further comprising:

a stationary support; and

a mounting support member coupling the camera to the stationary support, which mounting support member is rotatable about a rotation axis which bisects and is coplanar with a plane corresponding to a 180°

field of view of the lens of the camera.

35. The system as recited in claim 22, further comprising:

a stationary support; and

a mounting support member coupling the camera to the stationary support, which mounting support member is rotatable about a rotation axis which bisects and is coplanar with a plane that is perpendicular to the optical axis of the lens of the camera and which corresponds to a 180°

field of view of the lens of the camera.

36. The system as recited in claim 22, wherein:

the lens has a plane corresponding to a 180°

field of view;

the system further comprising;

a stationary support; and

a mounting support member coupling the camera to the stationary support, wherein;

the mounting support member is rotatable about a rotation axis which is coplanar with the plane;

the camera does not intersect the rotation axis; and

the rotation axis intersects the lens.

37. The system as recited in claim 22, wherein the processor instantiates software for digitally stitching together the at least two respective digital images.

38. A system for acquiring a substantially spherical visual image data set using a fisheye lens having at least a 180°

field of view defined by a plane intersecting the fisheye lens, and for processing at least a selected portion of the substantially spherical visual image data set for display as a two-dimensional (2D) image, comprising;

an assembly comprising the fisheye lens coupled to a camera;

a stationary support;

a mounting support member, coupling the assembly to the stationary support, which mounting support member has a rotation axis that is coplanar with the plane and which supports the assembly while the assembly acquires first and second hemispheric images, each representing the 180°

field of view;

a memory which stores the substantially spherical visual image data set corresponding to the respective hemispherical images; and

a processor which generates the 2D image responsive to the selected portion of the substantially spherical visual image data set.

39. The system as recited in claim 38, wherein the mounting support member includes at least one detent mechanism which permits the assembly to rotate between first and second predetermined positions for acquiring the respective hemispherical images.

40. The system as recited in claim 38, wherein thc rotation axis is not vertical.

41. The system as recited in claim 38, wherein the stationary support comprises a tripod.

42. The system as recited in claim 38, wherein the plane corresponds to a seam between respective hemispherical images.

43. A system for acquisition of a substantially seamless spherical data set and subsequent display of a selected portion thereof on a display, comprising:

means for capturing at least two circular images from a single point and different directions of view;

means for transferring the circular images to a memory of a computer;

means for combining the circular images into a substantially seamless spherical data set; and

means for processing a selected portion of the substantially seamless spherical data set for presentation on the display of the computer, the processing means comprising;

a first software module for reading the selected portion of the substantially seamless spherical data set to thereby generate a read spherical data; and

a second software module for processing the read spherical data to thereby generate a rectangular image in linear perspective suitable for output on the computer display.

44. A system for acquisition of image content corresponding to all pixels visible from a predetermined point and subsequent display of a selected portion of the image content thereof on a display, comprising:

means for capturing two hemispherical images from the predetermined point and different directions of view;

means for storing the hemispherical images corresponding to the pixels visible from the predetermined point; and

means for processing a selected portion of the pixels for presentation on the display of the computer, the processing means comprising;

a first software module for reading selected pixels corresponding to a first portion of a first of the hemispherical images and a second portion of a second of the hemispherical images to thereby generate a first and second read pixel data; and

a second software module for automatically processing the first and second read pixel data to thereby generate a singular rectangular image in linear perspective suitable for output on the computer display.