Method for generating and interactively viewing spherical image data

• US 6,271,853 B1
• Filed: 07/08/1999
• Issued: 08/07/2001
• Est. Priority Date: 12/14/1992
• Status: Expired due to Term

• Current Assignee: Grandeye Limited (Oncam Global Group AG)
• Original Assignee: Ford Oxaal
• Inventors: Oxaal, Ford
• Primary Examiner(s): Nguyen, Phu K.
• Application Number:
  US09/349,240
• Time in Patent Office: 761 Days
• Field of Search: 345/427, 345/425, 345/419, 345/418, 345/433, 382/293
• US Class Current: 715/848
• CPC Class Codes: G03B 37/02 with scanning movement of lens or cameras
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations
• CPC Class Codes: G06T 3/18 Image warping, e.g. rearranging pixels individually
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

A method of modeling of the visible world using a spherical image data set includes steps for generating the spherical image data set, selecting a viewpoint within a p-surface, and texture mapping selected data from the spherical image data set onto the p-surface such that the resultant texture map is substantially equivalent to projecting the selected data onto the p-surface from the viewpoint to thereby generate a texture mapped p-surface. According to one aspect of the invention, the method also includes a step for either rotating the texture mapped p-surface or changing the direction of view to thereby expose a new portion of the texture mapped p-surface. According to another aspect of the invention, a first the texture mapped p-sphere is replaced by a second texture mapped p-sphere by interactively selecting the new viewpoint from viewpoints within the second texture mapped p-sphere.

Claims

1. A method of modeling the visible world using a seamless spherical data set, said method comprising:

taking N different circular images from a common focal point with a wide angle lens;

digitizing the N circular images to thereby generate N circular image data sets, respectively;

combining the N circular image data sets to minimize coincidence between the N circular image data sets to thereby form the seamless spherical data set;

selecting a viewpoint within a p-surface; and

texture mapping selected data from the spherical data set onto said p-surface such that the resultant texture map is substantially equivalent to projecting the selected data onto the p-surface from said viewpoint to thereby generate a texture mapped p-surface.

2. The method as recited in claim 1, further comprising rotating said texture mapped p-surface so as to simulate rotating the direction of view in the opposite direction.

3. The method as recited in claim 1, wherein said selecting step comprises selecting the viewpoint and a direction of view, and wherein said method further comprises interactively changing said direction of view to thereby expose a corresponding portion of said texture mapped p-surface.

4. The method as recited in claim 1, further comprising displaying a predetermined portion of said texture mapped p-surface.

5. The method as recited in claim 4, wherein the viewer is allowed to interactively alter at least one of focal length or an angle of view relative to said textured mapped p-surface to thereby vary the displayed portion of said texture mapped p-surface.

6. The method as recited in claim 1, further comprising:

displaying a predetermined portion of said p-surface;

selecting a new viewpoint;

repeating said texture mapping step using said new viewpoint; and

redisplaying said predetermined portion of said p-surface,

whereby a first image portion occupying said predetermined portion displayed during the displaying step is different than a second image portion occupying said predetermined portion during the redisplaying step.

7. The method as recited in claim 6, wherein said selecting step comprises interactively selecting said new viewpoint.

8. The method as recited in claim 7, wherein a first said texture mapped p-surface is replaced by a second texture mapped p-surface by interactively selecting said new viewpoint from viewpoints within said second texture mapped p-surface.

9. The method as recited in claim 1, further comprising:

selecting a new viewpoint; and

displaying said texture mapped p-surface from said new viewpoint.

10. The method as recited in claim 9, wherein the new viewpoint is proximate the surface of said p-surface.

11. A method comprising:

taking N different circular images from a common focal point with a wide angle lens;

digitizing the N circular images to thereby generate N circular image data sets, respectively;

combining the N circular image data sets in such a manner as to form a seamless spherical data set;

selecting a viewpoint within a p-surface;

texture mapping selected data of the spherical data set onto said p-surface such that the resultant texture map is substantially equivalent to projecting the selected data onto the p-surface from said viewpoint to thereby generate a texture mapped p-sphere; and

displaying a predetermined portion of said texture mapped p-sphere.

12. The method as recited in claim 11, further comprising rotating said texture mapped p-sphere so as to simulate rotating the direction of view in the opposite direction.

13. The method as recited in claim 11, wherein said selecting step comprises selecting the viewpoint and a direction of view, and wherein said method further comprises interactively changing said direction of view to thereby display another portion of said texture mapped p-sphere.

14. The method as recited in claim 11, further comprising:

selecting a new viewpoint; and

repeating said texture mapping and said displaying steps using said new viewpoint.

15. The method as recited in claim 14, wherein a first said texture mapped p-sphere is replaced by a second said texture mapped p-sphere by interactively selecting said new viewpoint from viewpoints within said second texture mapped p-sphere.

16. The method as recited in claim 11, further comprising:

selecting a new viewpoint; and

displaying said predetermined portion of said texture mapped p-surface using said new viewpoint.

17. The method as recited in claim 16, wherein said selecting step comprises interactively selecting said new viewpoint.

18. A method comprising using a programmed computer to digitally stitch together at least two digital circular images that represent the visible world as seen from a fixed point and at least two respective directions of view, in such a manner as to generate a spherical image data set.

19. The method as set forth in claim 18, further comprising using the spherical image data set to display a seamless image on a display device.

20. The method as set forth in claim 18, further comprising using a programmed computer graphics system to generate an image in a selected one of two different perspectives in response to the spherical image data set.

21. The method as set forth in claim 18, wherein each of the at least two digital images is a circular digital image generated using a fisheye lens.

22. A method comprising:

capturing at least two circular images from a single focal point and different directions of view;

digitizing each of the at least two circular images to thereby generate at least two respective image data sets; and

,

generating a spherical image data set by combining the at least two respective image data sets.

23. The method as set forth in claim 22, wherein each of the at least two images is a circular image generated by a fisheye lens.

24. The method as set forth in claim 22, further comprising displaying a seamless image in response to the spherical image data set.

25. The method as set forth in claim 24, wherein the spherical data set represents a portion of the visible world that would be seen by a viewer having a 360°

field of view from the single point.

26. The method as set forth in claim 24, wherein a computer generated image produced using the spherical data set constitutes a visual depiction of a 360°

field of view of the visible world from the single point.

27. The method as set forth in claim 24, wherein the spherical data set represents a 360°

field of view of the visible world from a specified viewpoint.

28. The method as set forth in claim 22, further comprising using a programmed computer graphics system to generate a displayed image in one of linear perspective and circular perspective in response to the spherical image data set.

29. The method as set forth in claim 22, wherein the capturing is performed using a single wide-angle lens.

30. The method as set forth in claim 22, wherein the capturing is performed using a single fisheye lens.

31. The method as set forth in claim 22, wherein the capturing is performed by taking at least two respective shots with a single camera with each respective shot being aligned with respect to the previously-taken shot.

32. The method as set forth in claim 22, wherein the capturing is performed by taking at least two respective shots with a single camera with each respective shot being precisely registered with respect to the previously-taken shot.

33. A method comprising:

using a camera to take a first circular picture from a focal point and a first direction of view;

pivoting the camera about a fixed pivot axis;

using the camera to take a second circular picture from the focal point and a second direction of view opposite to the first direction of view;

digitizing the first and second circular pictures to thereby generate first and second digital pictures; and

,

combining the first and second digital pictures to produce a digital spherical data set.

34. The method as set forth in claim 33, wherein the camera is a digital camera.

35. The method as set forth in claim 33, wherein the first and second circular pictures are each generated using a fisheye lens optically connected to the camera.

36. The method as set forth in claim 33, wherein the pivoting is performed in such a manner as to precisely register the first and second directions of view.

37. A method comprising:

precisely aligning a camera with respect to a first direction of view;

taking a first circular picture with the camera from a prescribed point;

aligning the camera with respect to a second direction of view substantially co-linear with the first direction of view;

taking a second circular picture with the camera from the prescribed point;

using the first and second circular pictures to generate a substantially seamless spherical data set.

38. The method as set forth in claim 37, wherein the substantially seamless spherical data set represents a 360°

field of view of the visible world from the prescribed focal point.

39. The method as set forth in claim 37, wherein the first and second circular pictures are each generated by a fisheye lens optically coupled to the camera.

40. The method as set forth in claim 37, wherein the camera is a digital camera.

41. The method as set forth in claim 37, wherein the camera is equipped with a wide angle lens.

42. The method as set forth in claim 37, wherein the camera is equipped with a fisheye lens.

43. A method for generating a substantially seamless spherical image data set using a camera equipped with a wide angle lens, comprising:

taking N different circular images from a common point within the wide angle lens;

digitizing the N circular images to thereby generate N circular image data sets, respectively; and

combining the N circular image data sets to minimize coincidence between the N circular image data sets to thereby form the substantially seamless spherical image data set.

44. A method for generating a seamless spherical image data set using a digital camera equipped with a wide angle lens, comprising:

capturing a sequence of N circular image data sets sufficient to form the seamless spherical image data set through the wide angle lens, each of the N circular image data sets representing a unique direction of view, and all of the respective directions of view intersecting a common point within the wide angle lens; and

processing the N circular image data sets to eliminate redundant circular image data to thereby form the seamless spherical image data set,

where N is a positive integer greater than 1.

45. The method as recited in claim 44, wherein:

the wide angle lens comprises a fisheye lens, and

each of the N circular image data sets corresponds to a projection of a hemisphere on a plane.

46. The method as recited in claim 44, wherein:

the wide angle lens comprises a fisheye lens; and

N is equal to 2.

47. The method as recited in claim 44, wherein:

the wide angle lens comprises a lens having a viewing angle of at least 180°

; and

N is greater than or equal to 2.

48. The method as recited in claim 44, wherein:

the wide angle lens comprises a lens having a viewing angle of at least 180°

;

all of the direction of views define a plane; and

each direction of view is separated from an adjacent direction of view by an angle equal to 360°

divided by N.