System and method for creation, processing and visualization of omni-directional images

• US 20020141659A1
• Filed: 02/06/2001
• Published: 10/03/2002
• Est. Priority Date: 02/06/2001
• Status: Active Grant

• Current Assignee: Tour Technology Software, Inc.
• Original Assignee: Richard Wilson Jr.
• Inventors: Serban, Sever, Florin, Rusu Mihai
• Granted Patent: US 6,754,400 B2
• Time in Patent Office: Days
• US Class Current: 382/285
• CPC Class Codes: G06T 15/205 Image-based rendering

Abstract

A system and method for digitally rendering omni-directional images includes capturing images of a region 360 degrees about an origin point in a single plane, and assembling the images in a digital format to create a complete spherical image surrounding the origin point. The spherical image is projected onto faces of a cube surrounding the spherical image. Images which were projected on the faces of the cube may then be rendered to provide an omni-directional image.

Claims

1. A method for digitally rendering omni-directional images comprising the steps of:

capturing images surrounding an origin point in a at least two hemispheres surrounding the origin point;

assembling the images in a digital format to create a complete spherical image surrounding the origin point;

projecting the spherical image onto faces of a to cube surrounding the spherical image; and

storing images projected on the faces of the cube to provide an omni-directional image.

2. The method as recited in claim 1, wherein the step of assembling the images in a digital format includes the step of transferring two planar fish-eye images to hemispherical images while removing distortions in the planar fish-eye images.

3. The method as recited in claim 1, wherein the step of assembling the images in the digital format includes the step of removing a demarcation line between the images in the digital format by averaging color and brightness characteristics between the images in the digital format.

4. The method as recited in claim 1, wherein the step of assembling the images in the digital format includes the step of removing a demarcation line between the images in the digital format by employing an error function between pixels of the images.

5. The method as recited in claim 1, wherein the images are taken with a fish-eye lens and further comprising the step of removing a demarcation line by removing pixels from the images in the digital format.

6. The method as recited in claim 1, wherein the step of projecting the spherical image onto faces of a cube surrounding the spherical image includes the step of providing a uniform resolution on a whole surface the cube.

7. The method as recited in claim 1, wherein the step of projecting the spherical image onto faces of a cube surrounding the spherical image includes the step of changing a resolution of the image projected on the cube.

8. The method as recited in claim 1, further comprising the step of displaying a portion of the image projected on the cube.

9. The method as recited in claim 8, wherein the step of displaying includes the step of resealing the image projected on the cube to provide a visualized image that creates a sensation of rectilinear movement in the visualized image.

10. The method as recited in claim 1, further comprising the step of dynamically exploring at least portions of a complete image projected on the faces of the cube.

11. The method as recited in claim 10, wherein the step of dynamically exploring includes providing rotations and translations in all degrees of freedom to permit exploring of the cube in a three-dimensional space.

12. The method as recited in claim 1, further comprising the step of transferring from one omni-directional image to another.

13. The method as recited in claim 1, wherein the images are captured by a fish-eye lens and further comprising the step of calibrating the fish-eye lens to determine radial distortion coefficients for determining specific distortions of the lens.

14. The method as recited in claim 1, further comprising the step of providing a user interface to permit manipulation of a displayed portion of the omni-directional image.

15. The method as recited in claim 1, further comprising the step of determining a center of the image to eliminate geometric distortions.

16. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps as recited in claim 1.

18. The method as recited in claim 17, wherein the step of converting the two fish-eye images to a digital format includes the step of transferring planar fish-eye images to a hemispherical image to remove distortions.

19. The method as recited in claim 17, wherein the step of assembling the two fish-eye images in the digital format to create a spherical image surrounding the origin point includes the step of removing a demarcation line between the two fish-eye images in the digital format by averaging color and brightness characteristics between the two fish-eye images in the digital format.

20. The method as recited in claim 19, wherein the step of removing a demarcation line includes removing halo regions from the two fish-eye images in the digital format.

21. The method as recited in claim 17, wherein the step of projecting the spherical image onto faces of a cube surrounding the spherical image includes the step of providing a uniform resolution on a whole surface the cube.

22. The method as recited in claim 17, further comprising the step of changing a resolution of the omni-directional image projected on the cube.

23. The method as recited in claim 17, further comprising the step of resealing the omni-directional image projected on the cube to provide a visualized image that creates a sensation of rectilinear movement in the visualized image.

24. The method as recited in claim 17, further comprising the step of dynamically exploring at least portions of the omni-directional image on the faces of the cube.

25. The method as recited in claim 24, wherein the step of dynamically exploring includes providing rotations and translations in all degrees of freedom to permit exploring of the cube in a three-dimensional space.

26. The method as recited in claim 17, wherein the step of transferring a portion of the images includes the step of transferring from one omni-directional image to another.

27. The method as recited in claim 17, further comprising the step of calibrating the fish-eye lens to determine radial distortion coefficients for determining specific distortions of the lens.

28. The method as recited in claim 17, wherein the step of assembling the two fish-eye images includes the step of removing a demarcation line between the images in the digital format by employing an error function between pixels of the images.

29. The method as recited in claim 17, further comprising the step of determining a center of the image to eliminate geometric distortions.

30. A program storage device readable by machine, tangibly embodying a program of instructions executable by the machine to perform method steps as recited in claim 17.

17. A method for digitally rendering omni-directional images comprising the steps of:

capturing two fish-eye images, the fish-eye images being taken 180 degrees apart with respect to a plane from an origin point;

converting the two fish-eye images to a digital format;

assembling the two fish-eye images in the digital format to create a spherical image surrounding the origin point;

projecting the spherical image onto faces of a cube surrounding the spherical image;

storing images projected on the faces of the cube; and

transferring a portion of the images projected on the faces of the cube to provide an omni-directional image to for visualization.

31. A system for digitally rendering omni-directional images, comprising:

a computer device which receives digital images of a spherical region about an origin point;

the computer device including a program storage device readable by the computer device, tangibly embodying a program of instructions executable by the computer device to;

assemble the images to create a complete spherical image surrounding the origin point; and

project the spherical image onto faces of a cube which surrounds the spherical image; and

a display coupled to the computer device for displaying an omni-directional image mapped on the faces of the cube such that an entire surface of the cube is capable of being explored.

32. The system as recited in claim 31, further comprising a user interface to permit manipulation of a displayed portion of the omni-directional image.

33. The system as recited in claim 32, wherein the user interface permits resealing of the image projected on the cube to provide a visualized image that creates a sensation of rectilinear movement in the visualized image.

34. The system as recited in claim 32, wherein the user interface permits dynamical exploration of at least portions of a complete image projected on the faces of the cube.

35. The system as recited in claim 31, wherein the manipulation of a displayed portion of the omni-directional image includes rotation and rectilinear motion in all directions.

36. The system as recited in claim 31, further comprising a camera including a fish-eye lens which captures two images encompassing a sphere surrounding the origin point.

37. The system as recited in claim 31, wherein the program storage device stores color and brightness for points in the images.

38. The system as recited in claim 31, wherein the faces of the cube include a uniform resolution on a whole surface the cube.

39. The system as recited in claim 31, wherein the system permits rotations and translations of the displayed image in all degrees of freedom to permit exploring of the cube in a three-dimensional space.

40. The system as recited in claim 31, wherein the system permits a transfer from one omni-directional image to another.