Image-based method and system for building spherical panoramas

• US 20030063089A1
• Filed: 09/04/2002
• Published: 04/03/2003
• Est. Priority Date: 05/27/1998
• Status: Abandoned Application

• Current Assignee: Ju-Wei Chen, Shu-Cheng Huang, Tse Cheng
• Original Assignee: Ju-Wei Chen, Shu-Cheng Huang, Tse Cheng
• Inventors: Huang, Shu-Cheng, Chen, Ju-Wei, Cheng, Tse
• Application Number:
  US10/235,190
• Publication Number:
  US 20030063089A1
• Time in Patent Office: Days
• US Class Current: 345/473
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 3/047 Fisheye or wide-angle transformations
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/32 using correlation-based methods
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

The invention relates to building spherical panoramas for image-based virtual reality systems. The image-based spherical panoramas can be navigated in any desired view direction (360-degrees) by suitable three-dimensional image browsers or viewers. The method and system also includes computing the number of photographs required to be taken and the azimuth angle of the center point of each photograph for building a spherical environment map representative of the spherical panorama. The method and system also includes an algorithm for computing the accurate azimuth angles of these taken photographs and seaming them together to build the spherical environment map.

Claims

1. A method of photographing pictures of a panoramic scene using a camera system, the pictures used to build a spherical panorama, the method comprising:

determining a number of circles of latitude to build the spherical panorama on the basis of the focal length of the camera system, the height of photographic film used in the camera system, and an overlapping ratio between contiguous ones of the pictures, each circle of latitude including a series of contiguous pictures; and

photographing the determined number of pictures.

2. The method of claim 1 wherein determining the number of circles of latitude includes calculating the following equation:

3. The method of claim 1 wherein the number of circles of latitude is further determined on the basis of the width of the photographic film.

4. The method of claim 3 wherein determining the number of circles of latitude includes calculating the following equation:

5. A method of transforming a plurality of photographic images of a panoramic scene into a spherical environment map, each photographic image taken at a predetermined sight direction characterized by an elevation angle, the method comprising:

warping each of the plurality of photographic images into a parametric spherical environment map; and

image cutting each of the warped photographic images into a rectangular image based on the elevation angle of the sight direction for each of the warped photographic images.

6. The method of claim 5 wherein during image cutting, the warped photographic images are cut either horizontally or vertically on the basis of the elevation angle of the sight direction for each of the warped photographic images.

7. The method of claim 6 wherein:

if the elevation angle of the sight direction for a warped photographic image is less than 10°

or greater than 170°

, the warped photographic image is cut into a rectangular region defined by one horizontal line, which is either a bottom or top edge of the rectangular warped image of the photograph; and

if the elevation angle of the sight direction for a warped photographic image is between 30° and

150°

, the warped photographic image is cut into a rectangular region defined by two opposing horizontal lines and two opposing vertical lines.

8. The method of claim 5 wherein warping of each of the plurality of photographic images includes mapping each pixel of the warped photographic image from at least one pixel of an associated photographic image, the attributes of each pixel of the warped photographic image being derived from at least one pixel of the associated photographic image.

9. A method of stitching together a two dimensional array of photographic images to build a spherical panorama, the method comprising:

computing sight directions for each of the photographic images;

adjusting the intensity of the photographic images to an intensity related to an average of the intensities of the photographic images; and

stitching together contiguous ones of the photographic images.

10. The method of claim 9 wherein computing sight directions includes:

performing vertical registration of contiguous photographic images for each column of the two dimensional array; and

performing horizontal registration of contiguous photographic images for each row of the two dimensional array.

11. The method of claim 10 wherein performing horizontal and vertical registration includes using a correlation-based algorithm which computes image phase correlation for all possible positions of image alignment and selecting the one with the minimum alignment difference.

12. The method of claim 10 further including inspecting the results of performing horizontal and vertical registration with a software tool.

13. The method of claim 9 wherein performing vertical registration of contiguous photographic images for each column of the two dimensional array includes:

rotating each photographic image 90°

;

image warping the rotated photographic image with φ

_L set to 90°

;

performing horizontal registration of contiguous photographic images for each column after being rotated 900 clockwise of the two dimensional array; and

determining sight directions from the image positions of image registration.

14. The method of claim 10 wherein horizontal registration is performed after vertical registration and further including performing a second “

fine-tuning”

registration.

15. The method of claim 9 wherein stitching together contiguous photographic images further includes image blending overlapping regions of the contiguous photographic images.

16. The method of claim 9 wherein image blending between contiguous photographic images includes determining intensity levels of pixels in the overlapping regions using the following equation:

17. A system for photographing pictures of a panoramic scene for use in building a spherical panorama, the system comprising:

a camera system for photographing the determined number of pictures; and

a computer-readable medium including computer instructions for determining a number of circles of latitude to build the spherical panorama on the basis of the focal length of the camera system, the height of photographic film used in the camera system, and an overlapping ratio between contiguous ones of the pictures, each circle of latitude including a series of contiguous pictures.

18. The system of claim 17 wherein the computer-readable medium further includes computer instructions for determining the number of circles of latitude including calculating the following equation:

19. The system of claim 17 wherein the computer-readable medium further includes computer instructions for determining the number of circles of latitude on the basis of the width of the photographic film.

20. The system of claim 19 wherein the computer-readable medium further includes computer instructions for determining the number of circles of latitude includes calculating the following equation:

21. A system for transforming a plurality of photographic images of a panoramic scene into a spherical environment map, each photographic image taken at a predetermined sight direction characterized by an elevation angle, the system comprising:

a computer-readable medium including;

computer-readable instructions for warping each of the plurality of photographic images into a parametric spherical environment map; and

computer-readable instructions for cutting each of the warped photographic images into a rectangular image based on the elevation angle of the sight direction for each of the warped photographic images.

22. The system of claim 21 wherein the computer-readable medium further includes computer instructions for cutting the warped photographic images during image cutting either horizontally or vertically on the basis of the elevation angle of the sight direction for each of the warped photographic images.

23. The system of claim 22 wherein the computer-readable medium further includes computer instructions for:

cutting, if the elevation angle of the sight direction for a warped photographic image is less than 10°

or greater than 170°

, the warped photographic image into a rectangular region defined by one horizontal line, which is either a bottom or top edge of the rectangular warped image of the photograph; and

cutting, if the elevation angle of the sight direction for a warped photographic image is between 30° and

150°

, the warped photographic image into a rectangular region defined by two opposing horizontal lines and two opposing vertical lines.

24. The system of claim 21 wherein the computer instructions for warping each of the plurality of photographic images includes computer instructions for mapping each pixel of the warped photographic image from at least one pixel of an associated photographic image, the attributes of each pixel of the warped photographic image being derived from at least one pixel of the associated photographic image.

25. A system for stitching together a two dimensional array of photographic images to build a spherical panorama, the system comprising a computer-readable medium including:

computer instructions for computing sight directions for each of the photographic images;

computer instructions for adjusting the intensity of the photographic images to an intensity related to an average of the intensities of the photographic images; and

computer instructions for stitching together contiguous ones of the photographic images.

26. The system of claim 25 wherein the computer instructions for computing sight directions includes:

computer instructions for performing vertical registration of contiguous photographic images for each column of the two dimensional array; and

computer instructions for performing horizontal registration of contiguous photographic images for each row of the two dimensional array.

27. The system of claim 26 wherein the computer instructions for performing horizontal and vertical registration includes a correlation-based algorithm which computes image phase correlation for all possible positions of image alignment and selecting the one with the minimum alignment difference.

28. The system of claim 26 further including computer instructions for inspecting the results of performing horizontal and vertical registration with a software tool.

29. The system of claim 25 wherein the computer instructions for performing vertical registration of contiguous photographic images for each column of the two dimensional array includes:

computer instructions for rotating each photographic image 90°

;

image warping the rotated photographic image with φ

_L set to 90°

;

computer instructions for performing horizontal registration of contiguous photographic images for each column after above rotation of the two dimensional array; and

computer instructions for determining sight directions from the image positions of image registration.

30. The system of claim 26 wherein the computer instructions for performing horizontal registration and vertical registration further includes computer instructions for performing a second “

fine-tuning”

registration.

31. The system of claim 25 wherein the computer instructions for stitching together contiguous on of the photographic images further includes computer instructions for image blending overlapping regions of the contiguous photographic images.

32. The system of claim 25 wherein the computer instructions for image blending between contiguous photographic images includes computer instructions for determining intensity levels of pixels in the overlapping regions using the following equation: