SYSTEM AND METHOD FOR CREATING INTERACTIVE PANORAMIC WALK-THROUGH APPLICATIONS

• US 20110211040A1
• Filed: 11/05/2009
• Published: 09/01/2011
• Est. Priority Date: 11/05/2008
• Status: Abandoned Application

• Current Assignee: David Lindemann, Gérard Crittin, Pierre-Alain Lindemann
• Original Assignee: David Lindemann, Gérard Crittin, Pierre-Alain Lindemann
• Inventors: Crittin, Gérard, Lindemann, Pierre-Alain, Lindemann, David
• Application Number:
  US13/127,474
• Publication Number:
  US 20110211040A1
• Time in Patent Office: Days
• US Class Current: 348/36
• CPC Class Codes: G01C 11/02 Picture taking arrangements specially adapted for photogrammetry or photographic surveying, e.g. controlling overlapping of pictures
• CPC Class Codes: G01S 19/14 specially adapted for specific applications
• CPC Class Codes: G03B 37/06 involving anamorphosis G03B37/02, G03B37/04 take precedence
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06T 3/12 Panospheric to cylindrical image transformations

Abstract

System and method of the invention provides for creating, storing and broadcasting interactive panoramic walk-through applications. The combination of images is determined by the user'"'"'s choice of direction of displacement at each intersection point and from each view point or geographical coordinate, in order to provide a complete view from a first person'"'"'s point of view. The system provides a visual perspective comparable to a human visual experience.

Claims

1. A method for creating panoramic images for use in walk-through applications comprises the steps of:

(a) capturing panoramic images using a panoramic optic system (30, 30′

);

(b) unwarping images, comprising the sub-steps of;

(i) centering and aligning the image (210) relative to a grid;

(ii) opening up the images after slicing a section from the image center to an edge to form an image (211) having a circular arc shape; and

(iii) unwarping the circular arc shape image to form an unwarped image (212) of rectangular shape;

(c) vertical stretching (322) of the image proportional to the divergence of the field of view to the horizon of the optical system; and

(d) expanding horizontal edges (332), comprising the sub-steps of

(i) cropping and copying a section of the image (222) located on the left or right edge of the image; and

(ii) pasting said section on the opposite edge of the image to form an expanded image (232), so that two identical sub-sections of the image appear on opposite sides of the image.

2. The method of claim 1 wherein the vertical stretching of image step (322) comprises the sub-steps of:

(i) setting up a data buffer corresponding to a destination image (222);

(ii) using a function curve to determine correspondence between coordinates in the source image (212) and destination image (222); and

(iii) copying pixel values of source image (212) to pixel values at corresponding coordinates on destination image (222) on a pixel per pixel basis, using an image sampling method selected from a group consisting of “

median”

, “

summed area”

, “

bilinear” and

“

trilinear”

image sampling methods.

6. The method of claim 2 wherein the pixel values comprise values selected from a group consisting of “

Hue, Lightness, Saturation values”

, “

CMYK values” and

“

pixel RGB values”

.

10. The method of claim 1, wherein the image modification steps further include the step of automatic blurring of portion of images, such as faces or numerical car plates.

11. The method of claim 1, wherein image modification steps processes images in two resolutions;

low resolution for interactive walkthrough panoramic motion, and high resolution for interactive panoramic motion.

12. A system implementing the method of claim 1, comprising

(a) an image capture apparatus (20) for capturing panoramic images, further comprising a camera (40), an optic system (30, 30′

) connected to the camera, the optic system providing a substantially 360°

field of view, and a memory device (50);

(b) a communication device for transferring data from the memory device to a computer (80);

(d) instructions executable on a computer for performing image modification steps; and

(e) a database for storing image related data.

13. The system of claim 12, wherein the image capture apparatus further comprises a memory device to store data including images, geographic coordinate information related to said images, and distance information.

14. The system of claim 12, wherein the image capture system comprises a location measurement device (GPS) and a distance measurement device.

15. The system of claim 12, wherein the image capture system comprises an inertial measurement unit for measuring rates of acceleration and changes in rotational attributes of the vehicle, fixation means or camera.

16. The system of claim 12, wherein the image related data includes the image, date and time of image capture, geographical coordinate information, project or group related information, so as to identify for example a particular district or street, and camera settings such as aperture, speed and ISO.

17. The system of claim 1, wherein the computer which executes instructions for performing image modification steps, processes images in two resolutions;

low resolution for interactive walkthrough panoramic motion, and high resolution for interactive panoramic motion.

19. A system implementing the method of claim 2, comprising

(a) an image capture apparatus (20) for capturing panoramic images, further comprising a camera (40), an optic system (30, 30′

) connected to the camera, the optic system providing a substantially 360°

field of view, and a memory device (50);

(b) a communication device for transferring data from the memory device to a computer (80);

(d) instructions executable on a computer for performing image modification steps; and

(e) a database for storing image related data.

3. A method for computing a virtual 360 degree panoramic image from a virtual 360 degree scene (240), for use in walk-through applications, comprising the steps of:

(a) rendering (253) the scene in four sections (244), each having a 90°

horizontal field of view, each section being oriented, from the nodal point (241), at a view angle of 90°

, from the preceding and following sections, said sections being represented in a single image (250) of rectangular shape;

(b) vertically stretching (352) the single image, section per section, along a sinusoidal curve; and

(c) removing (354) upper and lower edges sections to provide a resulting image (254) of rectangular shape.

4. The method of claim 3, further comprising the additional step of expanding (332), horizontal edges comprising the sub-steps of:

(a) cropping and copying a section of the image (254) located on the horizontal edge of the image; and

(b) pasting the section on the opposite edge of the image to form an elongated image (232) which then comprises the section on both of its opposite sides.

5. The method of claim 3 wherein the vertical stretching step (352) comprises the sub-steps of:

(i) setting up a data buffer corresponding to a destination image (252);

(ii) sinusoidal stretching (352) using a function curve to compute the vertical coordinates in the destination image (252) to find the corresponding coordinates in the source image (250); and

(iii) copying pixel values of source image (250) to pixel values at corresponding coordinates on destination image (252) on a pixel per pixel basis, using an image sampling method selected from a group consisting of “

median”

, “

summed area”

, “

bilinear” and

“

trilinear”

image sampling methods.

18. The method of claim 5 wherein the pixel values comprise values selected from a group consisting of “

Hue, Lightness, Saturation values”

, “

CMYK values” and

“

pixel RGB values”

.

20. A system implementing the method of claim 3, comprising

(a) an image capture apparatus (20) for capturing panoramic images, further comprising a camera (40), an optic system (30, 30′

) connected to the camera, the optic system providing a substantially 360°

field of view, and a memory device (50);

(b) a communication device for transferring data from the memory device to a computer (80);

(d) instructions executable on a computer for performing image modification steps; and

(e) a database for storing image related data.

7. A method for computing a virtual 360 degree panoramic image based on two points perspective, for use in assembling and broadcasting of walkthrough applications, providing panoramic images closer to human vision during panoramic motion, comprising the steps of:

(a) dividing the viewable portion (234) of a given panoramic image (232), in vertical slices (235);

(b) stretching said vertical slices (235) along sinusoidal lines to recover a two points perspective, wherein a group of vertical slices is in a horizontal fixed position relative to a viewport;

(c) providing vertical panoramic interactive motion by vertical translation of said vertical slices (235) without affecting sinusoidal distortion, wherein said group of vertical slices are vertically compressed proportional to the angular divergence of the field of view to the horizon, in order to minimise vertical distortion of two points perspective.

8. The method of claim 7, wherein said vertical stripes are of non-uniform width.

9. The method of claim 7, wherein said steps are performed in real time and broadcasted by a web server (82) to a user terminal, in response to user motion in a walkthrough space.