Method and apparatus for creating interactive virtual tours
First Claim
1. An apparatus for creating, managing and publishing interactive virtual tours, the apparatus comprising:
- a panorama data acquisition unit implementing means of capturing panoramic data representing a plurality of environment maps and preparing said panoramic data for further processing, wherein said panoramic data acquisition unit comprises at least one versatile device for creating representations of stimuli covering substantially all directions around a given reference or view point, said versatile device for creating representations of stimuli comprising at least one grid of one or more focusing elements disposed on an N-dimensional shaped surface, at least one grid of one or more sensor elements disposed on an N-dimensional shaped surface, and at least one grid of one or more stimulus guide elements disposed on an N-dimensional shaped surface, wherein said focusing element grid is adapted to focus stimuli covering substantially all directions around a given reference or view point onto the sensor element grid via the stimulus guide unit, wherein each focusing element or group of focusing elements is associated with and focuses a subset of the entire stimulus space onto a sensor element or group of sensor elements responsive to the stimuli, wherein when a stimulus guide element grid is provided, the focusing element grid is adapted to focus the stimuli onto the stimulus guide element grid for formatting and onward transmission to the sensor element grid, wherein when used, each stimulus guide element or group of elements is associated with and receives stimuli from a focusing element or group of focusing elements and is in turn associated with and transmits stimuli to a sensor element or group of sensor elements;
a transform engine responsive to said panoramic data and implementing means of correcting distortions in said panoramic data and/or performing automatic, manual or interactive calibration of said panoramic data and/or transforming said panoramic data into a desired format or set of formats when necessary, wherein said distortion correction is accomplished through steps of;
a) loading data including reference data that could be used to derive panoramic imaging system distortion profiles;
b) specifying a linear or other predetermined distortion function or set of distortion functions and displaying representations of said distortion function or set of distortion functions;
c) using distortion function or set of distortion functions to build a distortion profile for the panoramic imaging system that was used to acquire the reference data;
d) performing transformation on panoramic data using distortion profile specified in c) and displaying results of said transformation;
e) determining whether distortion is satisfactorily corrected;
f) continuing with steps g) and h) if distortion is deemed not to have been satisfactorily corrected and continuing with step i) if distortion is deemed to have been satisfactorily corrected;
g) automatically or interactively modifying the distortion function or set of distortion functions using feedback from displayed distortion function representation or transformation results obtained in step d) or by using data loaded in step a);
h) repeating steps c) to g) until distortion is judged to have been satisfactorily corrected;
i) storing distortion profile obtained in step c) for use in performing transformations on panoramic data acquired using the panoramic imaging system for which data was loaded in step a);
a package generator adapted to generate virtual tour packages containing said panoramic data, commands and/or virtual tour data;
a viewing engine responsive to said panoramic data and/or virtual tour packages and implementing means for perspective correction, and user interaction with, said panoramic data and/or virtual tour data when necessary;
a control engine adapted to facilitate a higher level of interaction with said panoramic data and virtual tour data, wherein said control engine is connected operatively to and communicates bi-directionally with said viewing engine, renders representative information about all or parts of said virtual tour, permits a particular portion to be selected from said virtual tour and sends signals to said viewing engine that cause said viewing engine to permit the interactive navigation of said selected portion of said virtual tour, wherein said control engine also indicates or causes to be indicated what portion of said virtual tour is currently selected and what sub-part of said selected portion of said virtual tour is currently rendered, wherein said control engine is responsive to user input and commands from said viewing engine and is in turn capable of modifying said representative information about all or parts of said virtual tour in response to said user input and said commands from said viewing engine and is further capable of communicating information indicative of such externally induced modifications to said user and said viewing engine;
a display means for rendering output of said viewing engine, control engine, package generator, transform engine, and/or panoramic data acquisition unit.
0 Assignments
0 Petitions
Accused Products
Abstract
The present invention discloses a method and apparatus for creating interactive virtual tours. In one embodiment, panoramic imaging systems characterized by a 360-degree lateral field of view and a vertical field of view that is usually less than 180 degrees are utilized and polynomial-based interpolation algorithms are used to correct distortions in the input panoramic images, video or rendered scenes, permitting the use of an arbitrary panoramic imaging system characterized by the ability to generate a seamless 360-degree panoramic view of the imaged scene in each single image frame. The input panoramic images, video or rendered scenes are used to create spherical environment maps that are packaged into completely immersive (giving the impression of being there) virtual tours that are rendered on a display device by a viewing engine that corrects perspective distortions in the spherical environment maps contained in the virtual tour packages and permits a plurality of viewers to view any portion of any of the panoramic scenes in the virtual tour package in a manner that is free from distortions. The present invention also discloses a control engine for representing the individual panoramic scenes contained in the virtual tour package in multi-dimensional space and/or time and for transmitting control signals to the viewing engine that cause the viewing engine to render any particular panoramic scene selected by the viewer and any view window on the selected panoramic scene desired by the viewer. Furthermore, the control engine communicates bi-directionally with the viewing engine and provides a means of indicating which particular panoramic scene is currently viewed by the viewing engine as well as a means of indicating what portion of the selected panoramic scene is currently displayed by the viewing engine.
-
Citations
44 Claims
-
1. An apparatus for creating, managing and publishing interactive virtual tours, the apparatus comprising:
-
a panorama data acquisition unit implementing means of capturing panoramic data representing a plurality of environment maps and preparing said panoramic data for further processing, wherein said panoramic data acquisition unit comprises at least one versatile device for creating representations of stimuli covering substantially all directions around a given reference or view point, said versatile device for creating representations of stimuli comprising at least one grid of one or more focusing elements disposed on an N-dimensional shaped surface, at least one grid of one or more sensor elements disposed on an N-dimensional shaped surface, and at least one grid of one or more stimulus guide elements disposed on an N-dimensional shaped surface, wherein said focusing element grid is adapted to focus stimuli covering substantially all directions around a given reference or view point onto the sensor element grid via the stimulus guide unit, wherein each focusing element or group of focusing elements is associated with and focuses a subset of the entire stimulus space onto a sensor element or group of sensor elements responsive to the stimuli, wherein when a stimulus guide element grid is provided, the focusing element grid is adapted to focus the stimuli onto the stimulus guide element grid for formatting and onward transmission to the sensor element grid, wherein when used, each stimulus guide element or group of elements is associated with and receives stimuli from a focusing element or group of focusing elements and is in turn associated with and transmits stimuli to a sensor element or group of sensor elements; a transform engine responsive to said panoramic data and implementing means of correcting distortions in said panoramic data and/or performing automatic, manual or interactive calibration of said panoramic data and/or transforming said panoramic data into a desired format or set of formats when necessary, wherein said distortion correction is accomplished through steps of; a) loading data including reference data that could be used to derive panoramic imaging system distortion profiles; b) specifying a linear or other predetermined distortion function or set of distortion functions and displaying representations of said distortion function or set of distortion functions; c) using distortion function or set of distortion functions to build a distortion profile for the panoramic imaging system that was used to acquire the reference data; d) performing transformation on panoramic data using distortion profile specified in c) and displaying results of said transformation; e) determining whether distortion is satisfactorily corrected; f) continuing with steps g) and h) if distortion is deemed not to have been satisfactorily corrected and continuing with step i) if distortion is deemed to have been satisfactorily corrected; g) automatically or interactively modifying the distortion function or set of distortion functions using feedback from displayed distortion function representation or transformation results obtained in step d) or by using data loaded in step a); h) repeating steps c) to g) until distortion is judged to have been satisfactorily corrected; i) storing distortion profile obtained in step c) for use in performing transformations on panoramic data acquired using the panoramic imaging system for which data was loaded in step a); a package generator adapted to generate virtual tour packages containing said panoramic data, commands and/or virtual tour data; a viewing engine responsive to said panoramic data and/or virtual tour packages and implementing means for perspective correction, and user interaction with, said panoramic data and/or virtual tour data when necessary; a control engine adapted to facilitate a higher level of interaction with said panoramic data and virtual tour data, wherein said control engine is connected operatively to and communicates bi-directionally with said viewing engine, renders representative information about all or parts of said virtual tour, permits a particular portion to be selected from said virtual tour and sends signals to said viewing engine that cause said viewing engine to permit the interactive navigation of said selected portion of said virtual tour, wherein said control engine also indicates or causes to be indicated what portion of said virtual tour is currently selected and what sub-part of said selected portion of said virtual tour is currently rendered, wherein said control engine is responsive to user input and commands from said viewing engine and is in turn capable of modifying said representative information about all or parts of said virtual tour in response to said user input and said commands from said viewing engine and is further capable of communicating information indicative of such externally induced modifications to said user and said viewing engine; a display means for rendering output of said viewing engine, control engine, package generator, transform engine, and/or panoramic data acquisition unit. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22)
-
-
23. A method of creating, managing and publishing interactive virtual tours, said method comprising:
-
a panorama data acquisition step is performed by a panoramic image acquisition unit for capturing panoramic data representing a plurality of environment maps and preparing said panoramic data for further processing, wherein said panoramic data acquisition step utilizes at least one versatile device for creating representations of stimuli covering substantially all directions around a given reference or view point, said versatile device for creating representations of stimuli comprising at least one grid of one or more focusing elements disposed on an N-dimensional shaped surface, at least one grid of one or more sensor elements disposed on an N-dimensional shaped surface, and at least one grid of one or more stimulus guide elements disposed on an N-dimensional shaped surface, wherein said focusing element grid is adapted to focus stimuli covering substantially all directions around a given reference or view point onto the sensor element grid via the stimulus guide unit, wherein each focusing element or group of focusing elements is associated with and focuses a subset of the entire stimulus space onto a sensor element or group of sensor elements responsive to the stimuli, wherein when a stimulus guide element grid is provided, the focusing element grid can be adapted to focus the stimuli onto the stimulus guide element grid for formatting and onward transmission to the sensor element grid, wherein when used, each stimulus guide element or group of elements is associated with and receives stimuli from a focusing element or group of focusing elements and is in turn associated with and transmits stimuli to a sensor element or group of sensor elements; a transform step for correcting distortions in said panoramic data and performing automatic, manual or interactive calibration of said panoramic data and transforming said panoramic data into a desired format or set of formats when necessary, wherein said transform step for correcting distortions is accomplished through steps of; a) loading data including reference data that could be used to derive panoramic imaging system distortion profiles; b) specifying a linear or other predetermined distortion function or set of distortion functions and displaying representations of said distortion function or set of distortion functions; c) using distortion function or set of distortion functions to build a distortion profile for the panoramic imaging system that was used to acquire the reference data; d) performing transformation on panoramic data using distortion profile specified in c) and displaying results of said transformation; e) determining whether distortion is satisfactorily corrected; f) continuing with steps g) and h) if distortion is deemed not to have been satisfactorily corrected and continuing with step i) if distortion is deemed to have been satisfactorily corrected; g) automatically or interactively modifying the distortion function or set of distortion functions using feedback from displayed distortion function representation or transformation results obtained in step d) or by using data loaded in step a); h) repeating steps c) to g) until distortion is judged to have been satisfactorily corrected; i) storing distortion profile obtained in step c) for use in performing transformations on panoramic data acquired using the panoramic imaging system for which data was loaded in step a); a package generation step adapted to generate virtual tour packages containing said panoramic data, commands and/or optional virtual tour data; a viewing step responsive to said panoramic data and/or virtual tour packages and providing means for perspective correction of, and user interaction with, said panoramic data and/or virtual tour data when necessary; a control step adapted to facilitate a higher level of interaction with said panoramic data and/or virtual tour data, wherein said control step is connected operatively to and communicates bi-directionally with said viewing step, renders representative information about all or parts of said virtual tour, permits a particular portion to be selected from said virtual tour and sends signals to said viewing step that cause said viewing step to permit the interactive navigation of said selected portion of said virtual tour, wherein said control step also indicates or causes to be indicated what portion of said virtual tour is currently selected and what sub-part of said selected portion of said virtual tour is currently rendered, wherein said control step is responsive to user input and commands from said viewing step and is in turn capable of modifying said representative information about all or parts of said virtual tour in response to said user input and said commands from said viewing step and is further capable of communicating information indicative of such externally induced modifications to said user and said viewing step; a display step providing means for rendering output of said viewing step, control step, package generation step, transform step, and/or panoramic data acquisition step. - View Dependent Claims (24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44)
-
Specification