PERSONALIZED NAVIGATION THROUGH VIRTUAL 3D ENVIRONMENTS

• US 20110310088A1
• Filed: 06/17/2010
• Published: 12/22/2011
• Est. Priority Date: 06/17/2010
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Adabala, Neeharika, Prjapathi, Komal, Bagga, Gurpreet Singh
• Application Number:
  US12/817,497
• Publication Number:
  US 20110310088A1
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images

Abstract

Personalized navigation technique embodiments are presented that generally involve providing personalized navigation through a virtual three-dimensional (3D) environment. In one general embodiment this is accomplished by inputting a representation of a virtual 3D environment that is to be navigated, along with a number of user specified navigation preferences, and outputting a customized navigation experience. This navigation experience is produced by first establishing a path through the virtual 3D environment, and then optionally controlling the behavior of a virtual camera which reveals the virtual 3D environment to the user as the user traverses the path. Both the path and the virtual camera behavior are personalized to a user based on the aforementioned navigation preferences.

Claims

1. A computer-implemented process for providing personalized navigation through a virtual three-dimensional (3D) environment, comprising:

using a computer to perform the following process actions;

inputting a representation of the virtual 3D environment;

inputting user specified navigation preferences; and

establishing a path through the virtual 3D environment which is customized using the user specified navigation preferences.

2. The process of claim 1, wherein the representation of the virtual 3D environment is in the form of a photosynth comprising images of the environment, feature point cloud data and keyframe point data, wherein the feature point cloud data comprises a set of 3D matching points identified among the images making up the photosynth, and wherein the keyframe point data represents locations where a camera was situated when capturing a photosynth image.

3. The process of claim 2, wherein the process action of establishing a path through the virtual 3D environment, comprises the actions of:

defining a region in the virtual environment where the path will be laid out;

building a region of interest grid from the defined region; and

generating a course road-map path in the form of a graph through the virtual 3D environment based on the region of interest grid.

4. The process of claim 3, wherein the process action of defining a region in the virtual environment where the path will be laid out, comprises the actions of:

projecting the feature point cloud onto a two-dimensional (2D) X-Y plane;

overlaying the projected feature point cloud with an axis-parallel rectangular grid that extends along each of its X and Y grid axes so as to cover a maximum range of the feature point cloud points;

counting the number of feature point cloud points lying in each cell of grid; and

for each cell of the grid,

determining if the number of feature point cloud points therein is below a prescribed minimum point count threshold, and

whenever it is determined that the number of feature point cloud points in a grid cell is below the prescribed minimum point count threshold, designating the grid cell as an empty grid cell and discarding the feature point cloud points therein.

5. The process of claim 4, wherein the grid has rectangular shaped grid cells and the minimum point count threshold is set to 100*(width/wn)*(height/hn), wherein width and height are the width and height of the grid, and wn=number of grid cells along the width, and hn=number of grid cells along the height.

6. The process of claim 4, wherein the process action of building a region of interest grid from the defined region, comprises the actions of:

establishing a rectangular bounding box on the plane which aligns with the grid, and encompasses all the grid cells which are contiguous with each other and still having feature point cloud points therein;

projecting the keyframe points onto the plane; and

designating the portion of the grid inside the bounding box as a region of interest grid.

7. The process of claim 6, wherein the process action of generating a course road-map path in the form of a graph through the virtual 3D environment based on the region of interest grid, comprises the actions of:

clustering the keyframe points resident inside the region of interest grid into landmark regions;

for each landmark region,

computing a mean location of the region, and

designating the keyframe point location closest to the mean location as a node of a road-map graph;

connect the nearest neighboring nodes of the road-map graph with edges; and

designating the path formed by the road-map graph nodes and edges as the course road-map path through the virtual 3D environment.

8. The process of claim 7, wherein the process action of clustering the keyframe points resident inside the region of interest grid into landmark regions, comprises an action of clustering the keyframe points using an agglomerative hierarchical clustering technique based on the mean Euclidean distance between landmark locations and a prescribed distance threshold as metrics to achieve the clustering.

9. The process of claim 4, wherein the process action of projecting the feature point cloud onto a two-dimensional (2D) X-Y plane, comprises an action of projecting the feature point cloud onto a plane that transects the 3D virtual environment as represented by the feature point cloud of the photosynth such that it is substantially parallel with a prescribed ground level of the environment and at a height above the ground level approximately corresponding to an adult'"'"'s eye-level.

10. The process of claim 3, wherein the process action of establishing a path through the virtual 3D environment, further comprises the actions of:

constructing a region of interest graph; and

refining the course road-map path through the virtual 3D environment in a first refinement stage using the region of interest graph to produce a first refined path which comprises a sequence of nodes and which exhibits a resolution consistent with the region of interest grid.

11. The process of claim 10, wherein the process action of constructing the region of interest graph, comprises the actions of:

establishing a node of the region of interest graph at a corner of each grid cell of the region of interest grid, wherein the same grid cell corner is employed for each node established;

establishing edges between each node and its neighboring nodes, wherein only nodes corresponding to cells adjacent to that associated with a given node are deemed to be neighbors to that node; and

assigning a weight to each edge, wherein the edge weight is computed based on a prescribed set of parameters comprising the number of feature point cloud points and keyframe points in the cell associated with the node where the edge originates, the Euclidean edge length, and the number of feature point cloud points and keyframes in the cell associated with the node where the edge terminates.

12. The process of claim 11, wherein the process action of refining the course road-map path through the virtual 3D environment in the first refinement stage, comprises the actions of:

projecting the course road-map path graph onto the region of interest grid;

establishing a primary node location of a first refined path at a corner of each grid cell of the region of interest grid containing a node of the road-map path graph, wherein the same grid cell corner is employed for each primary node established;

employing an A-star technique to trace the first refined path along either a grid cell line of a grid cell of the region of interest grid or diagonally across a grid cell of the region of interest grid, between each successive primary node location, based on the edge weights computed for the edges in the region of interest graph which correspond to the grid cell lines of the region of interest grid; and

designating each grid cell corner along the first refined path as a node of the first refined path.

13. The process of claim 10, wherein the process action of establishing a path through the virtual 3D environment, further comprises the actions of:

constructing a static force grid; and

refining the first refined path through the virtual 3D environment to give the user a more realistic navigation experience in a second refinement stage to produce a final refined path using the static force grid.

14. The process of claim 13, wherein the process action of constructing a static force grid, comprises the actions of:

for each cell of the region of interest grid,

assigning a charge value to each keyframe and feature point cloud point in the cell, wherein each keyframe is assigned a prescribed attracting charge and each feature point cloud point is assigned a prescribed repulsing charge, and

computing a combined charge at an averaged charge location of the cell, wherein the averaged charge location is computed as the mean location of all the keyframe and feature point cloud points within the cell and the combined charge assigned to the mean location is computed as the sum of the charges assigned to each keyframe and feature point cloud point within the cell weighted by the distance of the point from the averaged charge location of the cell.

15. The process of claim 14, wherein the process action of refining the first refined path through the virtual 3D environment in the second refinement stage, comprises the actions of:

for each segment of the first refined path traversing a grid cell between nodes,

designating the point where the path enters the cell as the source location and the point where the path leaves the cell as the destination location;

assigning the destination location an attractive force;

identifying a set of neighboring grid cells depending on whether the path segment under consideration is horizontal or vertical or diagonal;

identifying the combined charge emanating from the averaged charge location of each cell of the static force grid that corresponds to one of the identified neighboring grid cells;

using a potential field-driven interpolation technique to establish a revised route for the path segment under consideration to take through the grid cell under consideration and a velocity associated with path through the cell, based on the identified combined charges associated with the neighboring cells and the attractive force assigned to the destination location of the grid cell under consideration.

16. The process of claim 15, wherein the process action of identifying the set of neighboring grid cells depending on whether the path segment under consideration is horizontal or vertical or diagonal, comprises the actions of:

identifying the grid cells immediately above and below the grid cell under consideration as the set of neighboring grid cells, whenever the path segment under consideration is horizontal along an edge of the grid cell;

identifying the grid cells immediately before and after the grid cell under consideration as the set of neighboring grid cells, whenever the path segment under consideration is vertical along an edge of the grid cell; and

identifying the grid cells immediately above, below, before and after the grid cell under consideration as the set of neighboring grid cells, whenever the path segment under consideration is diagonal through the grid cell.

17. A computer-implemented process for providing personalized navigation through a virtual three-dimensional (3D) environment, comprising:

using a computer to perform the following process actions;

inputting a representation of the virtual 3D environment;

establishing user navigation preferences; and

controlling the behavior of a virtual camera based on the user navigation preferences as the virtual camera reveals the virtual 3D environment to the user as the user traverses a pre-established a path through the virtual 3D environment.

18. The process of claim 17, wherein the process action of controlling the behavior of a virtual camera based on the user specified navigation preferences, comprises an action of controlling virtual camera motion and camera-based cinematographic effects to reflect personal interests established by the user specified navigation preferences.

19. The process of claim 17, wherein the process action of establishing user navigation preferences, comprises the actions of:

grouping together elementary camera behaviors that when employed during the navigation of the 3D environment reflect personal interests of a user and which are suited for the type of environment being navigated;

inputting information about the personality traits and navigational preferences of the user, as well as the nature of virtual environment being navigated;

matching the inputted information to one of the groupings of elementary camera behaviors; and

designating the grouping of elemental camera behaviors matched to the inputted information as the user navigation preferences.

20. A computer-implemented process for providing personalized navigation through a virtual three-dimensional (3D) environment, comprising:

using a computer to perform the following process actions;

inputting a representation of the virtual 3D environment;

inputting user specified navigation preferences; and

establishing a path through the virtual 3D environment which is customized using the user specified navigation preferences; and

controlling the behavior of a virtual camera based on the user specified navigation preferences, wherein the virtual camera reveals the virtual 3D environment to the user as the user traverses said path.