Multi-modal method for interacting with 3D models

• US 9,786,097 B2
• Filed: 06/24/2013
• Issued: 10/10/2017
• Est. Priority Date: 06/22/2012
• Status: Active Grant

• Current Assignee: Matterport, Inc
• Original Assignee: Matterport, Inc
• Inventors: Bell, Matthew, Beebe, Michael
• Primary Examiner(s): Crawford, Jacinta M
• Assistant Examiner(s): NGUYEN, PHONG X
• Application Number:
  US13/925,772
• Publication Number:
  US 20130342533A1
• Time in Patent Office: 1,569 Days
• Field of Search: 345607
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2210/04 Architectural design, interior design

Abstract

The present disclosure concerns a methodology that allows a user to “orbit” around a model on a specific axis of rotation and view an orthographic floor plan of the model. A user may view and “walk through” the model while staying at a specific height above the ground with smooth transitions between orbiting, floor plan, and walking modes.

Claims

1. A system, comprising:

a memory that stores computer executable components;

a processor that executes at least the following computer executable components stored in the memory;

an interface component that configures a graphical user interface that facilitates viewing and navigating a three-dimensional model from different perspectives of a virtual camera, wherein the different perspectives are based on different navigation modes including;

a floor plan mode that provides perspectives of the three-dimensional model from the virtual camera positioned outside and above the three-dimensional model and with a view of the three-dimensional model that has a pitch direction of the view locked substantially orthogonally orientated relative to a horizontal surface of the three-dimensional model, and wherein the floor plan mode enables spatial motion of the virtual camera while maintaining the substantially orthogonal orientation of the view relative to the horizontal surface;

an orbit mode that provides perspectives of the three-dimensional model from the virtual camera positioned outside of the three-dimensional model based upon a point of the three-dimensional model, and

a walking mode that provides perspectives of the three-dimensional model from the virtual camera positioned at an approximately consistent distance relative to a plane based on the three-dimensional model;

a rendering component configured to generate representations of the three-dimensional model from the different perspectives based on user input indicating position and orientation of the virtual camera and a selected mode;

wherein the interface component is further configured to, in response to a determination that a user volume associated with a target position creates a conflict with a portion of the three-dimensional model, iteratively place a bottom of the user volume at a local floor height at points of a series of successively larger areas surrounding the target position until a point is located that resolves the conflict, and set the point as the target position; and

a transition component configured to receive a request to transition from a first perspective of the three-dimensional model in a first mode of the different navigation modes to a second perspective of the three-dimensional model in a second mode of the different navigation modes, wherein based on the request the transition component is configured to determine a second set of parameters corresponding to the target position and orientation of the virtual camera relative to the three-dimensional model at the second perspective based in part on a first set of parameters corresponding to a position or orientation of the virtual camera relative to the three-dimensional model at the first perspective, and employ a transition function to interpolate from the first set of parameters to the second set of parameters to generate a smooth visual transition between a first representation of the three-dimensional model at the first perspective to a second representation of the three-dimensional model at the second perspective, wherein the transition function provides an accelerating portion of an interpolation between two values followed by a decelerating portion of the interpolation between the two values.

2. The system of claim 1, wherein the transition component is configured to employ interpolation over a time interval to generate the smooth visual transition between the first representation of the three-dimensional model and the second representation of the three-dimensional model.

3. The system of claim 1, wherein the perspectives associated with the walking mode simulate positions of a virtual user located on the plane.

4. The system of claim 1, further comprising:

a presentation component configured to display the representations of the three-dimensional model via a display screen of a device.

5. The system of claim 1 wherein, during orbit mode, the rendering component employs backface culling or a ceiling removal technique to generate a representation of the three-dimensional model at a perspective of the interior space from a point outside of the three-dimensional model.

6. The system of claim 1, wherein the orbit mode provides perspectives of the three-dimensional model from the virtual camera positioned outside of the three-dimensional model and relative to the point on, wherein the virtual camera is configured to rotate around a vertical axis at the point at various distances from the point, at various angles and pitches relative to the vertical axis.

7. The system of claim 1, wherein the orbit mode provides perspectives of the three-dimensional model as a position of the virtual camera moves horizontally relative to the three-dimensional model in response to user input.

8. The system of claim 1, wherein the orbit mode provides perspectives of the three-dimensional model as a position of the virtual camera moves toward or away from the point in response to user input.

9. The system of claim 1, wherein the orbit mode provides perspectives of the three-dimensional model as an orientation of the virtual camera changes in response to user input.

10. The system of claim 1, wherein the walking mode provides perspectives of the three-dimensional model as the virtual camera moves relative to defined surfaces of the three-dimensional model, in response to user input, to simulate a user walking through the three-dimensional model.

11. The system of claim 10, wherein the walking mode provides perspectives of the three-dimensional model based on user input identifying a direction in which the user is walking through the three-dimensional model.

12. The system of claim 10, wherein a position of the user relative to the three-dimensional model is based in part on a height of the user relative to the plane, wherein the height of the user is configurable in response to user input.

13. The system of claim 10, further comprising a collision checking component that prevents virtual collision between the user and three-dimensional boundaries of the three-dimensional model.

14. The system of claim 1, wherein the walking mode provides perspectives of the three-dimensional model as an orientation of the virtual camera changes in response to user input.

15. The system of claim 1, wherein the rendering component employs backface culling or a ceiling removal technique to generate a representation of the three-dimensional model in the floor plan mode.

16. The system of claim 1, wherein the first mode comprises orbit mode and the second mode comprises walking mode, wherein the transition component is configured to determine the second set of parameters based on user input selecting a location on the three-dimensional model and a defined surface of the three-dimensional model near the location.

17. The system of claim 16, wherein the transition component is configured to determine the second set of parameters using ray casting from the position of the virtual camera relative to the three-dimensional model at the first perspective to the location on the three-dimensional model near the defined surface.

18. The system of claim 1, wherein the first mode comprises walking mode and the second mode comprises orbit mode, wherein the transition component is configured to determine the second set of parameters based on a default configuration for the target position or orientation of the virtual camera in orbit mode.

19. The system of claim 1, wherein the first mode comprises walking mode and the second mode comprises orbit mode, wherein the transition component is configured to determine the second set of parameters based on a preset distance directly above the position of the virtual camera relative to the three-dimensional model at the first perspective.

20. The system of claim 19, wherein in response to transitioning to the second perspective, the virtual camera is configured to rotate around the three-dimensional model relative to the position of the virtual camera at the first perspective.

21. The system of claim 19, wherein an angle of the virtual camera relative to a horizontal floor of the three-dimensional model at the second perspective is set to a default value or a previous angle of the virtual camera relative to the horizontal floor when in orbit mode.

22. The system of claim 1, wherein the first mode comprises orbit mode and the second mode comprises floor plan mode, wherein the virtual camera is configured to rotate around the virtual model relative to a selected point on the three-dimensional model during orbit mode, and wherein the transition component is configured to determine the second set based on coordinates of the selected point.

23. The system of claim 1, wherein the first mode comprises floor plan mode and the second mode comprises walking mode, and wherein the transition component is configured to determine the second set of parameters using ray casting from a coordinate position of the virtual camera at the first perspective to an intersecting location on the three-dimensional model directly below the coordinate position.

24. The system of claim 1, wherein the transition function comprises an S-shaped transition function ƒ

(x) that comprises for a parameter x;

ƒ

(x)=(1−

cos(π

*x))/2, for 0≦

x≦

1.

25. The system of claim 24, wherein the transition component is further configured to employ the S-shaped transition function ƒ

(x) to interpolate parameter x at time t between a first parameter value x₁ at time t₁ and a second parameter value x₂ at time t₂ according to;

x=x₁+(x₂−

x₁)*ƒ

((t−

t₁)/(t₂−

t₁)).

26. The system of claim 1, wherein the height of the plane is based on a plurality of candidate points of the model having vectors within a threshold angle from a vertical plane.

27. The system of claim 26, wherein the height of the plane is an average or median of respective heights of the candidate points within a defined area of the three-dimensional model.

28. The system of claim 1, wherein the interface component is further configured to limit a rate of change of a parameter modified based on the user input to a threshold rate of change, wherein the parameter is from at least one of the first set of parameters or the second set of parameters.

29. The system of claim 1, wherein the interface component determines a height of the plane by a procedure that selects a plurality of candidate points, within a defined area of the three-dimensional model, having respective normal vectors within a threshold angle from a vertical plane of the three-dimensional model, and determines an average or median of respective heights of the candidate points.

30. A method comprising:

using a processor to execute the following computer executable instructions stored in a memory to perform the following acts;

configuring a graphical user interface that facilitates viewing and navigating a three-dimensional model from different perspectives of a virtual camera, wherein the different perspectives are based on different navigation modes including;

an orbit mode that provides perspectives of the three-dimensional model from the virtual camera positioned outside of the three-dimensional model and at various rotations around a vertical axis perpendicular to a floor of the three-dimensional model,

a floor plan mode that provides perspectives of the three-dimensional model from the virtual camera positioned outside and above the three-dimensional model and with a view of the three-dimensional model that has a pitch direction of the view locked substantially orthogonally orientated relative to the floor of the three-dimensional model, and wherein the floor plan mode enables spatial motion of the virtual camera while maintaining the substantially orthogonally orientation of the view relative to the horizontal surface, and

a walking mode that provides perspectives of the three-dimensional model from the virtual camera positioned at an approximately consistent distance relative to a defined surface of the three-dimensional model;

generating representations of the three-dimensional model from the different perspectives based on user input identifying position and orientation of the virtual camera and a selected mode;

in response to determining that a user volume associated with a target position creates a conflict with a portion of the three-dimensional model, iteratively placing a bottom of the user volume at a local floor height at points of a series of successively larger areas surrounding the target position until a point is located that resolves the conflict, and setting the point as the target position;

receiving a request to transition from a first perspective of the three-dimensional model in a first mode of the different navigation modes to a second perspective of the three-dimensional model in a second mode of the different navigation modes;

in response to the request, determining a second set of parameters corresponding to the target position and orientation of the virtual camera relative to the three-dimensional model at the second perspective, based in part on a first set of parameters corresponding to a position or orientation of the virtual camera relative to the three-dimensional model at the first perspective; and

generating a smooth visual transition between a first representation of the three-dimensional model at the first perspective to a second representation of the three-dimensional model at the second perspective by interpolating from the first set of parameters to the second set of parameters using a transition function, wherein the transition function provides an accelerating portion of an interpolation between two values followed by a decelerating portion of the interpolation between the two values.

31. The method of claim 30, wherein the perspectives associated with the walking mode simulate positions of a virtual user located on the defined surfaces of the three-dimensional model.

32. The method of claim 30, further comprising:

displaying the representations of the three-dimensional model via a display screen of a device.

33. The method of claim 30, wherein the generating comprises, during orbit mode, employing backface culling or a ceiling removal technique and generating a representation of the three-dimensional model during the orbit mode at a perspective of the interior space from a point outside of the three-dimensional model.

34. The method of claim 30, wherein generating comprises, during orbit mode, rotating the virtual camera around the vertical axis at various distances relative to the floor of the three-dimensional model.

35. The method of claim 30, wherein the generating comprises, during orbit mode, moving the virtual camera at various distances relative to the vertical axis.

36. The method of claim 30, wherein the generating comprises, during orbit mode, generating representations of the three-dimensional model from positions of the virtual camera as it moves toward or away from a point of the three-dimensional model in response to user input.

37. The method of claim 30, wherein the generating comprises, during walking mode, generating representations of the three-dimensional model from positions of the virtual camera as it moves relative to the defined surfaces of the three-dimensional model, in response to user input, to simulate a user walking through the three-dimensional model.

38. The method of claim 37, wherein the walking mode provides perspectives of the three-dimensional model based on user input identifying a direction in which the user is walking through the three-dimensional model.

39. The method of claim 37, wherein a position of the user relative to the three-dimensional model is based in part on a height of the user relative to the defined surfaces.

40. The method of claim 37, further comprising preventing a virtual collision between the user and three-dimensional boundaries of the three-dimensional model during walking mode.

41. The method of claim 30, wherein the generating comprises, during floor plan mode, generating a representation of the three-dimensional model using backface culling or a ceiling removal technique.

42. The method of claim 30, wherein the first mode comprises orbit mode and the second mode comprises walking mode, wherein the determining further comprises, determining the second set of parameters based on user input selecting a location on the three-dimensional model and a defined surface of the three-dimensional model near the location.

43. The method of claim 42, wherein the determining further comprises, determining the second set of parameters using ray casting from the position of the virtual camera relative to the three-dimensional model at the first perspective to the location on the three-dimensional model near the defined surface.

44. The method of claim 30, wherein the first mode comprises walking mode and the second mode comprises orbit mode, wherein the determining further comprises, determining the second set of parameters based on a default configuration for the target position and orientation of the virtual camera in orbit mode.

45. The method of claim 30, wherein the first mode comprises walking mode and the second mode comprises orbit mode, wherein the determining further comprises determining the second set of parameters based on a preset distance directly above the position of the virtual camera relative to the three-dimensional model at the first perspective.

46. The method of claim 30, wherein the position of the virtual camera relative to the three-dimensional model at the first perspective becomes the central point of the vertical axis for orbit mode at the second perspective.

47. The method of claim 30, wherein the first mode comprises orbit mode and the second mode comprises floor plan mode, wherein during orbit mode, the virtual camera is configured to rotate around the virtual model relative to a selected point on the three-dimensional model, and wherein the determining further comprises determining the second set of parameters based on coordinates of the selected point.

48. The method of claim 30, wherein the first mode comprises floor plan mode and the second mode comprises walking mode, wherein the determining the second set of parameters further comprises using ray casting from a coordinate position of the virtual camera at the first perspective to an intersecting location on the three-dimensional model directly below the coordinate position.

49. The method of claim 30, further comprising determining a height of the defined surface by selecting a plurality of candidate points, within a defined area of the three-dimensional model, having respective normal vectors within a threshold angle from a vertical plane three-dimensional model, and determining an average or median of respective heights of the candidate points.

50. A non-transitory computer-readable storage medium comprising computer-readable instructions that, in response to execution, cause a computing system to perform operations, comprising:

configuring a graphical user interface that facilitates viewing and navigating a three-dimensional model from different perspectives of a virtual camera, wherein the different perspectives are based on different navigation modes including;

a walking mode that provides perspectives of the three-dimensional model from the virtual camera positioned at an approximately consistent distance relative to defined surfaces of the three-dimensional model, and

a floor plan mode that provides perspectives of the three-dimensional model from the virtual camera positioned outside and above the three-dimensional model and with a view of the three-dimensional model that has a pitch direction of the view locked substantially orthogonally orientated relative to a horizontal surface of the three-dimensional model, and wherein the floor plan mode enables spatial motion of the virtual camera while maintaining the substantially orthogonally orientation of the view relative to the horizontal surface;

in response to determining that a user volume associated with a target position creates a conflict with a portion of the three-dimensional model, iteratively placing a bottom of the user volume at a local floor height at points of a series of successively larger areas surrounding the target position until a point is located that resolves the conflict, and setting the point as the target position;

generating representations of the three-dimensional model from the different perspectives based on user input identifying position and orientation of the virtual camera and a selected mode;

receiving a request to transition from a first perspective of the three-dimensional model in a first mode of the different navigation modes to a second perspective of the three-dimensional model in a second mode of the different navigation modes;

in response to the request, determining a second set of parameters corresponding to a target position and orientation of the virtual camera relative to the three-dimensional model at the second perspective, based in part on a first set of parameters corresponding to a position or orientation of the virtual camera relative to the three-dimensional model at the first perspective; and

generating a smooth visual transition between a first representation of the three-dimensional model at the first perspective to a second representation of the three-dimensional model at the second perspective by interpolating from the first set of parameters to the second set of parameters using a transition function, wherein the transition function provides an accelerating portion of an interpolation between two values followed by a decelerating portion of the interpolation between the two values.

51. The non-transitory computer-readable storage medium of claim 50, wherein the perspectives associated with the walking mode simulate positions of a virtual user located on the defined surfaces of the three-dimensional model.

52. The non-transitory computer-readable storage medium of claim 50, further comprising:

displaying the representations of the three-dimensional model via a display screen of a device.

53. The non-transitory computer-readable storage medium of claim 50, wherein the generating comprises, during walking mode, generating representations of the three-dimensional model from positions of the virtual camera as it moves relative to the defined surfaces of the three-dimensional model, in response to user input, to simulate a user walking through the three-dimensional model.

54. The non-transitory computer-readable storage medium of claim 53, wherein the generating comprises, during walking mode, generating the representations of the three-dimensional model based on user input identifying a direction in which the user is walking through the three-dimensional model.

55. The non-transitory computer-readable storage medium of claim 53, wherein a position of the user relative to the three-dimensional model is based in part on a height of the user relative to the defined surfaces.

56. The non-transitory computer-readable storage medium of claim 50, wherein the generating comprises, during floor plan mode, generating representations of the three-dimensional model as orthographic projections.

57. The non-transitory computer-readable storage medium of claim 50, wherein the generating comprises, during floor plan mode, generating representations of the three-dimensional model using backface culling or a ceiling removal technique.

58. The non-transitory computer readable medium of claim 50, wherein the first mode comprises floor plan mode and the second mode comprises walking mode, and wherein the determining the second set of parameters comprises using ray casting from a coordinate position of the virtual camera at the first perspective to an intersecting floor of the three-dimensional model directly below the coordinate position.