Dynamic and selective model clipping for enhanced augmented hypermodel visualization

• US 9,761,045 B1
• Filed: 07/18/2013
• Issued: 09/12/2017
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Bentley Systems Incorporated
• Original Assignee: Bentley Systems Incorporated
• Inventors: Ct, Stphane, Snyder, Robert, Gervais, Renaud
• Primary Examiner(s): Wang, Yi
• Application Number:
  US13/945,552
• Time in Patent Office: 1,517 Days
• Field of Search: None
• CPC Class Codes: G06T 15/30 Clipping
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00

Abstract

In one embodiment, a three-dimensional (3-D) model is aligned with a view of a physical environment captured by a camera. An augmented reality view is generated by superposing elements of the 3-D model and the view of the physical environment. At least some of the elements of the 3-D model are initially hidden within the augmented reality view. A virtual window is imposed that penetrates a surface within the augmented reality view. The virtual window clips the view of the physical environment and elements of the 3-D model that intersect the virtual window to reveal initially-hidden elements of the 3-D model located beyond the virtual window. The augmented reality view with the virtual window is displayed on a display device.

Claims

1. A method comprising:

aligning a three-dimensional (3-D) model with a view of a physical environment captured by a camera;

generating an augmented reality view by superposing elements of the 3-D model and the view of the physical environment, at least some of the elements of the 3-D model being initially hidden behind the view of the physical environment within the augmented reality view;

configuring an exclusion list for elements of the 3-D model;

imposing a virtual window that penetrates a wall of a structure within the augmented reality view, the virtual window clipping the view of the physical environment and elements of the 3-D model that are not on the exclusion list that intersect the virtual window, while preserving any elements of the 3-D model that are on the exclusion list that intersect the virtual window, to produce a display that shows initially-hidden excluded elements of the 3-D model located within the virtual window in the context of initially-hidden elements of the 3-D model located beyond the virtual window, in the larger context of the view of the physical environment; and

displaying the augmented reality view with the virtual window on a display device.

2. The method of claim 1, wherein the imposing further comprises:

determining boundary surfaces of the virtual window;

removing a portion of the view of the physical environment that falls within the boundary surfaces; and

removing elements or portions of elements of the 3-D model that are not on the exclusion list that fall within the boundary surfaces.

3. The method of claim 1, wherein the imposing further comprises:

determining a position or size of the virtual window has been changed; and

updating the augmented reality view to show the virtual window with the changed position or size.

4. The method of claim 1, wherein the exclusion list is configured in response to user input.

5. The method of claim 1, wherein the imposing further comprises:

determining the exclusion list has been changed; and

updating the augmented reality view to show the virtual window with the changed exclusion list.

6. The method of claim 1, wherein the aligning aligns the 3-D model with a first view of the physical environment and a second view of the physical environment, the generating generates a first augmented reality view and a second augmented reality view by superposing elements of the 3-D model and the first view of the physical environment and the second view of the physical environment, respectively, wherein the first augmented reality view is associated with a first augmentation point in a model space of the 3-D model, and the second augmented reality view is associated with a second augmentation point in the model space of the 3-D model.

7. The method of claim 6, further comprising:

transitioning between the first augmented reality view and the second augmented reality view by entering a virtual mode in which a virtual view that includes only the 3-D model is displayed on the display device.

8. The method of claim 7, further comprising:

moving a virtual camera within a model space of the 3-D model to show the 3-D model from one or more different perspectives while in the virtual view.

9. The method of claim 7, further comprising:

maintaining the virtual window in the virtual view, the virtual window in the virtual view clipping elements of the 3-D model that intersect the virtual window.

10. The method of claim 1, wherein the structure is a building, and the virtual window reveals elements of the 3-D model that correspond to internal portions of the building.

11. The method of claim 1, wherein the virtual window extends to an inner boundary surface that defines a depth it penetrates into the surface, and is defined by side boundary surfaces that define a width it extends in the surface, and top and bottom boundary surfaces that define a height it extends in the surface.

12. An electronic device comprising:

a display device;

a processor; and

a memory coupled to the processor and configured to store instructions for an augmented reality application that is executable on the processor, the instructions for the augmented reality application, when executed, operable to;

generate an augmented reality view by superposing elements of a three dimensional (3-D) model and a view of the physical environment captured by a camera, at least some of the elements of the 3-D model being initially hidden behind the view of the physical environment within the augmented reality view,

impose a virtual window within the augmented reality view that penetrates a wall of a structure within the augmented reality view, the virtual window clipping the view of the physical environment and elements of the 3-D model that are not on an exclusion list that intersect the virtual window, while preserving any elements of the 3-D model that are on the exclusion list that intersect the virtual window, to produce a display that shows initially-hidden excluded elements of the 3-D model located within the virtual window in the context of initially-hidden elements of the 3-D model located beyond the virtual window, in the larger context of the view of the physical environment; and

display the augmented reality view with the virtual window on the display device.

13. The electronic device of claim 12, wherein the instructions that, when executed, are operable to impose are further operable to:

determine boundary surfaces of the virtual window;

remove a portion of the view of the physical environment that falls within the boundary surfaces; and

remove elements or portions of elements of the 3-D model that are not on the exclusion list that fall within the boundary surfaces.

14. The electronic device of claim 12, wherein the instructions that, when executed, are operable to impose are further operable to:

determine a position or size of the virtual window has been changed; and

update the augmented reality view to show the virtual window with the changed position or size.

15. The electronic device of claim 12, wherein the instructions that, when executed, are operable to impose are further operable to:

determine the exclusion list has been changed; and

update the augmented reality view to show the virtual window with the changed exclusion list.

16. A non-transitory computer-readable medium containing executable instructions, the executable instructions, when executed by one or more processors, operable to:

align a three-dimensional (3-D) model with a first view of a physical environment captured by a camera and a second view of the physical environment captured by the camera;

generate a first augmented reality view and a second augmented reality view by superposing elements of the 3-D model and the first view of the physical environment and the second view of the physical environment;

associate the first augmented reality view with a first augmentation point in a model space of the 3-D model, and the second augmented reality view with a second augmentation point in the model space of the 3-D model;

display the first augmented reality view; and

transition between the first augmented reality view and the second augmented reality view by

entering a virtual mode in which a virtual view that includes only the 3-D model is displayed,

showing the virtual view that includes only the 3-D model while a virtual camera is moved within the model space of the 3-D model between the first augmentation point and the second augmentation point, and

upon reaching the second augmentation point, exiting the virtual mode to display the second augmented reality view.

17. The non-transitory computer-readable medium of claim 16 wherein the executable instructions, when executed by one or more processors, are further operable to:

impose a virtual window that penetrates a wall of a structure within the virtual view, the virtual window in the virtual view clipping elements of the 3-D model that intersect the virtual window.

18. The non-transitory computer-readable medium of claim 16, wherein the entering, showing and exiting of the transition are performed automatically absent manual movement of the virtual camera by a user.

19. A method comprising:

aligning a three-dimensional (3-D) model with a first view of a physical environment captured by a camera and a second view of the physical environment captured by the camera;

generating a first augmented reality view by combining elements of the 3-D model as viewed from a first position in model space of the 3-D model with the first view, and a second augmented reality view by combining elements of the 3-D model as viewed from a second position in model space of the 3-D model with the second view;

displaying the first augmented reality view on a display device; and

transitioning between the first augmented reality view and the second augmented reality view by

entering a virtual mode in which a virtual view that includes only the 3-D model is displayed,

showing the virtual view that includes only the 3-D model while a virtual camera is moved within the model space of the 3-D model between the first position and the second position, and

upon reaching the second position, exiting the virtual mode to display the second augmented reality view.

20. The method of claim 19, further comprising:

impose a virtual window that penetrates a wall of a structure within the virtual view, the virtual window in the virtual view clipping elements of the 3-D model that intersect the virtual window.