Method and system for displaying and navigating an optimal multi-dimensional building model

• US 10,127,721 B2
• Filed: 09/29/2017
• Issued: 11/13/2018
• Est. Priority Date: 07/25/2013
• Status: Active Grant

• Current Assignee: Hover Inc. (Tucows Incorporated)
• Original Assignee: Hover Inc. (Tucows Incorporated)
• Inventors: Upendran, Manish, Altman, Adam J.
• Primary Examiner(s): Broome, Said
• Application Number:
  US15/721,062
• Publication Number:
  US 20180025542A1
• Time in Patent Office: 410 Days
• Field of Search: None
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06T 2210/04 Architectural design, interior design
• CPC Class Codes: G06T 2210/56 Particle system, point based geometry or rendering
• CPC Class Codes: G06V 20/176 Urban or other man-made structures
• CPC Class Codes: G06V 20/653 by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces

Abstract

A method and system is provided for automatic generation and navigation of optimal views of facades of multi-dimensional building models based on where and how the original images were captured. The system and method allows for navigation and visualization of facades of individual or multiple building models in a multi-dimensional building model visualization system.

Claims

1. A method of visualizing a multi-dimensional building model on a computer display, the method comprises:

retrieving a multi-dimensional building model from computer memory;

defining a look angle for the multi-dimensional building model, the defining a look angle linking a real camera to virtual camera positioning based on contextual information obtained during an image capture sequence of the multi-dimensional building model, wherein the contextual information includes at least an image capture sequence number describing and associating each image with a virtual location;

defining a field of view for the multi-dimensional building model by defining up, down, left and right angles which define an extent of the multi-dimensional building model to be viewed on the computer display;

calculating a first main axis and a second main axis for the retrieved multi-dimensional building model to define an orientation of the multi-dimensional building model;

calculating an optimal camera position for the retrieved multi-dimensional building model based on the defined look angle, the defined field of view and the orientation, the optimal camera position defining an optimal view of the retrieved multi-dimensional building model; and

displaying the optimal view on the computer display.

2. The method of claim 1, wherein the contextual information about the image capture sequence further includes, but is not limited to:

GPS Latitude Ref;

GPS Longitude Ref;

GPS Altitude;

GPS Dilution of Precision;

GPS Img (image) Direction Ref (reference);

GPS Img (image) Direction;

metadata for gravity;

or

metadata for camera orientation.

3. The method of claim 1, wherein the image capture sequence includes image capture starting at a front side, with a counter-clockwise or clockwise capture to sequentially capture images of one or more of:

front right corner, right side, back left corner, back side, back right corner, side, front left corner in an image capture sequence with a name associated with images in the image capture sequence.

4. The method of claim 1, wherein, if any image of an image capture sequence is recaptured, the image capture sequence number is preserved.

5. The method of claim 1, wherein the contextual information is included with each image via an EXIF (exchangeable image file) format tag or separately via a metadata file.

6. The method of claim 1 further comprises plotting default look angles for multi-dimensional building models based on an initial position of the real camera.

7. The method of claim 1 further comprises determining a first look angle based on a first image in the image capture sequence.

8. The method of claim 1 further comprises determining the look angle based on the contextual information including any of:

an arbitrary height relative to a roof of a building;

a corner of a building;

architectural features plotted in sequence;

a specific altitude above the multi-dimensional building model;

labeled locations;

or

arbitrary factors used to locate the image capture as it depends on local building constraints wherein the optimal view includes an unobstructed view.

9. The method of claim 1, wherein the retrieved multi-dimensional building model includes pre-stored, previously calculated ones of centered, at least partially unobstructed, optimal views for each faç

ade.

10. A system for visualizing a multi-dimensional building model, the system comprising:

a view processor for processing an optimal view of a multi-dimensional building model, the processor configured to;

retrieve the multi-dimensional building model from computer memory;

define a look angle for the multi-dimensional building model, the defining a look angle including linking a real camera to virtual camera positioning based on contextual information obtained during an image capture sequence of the multi-dimensional building model, wherein the contextual information includes at least an image capture sequence number describing and associating each image with a virtual location;

define a field of view for the multi-dimensional building model by defining up, down, left and right angles which define an extent of the multi-dimensional building model to be viewed on a computer display;

calculate a first main axis and a second main axis for the retrieved multi-dimensional building model to define an orientation of the multi-dimensional building model;

calculate an optimal camera position for the retrieved multi-dimensional building model based on the defined look angle, the defined field of view and the orientation, the optimal camera position defining an optimal view of the retrieved multi-dimensional building model; and

communicate the optimal view to a remote electronic device to be displayed on the computer display of the remote electronic device.

11. The system of claim 10, wherein the contextual information about the image capture sequence further includes, but is not limited to:

GPS Latitude Ref;

GPS Longitude Ref;

GPS Altitude;

GPS Dilution of Precision;

GPS Img (image) Direction Ref (reference);

GPS Img (image) Direction;

metadata for gravity;

or

metadata for camera orientation.

12. The system of claim 10, wherein the image capture sequence includes image capture starting at a front side, with a counter-clockwise or clockwise capture to sequentially capture images of one or more of:

front right corner, right side, back left corner, back side, back right corner, side, front left corner in an image capture sequence with a name associated with images in the image capture sequence.

13. The system of claim 10, wherein, if any image of the image capture sequence is recaptured, the image capture sequence number is preserved.

14. The system of claim 10, wherein the processor is further configured to plot default look angles for a multi-dimensional building model based on an initial real camera position.

15. The system of claim 10, wherein the processor is further configured to determine a first look angle based on a first image in the image capture sequence.

16. The system of claim 10, wherein the processor is further configured to determine the look angle based on metadata associated with the contextual information including any of:

an arbitrary height relative to a roof of a building;

a corner of a building;

architectural features plotted in sequence;

a specific altitude above the multi-dimensional building model;

labeled locations;

or

arbitrary factors used to locate the image capture as it depends on local building constraints wherein the optimal view includes an unobstructed view.

17. A method of visualizing a multi-dimensional building model on a mobile electronic device display, the method comprises:

selecting a multi-dimensional building model based on a location of the multi-dimensional building model;

wirelessly retrieving, from remote computer memory, a previously stored optimal view of the selected multi-dimensional building model, the optimal view based on at least linking a real camera to virtual camera positioning based on contextual information obtained during an image capture sequence of the multi-dimensional building model, wherein the image capture sequence includes a sequential counter-clockwise or clockwise capture of building images of one or more of a;

front side, front right corner, right side, back left corner, back side, back right corner, side, front left corner, with an image name associated with each image in the image capture sequence, and wherein for the selected multi-dimensional building model, the optimal view is based on a look angle, a field of view and an orientation of the selected multi-dimensional building model, wherein the optimal view includes a centered at least partially unobstructed view of the selected multi-dimensional building model; and

displaying the optimal view on the mobile electronic device display.

18. The method of claim 17, wherein the contextual information about the image capture includes, but is not limited to:

GPS Latitude Ref;

GPS Longitude Ref;

GPS Altitude;

GPS Dilution of Precision;

GPS Img (image) Direction Ref (reference);

GPS Img (image) Direction;

metadata for gravity;

metadata for camera orientation;

or

or an image capture sequence number.