Systems and methods for three-dimensional environmental modeling of a particular location such as a commercial or residential property

• US 10,607,328 B2
• Filed: 06/28/2019
• Issued: 03/31/2020
• Est. Priority Date: 12/03/2015
• Status: Active Grant

• Current Assignee: Echosense LLC
• Original Assignee: Quasar Blu, LLC
• Inventors: Thomas, Mark
• Primary Examiner(s): Ahmed, Samir A
• Application Number:
  US16/457,569
• Publication Number:
  US 20190325569A1
• Time in Patent Office: 277 Days
• CPC Class Codes: G06T 15/503 Blending, e.g. for anti-aliasing
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 2210/04 Architectural design, interior design
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/73 using feature-based methods

Abstract

A system, according to various embodiments, receives images such as photographs and videos from one or more portable computing devices associated with one or more individuals (e.g., construction workers or landscapers) while the portable computing devices are in a particular position within a particular location at a particular time. The system determines a virtual position within a 3-D representation of the particular location that generally corresponds to the particular position and combines the images with the 3-D representation to generate an enhanced 3-D representation of the particular location. This may allow, for example, owners of a particular property to track and quickly understand construction and landscaping work that has been done on their property and to easily contact those individuals regarding that work.

Claims

1. A computer-implemented method for generating an enhanced representation of a particular property, the method comprising:

i. receiving, by at least one computer processor, a first image from a first portable computing device associated with a first individual taken while the first portable computing device is in a first position relative to the property at a first time;

ii. receiving, by at least one computer processor, a second image from a second portable computing device associated with a second individual taken while the second portable computing device is in a second position relative to the property at a second time;

iii. determining, by at least one computer processor, a first virtual position within a 3-D representation of the particular property that generally corresponds to the first position;

iv. determining, by at least one computer processor, a second virtual position within the 3-D representation of the particular property that generally corresponds to the second position;

v. after receiving the first and second images, combining, by at least one computer processor, the first image, the second image, and the 3-D representation so that the first position of the first image generally corresponds to the first virtual position within the 3-D representation and the second position of the second image generally corresponds to the second virtual position within the 3-D representation; and

vi. at least partially in response to combining the first image, the second image, and the 3-D representation, generating, by at least one computer processor, an enhanced representation of the particular property that includes the first image, the second image, and the 3-D representation of the particular property.

2. The computer-implemented method of claim 1, wherein the property is selected from a group consisting of a residence and a commercial building.

3. The computer-implemented method of claim 2, wherein the first and second individuals are construction workers.

4. The computer-implemented method of claim 1, wherein the first and second individuals each respectively use a same particular software application on the first and second portable computing devices to capture, respectively, the first and second images.

5. The computer-implemented method of claim 1, wherein the first image is a 2-D video.

6. The computer-implemented method of claim 5, wherein the second image is a 2-D video.

7. The computer-implemented method of claim 6, wherein at least a portion of the first and second 2-D videos were taken at the same time.

8. The computer-implemented method of claim 7, wherein the enhanced representation of the particular location property is an enhanced 3-D representation of the particular property.

9. The computer-implemented method of claim 1, wherein the first image is a 2-D still picture.

10. The computer-implemented method of claim 9, wherein the second image is a 2-D still picture.

11. The computer-implemented method of claim 10, wherein the first and second individuals each respectively use a same particular software application on the first and second portable computing devices to capture, respectively, the first and second images.

12. The computer-implemented method of claim 11, wherein:

the first image is a 2-D image of a first particular construction element associated with the particular property;

the second image is a 2-D image of a second particular construction element associated with the particular property;

the enhanced representation of the particular property is an enhanced 3-D representation of the particular property that includes the first and second images.

13. The computer-implemented method of claim 1, wherein the step of combining the first image, the second image, and the 3-D representation further comprises:

a. combining a portion of the first image with a portion of the second image; and

b. generating the enhanced representation of the particular property that includes the portion of the first image, the portion of the second image, and the 3-D representation of the particular property, wherein;

i. the portion of the first image comprises less than all of the first image, and the portion of the second image comprises less than all of the second image.

14. The computer-implemented method of claim 1, wherein the method comprises:

a. identifying a first particular object within the first image;

b. identifying a second particular object within the second image;

c. identifying the first particular object within the 3-D representation of the particular property;

d. identifying the second particular object within the 3-D representation of the particular property;

e. superimposing the first image over the 3-D representation of the space so that the first object is in substantially the same position in the first image and the 3-D representation; and

f. superimposing the second image over the 3-D representation of the space so that the second object is in substantially the same position in the second image and the 3-D representation.

15. The computer-implemented method of claim 14, wherein the first particular object is an object selected from a group consisting of:

a. a portion of a particular wall;

b. an electrical outlet;

c. a portion of a particular ceiling;

d. an item of vegetation; and

e. a window.

16. A computer-implemented method for generating an enhanced representation of a particular property, the method comprising:

i. receiving, by at least one computer processor, a first image from a first portable computing device associated with a first individual taken while the first portable computing device is in a first position within a particular location at a first time;

ii. receiving, by at least one computer processor, a second image from a second portable computing device taken while the second portable computing device is in a second position within the particular property at a second time;

iii. determining, by at least one computer processor, a first virtual position within a 3-D representation of the particular property that generally corresponds to the first position;

iv. determining, by at least one computer processor, a second virtual position within the 3-D representation of the particular property that generally corresponds to the second position;

v. after receiving the first and second image, combining, by at least one computer processor, the first image, the second image, and the 3-D representation so that the first position of the first image generally corresponds to the first virtual position within the 3-D representation and the second position of the second image generally corresponds to the second virtual position within the 3-D representation; and

vi. at least partially in response to combining the first image, the second image, and the 3-D representation, generating, by at least one computer processor, an enhanced representation of the particular property that includes the first image, the second image, and the 3-D representation of the particular property.

17. The computer-implemented method of claim 16, wherein the property is a man-made structure.

18. The computer-implemented method of claim 16, wherein:

the first image is a 2-D image of a first particular construction element associated with the particular property;

the second image is a 2-D image of a second particular construction element associated with the particular property; and

the enhanced representation of the particular property is an enhanced 3-D representation of the particular property that includes the first and second images.