Surfacing notable changes occurring at locations over time

• US 9,437,004 B2
• Filed: 06/23/2014
• Issued: 09/06/2016
• Est. Priority Date: 06/23/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Siegel, Jonathan, Hutchison, Allen, Li, Wenfeng
• Primary Examiner(s): ANSARI, TAHMINA N
• Application Number:
  US14/311,989
• Publication Number:
  US 20150371389A1
• Time in Patent Office: 806 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10032 Satellite or aerial image; Remote sensing
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/30184 Infrastructure
• CPC Class Codes: G06T 7/254 involving subtraction of images

Abstract

An exemplary method for surfacing feature changes occurring over time at a location is provided. The method includes identifying sets of images captured at a location over a period of time. The sets of images depict objects at the location. Visual changes at the location over the period of time are determined for a given physical feature of the objects depicted in images from the sets of images based on at least a comparison of features in corresponding areas of a first and a second 3D geometry of the location associated with the images. The first and second 3D geometries depict physical features of objects at the location at different times. The sets of images may be ranked based on the determined visual changes for each set. A given set of images from the sets of images may be selected for display on a user interface based on the ranking.

Claims

1. A computer implemented method, comprising:

identifying, by one or more computing devices, sets of images captured at a location over a period of time, the sets of images depicting objects at the location;

generating, by the one or more computing devices, a first and a second 3D geometry of the location based on the sets of images, the first and second 3D geometries depicting physical features of objects at the location at different times, wherein each 3D geometry represents a 3D reconstruction of the objects in the sets of images;

determining, by the one or more computing devices, visual changes at the location over the period of time for a given physical feature of the objects depicted in images from the sets of images, based on at least a comparison of features in corresponding areas of the first and second 3D geometries of the location;

ranking, by the one or more computing devices, the sets of images based on the determined visual changes for each set of the sets of images; and

selecting, by the one or more computing devices, a given set of images from the sets of images based on the ranking, the given set of images depicting visual changes occurring over time at the location.

2. The computer implemented method of claim 1, further comprising:

receiving a request for imagery related to the location; and

in response to the request for imagery related to the location, providing one or more images from the selected set of images based on the ranking.

3. The method of claim 1, wherein the first 3D geometry depicts objects at the location at a first time and the second 3D geometry depicts objects at the location at a second time, the second time occurring after the first time.

4. The method of claim 1, wherein generating the first and second 3D geometries of the location comprises reconstructing a geometric view of the location using image data associated with images from the sets of images.

5. The method of claim 1, wherein generating the first and second 3D geometries of the location comprises extracting, from the sets of images, feature data regarding the physical features of objects depicted in the images.

6. The method of claim 1, wherein ranking the sets of images further comprises:

determining change scores for one or more regions in the location, the change scores determined based on a visual difference in the given physical feature depicted in corresponding areas of the first and second 3D geometries; and

wherein the selecting the given set of images is based at least in part on the change scores.

7. The method of claim 2, wherein the ranking further comprises ordering the sets of images based on an amount of visible changes determined for the given physical feature depicted in each set.

8. The method of claim 2, wherein providing the one or more of images from the selected set of images based on the ranking comprises:

determining whether the visual changes of the given physical feature in images from the selected set of images satisfies a predetermined change threshold level; and

the one or more images are provided when the visual changes satisfy the predetermined change threshold level.

9. The method of claim 2, wherein providing the one or more of images from the selected set of images comprises:

receiving a request for images of the location based on an identified physical feature; and

wherein the one or more images depict changes of the identified physical feature that occurred over time at the location.

10. The method of claim 2, wherein providing the one or more of images comprises:

identifying a landmark within a viewing angle of the selected images associated with the location; and

wherein the one or more images depict changes to the landmark over time.

11. A non-transitory computer readable medium, storing instructions that, when executed by one or more processors, cause the one or more processors to perform a method, the method comprising:

identifying, by the one or more processors, sets of images captured at a location over a period of time, the sets of images depicting objects at the location;

generating, by the one or more processors, a first and second a 3D geometry of the location based on the sets of images, the first and second 3D geometries depicting physical features of objects at the location at different times;

comparing features in corresponding areas of the first and second 3D geometries of the location;

determining change scores for one or more regions in the location over the period of time, the change scores determined based on a visual difference in physical features depicted in corresponding areas of the first and second 3D geometries;

ranking, by the one or more processors, the sets of images based on the change scores;

determining whether the change scores for the set of images associated with a particular region satisfy a predetermined change threshold; and

providing for display one or more of the set of images associated with the particular region if the change scores satisfy the predetermined change threshold, the one or more of the set of images depicting visual changes occurring over time at the location.

12. A system, comprising:

a memory; and

one or more processors coupled to the memory, the one or more processors being configured to;

identify sets of images captured at a location over a period of time, the sets of images depicting objects at the location;

generate a first and a second 3D geometry of the location based on the sets of images, the first and second 3D geometries depicting physical features of objects at the location at different times, wherein each 3D geometry represents a visual reconstruction of the objects in the sets of images;

determine visual changes at the location over the period of time for a given physical feature of the objects depicted in images from the sets of images, based on at least a comparison of features in corresponding areas of the first and second 3D geometries of the location;

rank the sets of images based on the determined visual changes for each set of the sets of images; and

select a given set of images from the sets of images based on the ranking, the given set of images depicting visual changes occurring over time at the location.

13. The system of claim 12, wherein the processors are further configured to:

receive a request for imagery related to the location; and

in response to the request for imagery related to the location, provide one or more images from the selected set of images.

14. The system of claim 12, wherein the first 3D geometry depicts objects at the location at a first time and the second 3D geometry depicts objects at the location at a second time, the second time occurring after the first time.

15. The system of claim 12, wherein to generate the first and second 3D geometries of the location comprises reconstructing a geometric view of the location using image data associated with images from the sets of images.

16. The system of claim 12, wherein ranking the sets of images comprises:

determining change scores for one or more regions in the location, the change scores determined based on a visual difference in the given physical feature depicted in corresponding areas of the first and second 3D geometries associated with the sets of images; and

wherein the selecting the given set of images is based at least in part on the change scores.

17. The system of claim 12, wherein the ranking further comprises ordering the sets of images based on an amount of visible changes determined for the given physical feature depicted in each set.

18. The system of claim 13, wherein to provide the one or more of images from the selected set of images based on the ranking comprises:

determine whether the visual changes of the given physical feature in images from the selected set of images satisfies a predetermined change threshold level; and

the one or more images are provided when the visual changes satisfy the predetermined change threshold level.

19. The system of claim 13, wherein to provide the one or more of images from the selected set of images comprises:

receiving a request for images of the location based on an identified physical feature; and

wherein the one or more images depict changes of the identified physical feature that occurred over time at the location.

20. The system of claim 13, wherein to provide the one or more of images from the selected set of images comprises:

identifying a landmark within a viewing angle of the selected images associated with the location; and

wherein the one or more images depict changes to the landmark over time.