THREE-DIMENSIONAL ANNOTATIONS FOR STREET VIEW DATA

• US 20170069121A1
• Filed: 10/11/2016
• Published: 03/09/2017
• Est. Priority Date: 01/15/2008
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Zhu, Jiajun, Filip, Daniel Joseph, Vincent, Luc
• Granted Patent: US 10,540,794 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/20101 Interactive definition of point of interest, landmark or seed
• CPC Class Codes: G06T 2210/12 Bounding box
• CPC Class Codes: G06T 2219/004 Annotating, labelling
• CPC Class Codes: G06T 7/75 involving models

Abstract

The present invention relates to annotating images. In an embodiment, the present invention enables users to create annotations corresponding to three-dimensional objects while viewing two-dimensional images. In one embodiment, this is achieved by projecting a selecting object onto a three-dimensional model created from a plurality of two-dimensional images. The selecting object is input by a user while viewing a first image corresponding to a portion of the three-dimensional model. A location corresponding to the projection on the three-dimensional model is determined, and content entered by the user while viewing the first image is associated with the location. The content is stored together with the location information to form an annotation. The annotation can be retrieved and displayed together with other images corresponding to the location.

Claims

1. A method, comprising:

receiving, by one or more computing devices, content to be associated with a selected portion of a first street-level, panoramic, photographic image;

extending, by the one or more computing devices, at least one ray from a determined focal point of the first image through a point of the selected portion to project the selected portion onto a projected portion of a three-dimensional model, at least some of the three-dimensional model representing content of the first image;

determining, by the one or more computing devices, a location in three-dimensional space of the projected portion in the three-dimensional model;

associating, by the one or more computing devices, the content with the location of the projected portion;

storing, by the one or more computing devices, the content with the location to form an annotation; and

generating, by the one or more computing devices, the annotation for display in a second street-level, panoramic, photographic image at a position corresponding to the location.

2. The method of claim 1, further comprising generating, by the one or more computing devices, the annotation for display on a map.

3. The method of claim 1, wherein generating the annotation for display in the second image comprises outlining or highlighting a portion the second image corresponding to the location.

4. The method of claim 1, wherein the three-dimensional model includes one or more street planes and one or more faç

ade planes.

5. The method of claim 1, wherein the content is received from a client device while displaying the first image.

6. The method of claim 1, wherein selection of the selected portion is received from a client device while displaying the first image.

7. The method of claim 6, wherein the selected portion is defined by a bounding box defined according to a received input signal.

8. A system, comprising one or more computing devices configured to:

receive content to be associated with a selected portion of a first street-level, panoramic, photographic image;

extend at least one ray from a determined focal point of the first image through a point of the selected portion to project the selected portion onto a projected portion of a three-dimensional model, at least some of the three-dimensional model representing content of the first image;

determine a location in three-dimensional space of the projected portion in the three-dimensional model;

associate the content with the location of the projected portion;

store the content with the location to form an annotation; and

generate the annotation for display in a second street-level, panoramic, photographic image at a position corresponding to the location.

9. The system of claim 8, wherein the one or more computing devices are further configured to generate the annotation for display on a map.

10. The system of claim 8, wherein generating the annotation for display in the second image comprises outlining or highlighting a portion the second image corresponding to the location.

11. The system of claim 8, wherein the three-dimensional model includes one or more street planes and one or more faç

ade planes.

12. The system of claim 8, wherein the content is received from a client device while displaying the first image.

13. The system of claim 8, wherein the selection of the selected portion is received from a client device while displaying the first image.

14. The system of claim 13, wherein the selected portion is defined by a bounding box defined according to a received input signal.

15. A non-transitory computer-readable storage medium on which computer readable instructions of a program are stored, the instructions, when executed by one or more computing devices, cause the one or more computing devices to perform a method comprising:

receiving content to be associated with a selected portion of a first street-level, panoramic, photographic image;

extending at least one ray from a determined focal point of the first image through a point of the selected portion to project the selected portion onto a projected portion of a three-dimensional model, at least some of the three-dimensional model representing content of the first image;

determining a location in three-dimensional space of the projected portion in the three-dimensional model;

associating the content with the location of the projected portion;

storing the content with the location to form an annotation; and

generating the annotation for display in a second street-level, panoramic, photographic image at a position corresponding to the location.

16. The non-transitory computer-readable storage medium of claim 15, wherein the method further comprises generating the annotation for display on a map.

17. The non-transitory computer-readable storage medium of claim 15, wherein generating the annotation for display in the second image comprises outlining or highlighting a portion the second image corresponding to the location.

18. The non-transitory computer-readable storage medium of claim 15, wherein the content is entered by a user while viewing the first image.

19. The non-transitory computer-readable storage medium of claim 15, wherein the selected portion is selected by a user while viewing the first image.

20. The non-transitory computer-readable storage medium of claim 19, wherein the selected portion is defined by a bounding box input by the user.