Method and system for projecting text onto surfaces in geographic imagery

• US 9,361,283 B2
• Filed: 11/30/2011
• Issued: 06/07/2016
• Est. Priority Date: 11/30/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Jones, Jonah, Simpson, Matthew Robert
• Primary Examiner(s): Xiao, Ke
• Assistant Examiner(s): Shin, Andrew
• Application Number:
  US13/307,381
• Publication Number:
  US 20150178257A1
• Time in Patent Office: 1,651 Days
• Field of Search: 345/629, 345/632, 345/633, 345/634, 345/636, 345/419, 340/988, 340/995.1, 340/995.14, 340/995.19, 340/995.28, 701/400, 701/420, 701/421, 701/431, 701/436, 701/440
• US Class Current: 1/1
• CPC Class Codes: G06F 40/169 Annotation, e.g. comment data or footnotes
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2219/004 Annotating, labelling
• CPC Class Codes: G08G 1/0969 having a display in the form of a map
• CPC Class Codes: G08G 1/205 Indicating the location of the monitored vehicles as destination, e.g. accidents, stolen, rental
• CPC Class Codes: G09G 5/14 Display of multiple viewports
• CPC Class Codes: H04N 21/25841 involving the geographical location of the client retrieval from the Internet by querying based on geographical locations G06F16/9537; arrangements for identifying locations of receiving stations in broadcast systems H04H60/51; location of the user terminal in data switching networks H04L67/52; services making use of the location of users or terminals in wireless networks H04W4/02; locating users or terminals in wireless networks H04W64/00
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4725 using interactive regions of the image, e.g. hot spots details of information retrieval from the Internet by using URLs G06F16/955; processing chained hypermedia data for information retrieval G06F16/94
• CPC Class Codes: H04N 21/6547 comprising parameters, e.g. for client setup
• CPC Class Codes: H04N 21/6587 Control parameters, e.g. trick play commands, viewpoint selection
• CPC Class Codes: H04N 5/44504 Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits

Abstract

A user interface for viewing imagery associated with a geographic area, such as street level imagery is disclosed. The interface includes at least one text annotation projected in the three-dimensional space defined by the imagery onto a surface, such as a generally vertical surface, of an object depicted in the imagery. For example, the text annotation can be rendered such that the annotation appears to be located along the façade of a building depicted in the imagery. The projection of text annotations in the three-dimensional space can provide a more immersive, augmented reality-style view of the geographic area of interest.

Claims

1. A computer-implemented method for displaying imagery, comprising:

receiving at a computing device at least a portion of an image of a geographic area over a network interface, the image defining a three-dimensional space and depicting at least one object captured by a camera;

presenting a viewport in the display of the computing device that displays at least a portion of the image;

obtaining instructions to render a text annotation associated with the object depicted in the image, the instructions specifying text effects for the text annotation, the text effects being determined based at least in part on surface information data providing a geographic position of a surface of the object in the geographic area; and

rendering a text annotation pursuant to the instructions with text effects such that the text annotation appears to be located in the three-dimensional space along at least a portion of the surface of the object depicted in the image, wherein the text annotation comprises at least a portion of travel directions;

wherein the text annotation is interactive.

2. The method of claim 1, wherein the text effects specify varying character sizes for the text annotation.

3. The method of claim 1, wherein the text effects specify an orientation of the text annotation such that the text annotation is rendered on a baseline positioned along at least a portion of the surface of the object depicted in the image.

4. The method of claim 1, wherein the text annotation is rendered in a fixed location relative the image such that as a user navigates the image, the text annotation remains in a fixed location along at least a portion of the surface of the object depicted in the image.

5. The method of claim 1, wherein the text annotation is rendered such that the text annotation appears to be located along the faç

ade of a building depicted in the image.

6. The method of claim 1, wherein the text annotation comprises information associated with the object on which the text annotation is rendered.

7. The method of claim 1, wherein upon user interaction with the text annotation, the method comprises presenting additional information to the user.

8. The method of claim 7, wherein upon user interaction with the text annotation, the method presents imagery associated with the next step in the travel directions.

9. The method of claim 1, wherein the image comprises street level imagery.

10. The method of claim 1, wherein the method comprises:

rendering a virtual object in the three-dimensional space defined by the image; and

rendering a text annotation such that the text annotation appears to be located along a surface of the virtual object.

11. The computer-implemented method of claim 1, wherein the image comprises a plurality of pixels, the surface information data comprising a value stored for each of the plurality of pixels, the value representing the geographic position of the surface of the object illustrated in the pixel.

12. The computer-implemented method of claim 1, wherein the surface information data is determined based at least in part on a three-dimensional model associated with the image.

13. The computer-implemented method of claim 1, wherein the surface information data comprises latitude, longitude, and altitude coordinates associated with the surface of the object.

14. A system for displaying imagery, comprising:

a display device;

one or more processors; and

at least one memory coupled to the one or more processors, the at least one memory comprising computer-readable instructions for execution by the one or more processors to cause said one or more processors to perform operations, comprising;

receiving at least a portion of an image of a geographic area over a network interface, the image defining a three-dimensional space and depicting at least one object captured by a camera;

presenting a viewport on the display device that displays at least a portion of the image;

receiving, via the network interface, instructions to render an interactive text annotation associated with the object depicted in the image, the instructions specifying text effects for the interactive text annotation, the text effects being determined at a remote server based at least in part on surface information data stored at a remote surface information database, the surface information data being associated with the image by the remote server prior to receiving the at least a portion of the image of the geographic area over the network interface; and

rendering an interactive text annotation pursuant to the instructions with text effects such that the text annotation appears to be located in the three-dimensional space along at least a portion of a surface of the object depicted in the image.

15. The system of claim 14, wherein the text effects specify an orientation of the text annotation such that the text annotation is rendered on a baseline positioned along at least a portion of the surface of the object depicted in the image.

16. The system of claim 14, wherein the text annotation is fixed relative the image such that as a user navigates the image, the text annotation remains in a fixed location along at least a portion of the surface of the object depicted in the image.

17. The system of claim 14, wherein the text annotation comprises at least a portion of travel directions.

18. The system of claim 14, wherein the text annotation comprises at least a portion of travel directions, wherein upon user interaction with the text annotation, the system is configured to present imagery associated with the next step in the travel directions.

19. The system of claim 14, wherein the surface information data comprises distance data representing the distance of the surface of the object to a camera.

20. A system for providing geographic images, the system comprising a processing device and a network interface, the processing device configured to:

receive a request for an image of a geographic area, the image defining a three dimensional space and depicting at least one object captured by a camera;

access a three-dimensional model associated with the geographic area depicted in the image to identify the location of a surface of the at least one object depicted in the image;

obtain instructions to render a text annotation with text effects such that the text annotation appears to be located in the three dimensional space along the surface of the at least one object, the text effects being determined based at least in part on the location of the surface identified in three-dimensional model; and

provide, via the network interface, at least a portion of the image and the instructions to another processing device.

21. The system of claim 20, wherein the text annotation is interactive.

22. The system of claim 21, wherein the text annotation comprises at least a portion of travel directions, wherein upon user interaction with the text annotation, the system is configured to present imagery associated with the next step in the travel directions.

23. A computer-implemented method for displaying panoramic imagery, comprising:

receiving at a computing device at least a portion of a panoramic image of a geographic area, the panoramic image defining a three-dimensional space and depicting at least one object captured by a camera;

presenting a viewport in the display of the computing device that displays at least a portion of the panoramic image;

obtaining instructions to render a text annotation associated with the object depicted in the image, the instructions specifying text effects for the text annotation; and

rendering the text annotation pursuant to the instructions with text effects such that the text annotation appears to be located in the three-dimensional space along at least a portion of the surface of the object depicted in the image;

wherein the text annotation comprises at least a portion of travel directions, wherein upon user interaction with the text annotation, the method comprises presenting panoramic imagery associated with the next step in the travel directions in the viewport.

24. The computer-implemented method of claim 23, wherein the user interaction with the text annotation comprises clicking or tapping the text annotation.