System and method of indicating the distance or the surface of an image of a geographical object

• US 9,477,368 B1
• Filed: 03/31/2009
• Issued: 10/25/2016
• Est. Priority Date: 03/31/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Filip, Daniel, Szybalski, Andrew
• Primary Examiner(s): Hailu, Tadeese
• Assistant Examiner(s): Hope, Darrin
• Application Number:
  US12/415,102
• Time in Patent Office: 2,765 Days
• Field of Search: 715/861, 715/854, 715/859, 715/856, 715/764, 715/700
• US Class Current: 1/1
• CPC Class Codes: G01C 21/367 Details, e.g. road map scale, orientation, zooming, illumination, level of detail, scrolling of road map or positioning of current position marker
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 3/048 Interaction techniques based on graphical user interfaces [GUI]
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects

Abstract

A system and method is provided that displays cursors for street level images, where the cursor changes appearance based on the objects in the image, such as the geographic distance between the objects and the camera position and the surface of the objects. For example, the cursor may appear to lie flat against the objects in the image change size based on the distance between the camera and object'"'"'s surface.

Claims

1. A method of displaying images comprising:

receiving, by one or more computing devices, a street-level image representing a geographical object captured by a camera at a camera position;

receiving, by the one or more computing devices, surface data representing a surface of the geographical object and the distance of the surface from the camera position, the surface data including data that represents the surface of the geographical object based on a three-dimensional coordinate system;

displaying, by the one or more computing devices, the street-level image on an electronic display;

receiving, by the one or more computing devices, a screen position, where the screen position identifies a position on the street-level image relative to the display at which to display a cursor;

selecting, by the one or more computing devices, surface data depending on whether the screen position corresponds with the location of the surface of the geographic object depicted within the street-level image;

determining, by the one or more computing devices, a visual characteristic of the cursor based on the selected surface data, an orientation of the surface relative to the camera, and the distance of the surface to the camera; and

displaying, by the one or more computing devices, the cursor with the visual characteristic on the electronic display to provide the appearance of the cursor conforming to the surface of the geographical object within the street-level image such that the shape of the cursor appears to lie flat against the surface of the geographical object, wherein the visual characteristic of the cursor is configured to change based on changes in the received screen position and corresponding changes in the selected surface data, an orientation of the surface relative to the camera, and the distance of the surface to the camera, and wherein a shape of the cursor is dependent on the selected surface data of the geographical object.

2. The method of claim 1, wherein the angle of the camera is substantially parallel to the ground.

3. The method of claim 1 wherein the shape of the cursor is generally circular if the surface data indicates the surface is generally facing the camera and generally elliptical if the surface data indicates the surface is angled away from the camera.

4. The method of claim 1 wherein the visual characteristic of the cursor depends on the type of the surface.

5. The method of claim 4 wherein a first type of surface is a vertically-oriented surface and a second type of surface is a horizontally-oriented surface.

6. The method of claim 1 wherein the visual characteristic of the cursor is its color.

7. The method of claim 1 wherein the screen position depends on the position of a mouse cursor.

8. The method of claim 1 wherein:

the street-level image represents a plurality of geographical objects captured by the camera;

the surface data represents a plurality of surfaces; and

the surface data is selected depending on whether the screen position corresponds with the location of one of the plurality of surfaces within the street-level image.

9. A method of displaying images comprising:

receiving, by one or more computing devices, an image representing a geographical object captured by a camera at a camera position at or above street level, the image including data that represents the geographical object based on a three-dimensional coordinate system;

displaying, by the one or more computing devices, the image on an electronic display;

receiving, by the one or more computing devices, a plurality of different screen positions representing a command to move a cursor in the image between a first position overlapping the geographical object and a second position not overlapping the geographical object; and

displaying, by the one or more computing devices, the cursor with a first visual characteristic on the electronic display when overlapping the geographical object and displaying, by the one or more computing devices, the cursor with a second visual characteristic when not overlapping the geographical object, wherein;

the first visual characteristic represents a distance between the geographical object and the camera, wherein the first visual characteristic is configured to change based on changes in the distance between the geographical object and the camera at different screen positions; and

the first visual characteristic of the cursor depends on the type of the geographical object, wherein the first visual characteristic is configured to change based on changes in the type of geographical object at different screen positions, and wherein a shape of the first visual characteristic of the cursor is dependent on the geographical object such that the shape of the cursor appears to lie flat against a surface of the geographical object.

10. The method of claim 9, wherein the angle of the camera is substantially parallel to the ground.

11. The method of claim 9, wherein the first visual characteristic of the cursor is its size.

12. The method of claim 9, wherein the first visual characteristic of the cursor is its color.

13. The method of claim 9, wherein the screen position depends on the position of a mouse cursor.

14. The method of claim 9, wherein the distance data represents, for each of a plurality of pixels of the image, the distance from the camera position to the surface of the object represented in the image at the pixel.

15. The method of claim 9, wherein the distance data represents a plurality of polygons having vertices at positions, the positions of the vertices being associated with the geographical position of a surface of the geographical object.

16. The method of claim 15 wherein the vertices are stored as values representing the latitude, longitude and altitude of the surface.

17. The method of claim 15 wherein the vertices are stored as values representing distances from the camera position.

18. The method of claim 9, wherein:

the distance data represents the distances of a plurality of surfaces captured in the image;

the distance data is selected depending on whether the screen position corresponds with the location of one of the plurality of surfaces within the image;

the first visual characteristic is the size of the cursor; and

the size of the cursor at a first one of the plurality of surfaces relative to a second one of the plurality of surfaces is proportional to the difference in distance between the first surface and the camera position and the distance between the second surface and the camera position.

19. A method of displaying images comprising:

receiving, by one or more computing devices, a street-level image representing a geographical object captured by a camera at a camera position;

receiving, by the one or more computing devices, distance data representing the distance between a surface of the geographical object and the camera position, the distance data including a two-dimensional array of distance values for each pixel of the received street-level image that represents the geographical object;

displaying, by the one or more computing devices, the image on an electronic display;

receiving, by the one or more computing devices, orientation data representing the orientation of the surface relative to the camera angle;

receiving, by the one or more computing devices, a screen position, where the screen position identifies a position on the street-level image relative to the display;

selecting, by the one or more computing devices, distance data and orientation data depending on whether the screen position corresponds with the location of the surface within the street-level image, wherein selecting the distance data includes selecting at least one distance value from the two-dimensional array based on the screen position;

determining, by the one or more computing devices, a visual characteristic of a cursor based on the selected distance data and orientation data; and

displaying, by the one or more computing devices, the cursor with the visual characteristic on the electronic display and overlapping the street-level image, wherein the visual characteristic of the cursor is configured to change based on changes in the received screen position and corresponding changes in the distance data and orientation data, and wherein a shape of the cursor is dependent on the selected data associated with the geographical object such that the shape of the cursor appears to lie flat against the surface of the geographical object.

20. The method of claim 19, wherein the angle of the camera is substantially parallel to the ground.

21. The method of claim 19 wherein the visual characteristic comprises the size and shape of the cursor, wherein the shape is selected so as to convey the appearance of resting on the surface, and wherein the size is determined to be indicative of the distance between the camera position and the surface.

22. The method of claim 19 wherein the visual characteristic of the cursor depends on the type of the object at the screen position.

23. The method of claim 19 wherein the screen position is determined based on the position of a mouse cursor.

24. A system comprising:

a user input device;

a memory storing instructions, image data representing buildings captured by a camera, and data representing the position of those surfaces of the buildings that are facing the camera, wherein the data representing the position of those surfaces represents the position of those surfaces based on a three-dimensional coordinate system;

a processor in communication with the user input device so as to process information received from the user input device in accordance with the instructions; and

a display in communication with, and displaying information received from, the processor;

the instructions comprising;

instructions for determining a position on the image based on information received from the user input device,

instructions for determining the surface of the building at said position of the image,

instructions for determining a shape and size of a cursor based on the data representing the position of said surface relative to distance to the viewpoint of the image, and

instructions for displaying, on the display, the cursor having the determined shape at the determined size and overlapping the surface of the building, wherein the shape and size of the cursor are configured to change based on changes in the determined data representing the position of said surface relative to distance to the viewpoint of the image, and wherein a shape of the cursor is dependent on the surface of the building such that the shape of the cursor appears to lie flat against the surface of the geographical object.

25. The system of claim 24 wherein the user input device is a mouse.

26. The system of claim 24 wherein the size depends on the distance between said surface and the position of the camera when the image was captured.

27. The system of claim 26 wherein the image is comprised of pixels and wherein the data representing the position of the surfaces comprises a value associated with a plurality of the pixels.

28. The system of claim 26 wherein each value represents the distance from the camera position to the surface of the object portrayed at the pixel.

29. The system of claim 24 wherein the shape of the cursor depends on the orientation of the surface relative to the camera angle.

30. The system of claim 24 wherein the determined shape is displayed proximate to the position on the image that was determined based on the information received from the user input device.

31. The system of claim 24 further comprising a network, a server storing the image and data representing the surface positions, wherein the memory receives the image and data from the server and over the network.

32. The system of claim 24 wherein the image is a still image.

33. The system of claim 24 wherein the image is a video.

34. The system of claim 24 wherein the data representing the position of the surfaces represents a plurality of points on the surface of the building.

35. The system of claim 34 wherein each point is associated with a distance and angle relative to the camera position.

36. The system of claim 34 wherein each point is associated with a latitude, longitude, and altitude.

37. The system of claim 24 wherein the data representing the position of the surfaces represents a plurality of polygons having vertices at locations corresponding with the surfaces.

38. A method of displaying images comprising:

receiving, by one or more computing devices, a street-level image representing a plurality of geographical objects captured by a camera at a camera position;

receiving, by the one or more computing devices, distance data representing the distance between the geographical objects and the camera position, the distance data including a two-dimensional array of distance values for each pixel of the received street-level image that represents a geographical object;

receiving, by the one or more computing devices, a cursor position, where the cursor position identifies a position on the street-level image at which to display a cursor, the cursor position corresponding to a surface of a first geographic object of the plurality of geographical objects;

displaying, by the one or more computing devices, on the display, the street-level image;

selecting, by the one or more computing devices, distance data associated with the surface of the first geographical object, wherein selecting the distance data includes selecting at least a first distance value from the two-dimensional array based on the cursor position;

determining, by the one or more computing devices, a visual characteristic of the cursor based on the selected distance data;

displaying, by the one or more computing devices, in the street-level image, the cursor with the visual characteristic so that the cursor conforms to the surface of the first geographic object in the street-level image, wherein a shape of the cursor is dependent on the selected data associated with the surface of the first geographical object such that the shape of the cursor appears to lie flat against the surface of the geographical object;

receiving, by the one or more computing devices, an updated cursor position, the updated cursor position identifying a second position on the street level image at which to display the cursor, the second cursor position corresponding to a surface of a second geographical object of the plurality of geographical objects;

selecting, by the one or more computing devices, second distance data associated with the surface of the second geographical object, wherein selecting the second distance data includes selecting at least a second distance value from the two-dimensional array based on the updated cursor position;

determining, by the one or more computing devices, a different visual characteristic of the cursor based on the second distance data; and

displaying, by the one or more computing devices, the cursor in the street-level image with the different visual characteristic so that the cursor conforms to the surface of the second geographical object in the street-level image.