Navigation through multidimensional images spaces

• US 10,217,283 B2
• Filed: 12/17/2015
• Issued: 02/26/2019
• Est. Priority Date: 12/17/2015
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Dillard, Scott Edward, Castaneda, Humberto, Leong, Su Chuin, Jones, Michael Cameron, Gray, Christopher, Parker, Evan Hardesty
• Primary Examiner(s): Salomon, Phenuel S
• Application Number:
  US14/972,843
• Publication Number:
  US 20170178404A1
• Time in Patent Office: 1,167 Days
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/0488 using a touch-screen or digitiser, e.g. input of commands through traced gestures
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 3/04886 by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

Aspects of the disclosure relate generally to providing a user with an image navigation experience. For instance, a first image of a multidimensional space is provided with an overlay line indicating a direction in which the space extends into the first image such that a second image is connected to the first image along a direction of the overlay line. User input indicating a swipe across a portion of the display is received. When swipe occurred at least partially within an interaction zone defining an area around the overlay line at which the user can interact with the space, the swipe indicates a request to display an image different from the first image. The second image is selected and provided for display based on the swipe and a connection graph connecting the first image and the second image along the direction of the overlay line.

Claims

1. A method of navigating multidimensional spaces, the method comprising:

providing, by one or more processors, a first image of a multidimensional space for display on a display of a client computing device and an overlay line extending across a portion of the first image and indicating a direction in which the multidimensional space extends into the first image such that a second image is connected to the first image along a direction of the overlay line;

receiving, by the one or more processors, user input indicating a swipe across a portion of the display, the swipe being defined by a starting pixel and an ending pixel of the display;

determining, by the one or more computing devices, based on the starting pixel and the ending pixel, that the swipe occurred at least partially within an interaction zone of the first image, the interaction zone defining an area around the overlay line at which the user can interact with the multidimensional space, the interaction zone encasing the area around the overlay line with the area around the overlay line expanding a width of the overlay line;

when the swipe occurred at least partially within the interaction zone, determining, by the one or more processors, that the swipe indicates a request to display an image different from the first image;

when the swipe indicates a request to display the image different from the first image, selecting, by the one or more computing devices, the second image based on a starting point of the swipe, an ending point of the swipe, and a connection graph connecting the first image and the second image along the direction of the overlay line; and

providing, by the one or more computing devices, the second image for display on the display in order to provide a feeling of movement in the multidimensional space.

2. The method of claim 1, further comprising, providing a transition image for display between the first image and the second image, the transition image being provided as a thumbnail image with less detail than the first image and the second image.

3. The method of claim 1, further comprising, providing instructions to fade the overlay line out after a threshold period of time has passed without any user action with the overlay line.

4. The method of claim 3, further comprising, after fading the overlay line out:

receiving second user input on the display; and

providing instructions to redisplay the overlay line in response to the second user input.

5. The method of claim 1, further comprising:

determining a direction and magnitude of the swipe based on the starting pixel of the swipe and the ending pixel of the swipe, and

wherein selecting the second image is further based on the direction and magnitude.

6. The method of claim 1, further comprising providing the second image with a second overlay line extending across a portion of the second image and indicating a direction in which the multidimensional space extends into the second image such that a third image is connected to the second image along a direction of the second overlay line in the connection graph.

7. The method of claim 6, further comprising:

receiving second user input indicating a second swipe;

determining that the second swipe is within a threshold angle perpendicular to the direction in which the multidimensional space extends into the second image; and

when the second swipe is within a threshold angle perpendicular to the direction in which the multidimensional space extends into the second image, panning across multidimensional space of the second image.

8. The method of claim 6, further comprising:

receiving second user input indicating a second swipe;

determining that the second swipe is within a threshold angle perpendicular to the direction in which the multidimensional space extends into the second image; and

when the second swipe is within a threshold angle perpendicular to the direction in which the multidimensional space extends into the second image, changing an orientation within the second image.

9. The method of claim 6, further comprising:

receiving second user input indicating a second swipe;

determining that the second swipe is within a threshold angle perpendicular to the direction in which the multidimensional space extends into the second image; and

when the second swipe is within a threshold angle perpendicular to the direction in which the multidimensional space extends into the second image, switching from the second image to a third image located on a second connection graph adjacent to the connection graph, the second image and the third image having no direction connection in the connection graph.

10. The method of claim 9, further comprising proving for display with the second image, a third overlay line, the third overlay line representing a second navigation path proximate to a current view of the second image, the third overlay line being provided such that the third overlay line and the second overlay line cross over one another when displayed with the second image.

11. The method of claim 10, further comprising:

receiving second user input along the third overlay line indicating a request to transition from an image along the second overlay line to an image along the third overlay line; and

in response to the second user input, providing a third image for display, the third image being arranged along the third overlay line in the connection graph.

12. The method of claim 11, further comprising:

selecting a set of images for display in series as a transition between the second image and the third image based on connections between images in the connection graph; and

providing the set of images for display on the display.

13. The method of claim 12, further comprising, prior to providing the set of images, filtering the set of images to remove at least one image based on a connection between two images of the set of images in a second connection graph different from the first connection graph such that the filtered set of images are provided for display as the transition between the second image and the third image.

14. A system comprising one or more computing devices, each having one or more processors, the one or more computing devices being configured to:

provide a first image of a multidimensional space for display on a display of a client computing device and an overlay line extending across a portion of the first image and indicating a direction in which the multidimensional space extends into the first image such that a second image is connected to the first image along a direction of the overlay line;

receive user input indicating a swipe across a portion of the display, the swipe being defined by a starting pixel and an ending pixel of the display;

determine, based on the starting pixel and the ending pixel, that the swipe occurred at least partially within an interaction zone of the first image, the interaction zone defining an area around the overlay line at which the user can interact with the multidimensional space, the interaction zone encasing the area around the overlay line with the area expanding a width of the overlay line;

when the swipe occurred at least partially within the interaction zone, determine that the swipe indicates a request to display an image different from the first image;

when the swipe indicates a request to display the image different from the first image, select the second image based on a starting point of the swipe, an ending point of the swipe, and a connection graph connecting the first image and the second image along the direction of the overlay line; and

provide the second image for display on the display in order to provide a feeling of movement in the multidimensional space.

15. The system of claim 14, wherein the one or more computing devices are further configured to provide a transition image for display between the first image and the second image, the transition image being provided as a thumbnail image with less detail than the first image and the second image.

16. The system of claim 14, wherein the one or more computing devices are further configured to provide instructions to fade the overlay line out after a threshold period of time has passed without any user action with the overlay line.

17. The system of claim 16, wherein the one or more computing devices are further configured to, after fading the overlay line out:

receive second user input on the display; and

provide instructions to redisplay the overlay line in response to the second user input.

18. The system of claim 14, wherein the one or more computing devices are further configured to:

determine a direction and magnitude of the swipe based on the starting pixel of the swipe and the ending pixel of the swipe, and

to select the second image further based on the direction and magnitude.

19. The system of claim 14, wherein the one or more computing devices are further configured to provide the second image with a second overlay line extending across a portion of the second image and indicating a direction in which the multidimensional space extends into the second image such that a third image is connected to the second image along a direction of the second overlay line in the connection graph.

20. A non-transitory, computer-readable storage device on which computer readable instructions of a program are stored, the instructions, when executed by one or more processors, cause the one or more processors to perform a method, the method comprising:

providing a first image of a multidimensional space for display on a display of a client computing device and an overlay line extending across a portion of the first image and indicating a direction in which the multidimensional space extends into the first image such that a second image is connected to the first image along a direction of the overlay line;

receiving user input indicating a swipe across a portion of the display, the swipe being defined by a starting pixel and an ending pixel of the display;

determining, based on the starting pixel and the ending pixel, that the swipe occurred at least partially within an interaction zone of the first image, the interaction zone defining an area around the overlay line at which the user can interact with the multidimensional space, the interaction zone encasing the area around the overlay line with the area expanding a width of the overlay line;

when the swipe occurred at least partially within the interaction zone, determining that the swipe indicates a request to display an image different from the first image;

when the swipe indicates a request to display the image different from the first image, selecting the second image based on a starting point of the swipe, an ending point of the swipe, and a connection graph connecting the first image and the second image along the direction of the overlay line; and

providing the second image for display on the display in order to provide a feeling of movement in the multidimensional space.