Presenting image transition sequences between viewing locations

• US 10,643,386 B2
• Filed: 04/11/2018
• Issued: 05/05/2020
• Est. Priority Date: 04/11/2018
• Status: Active Grant

• Current Assignee: MFTB Holdco Incorporated
• Original Assignee: Zillow Group, Inc.
• Inventors: Li, Yuguang, Guan, Li
• Primary Examiner(s): Nguyen, Phu K
• Application Number:
  US15/950,881
• Publication Number:
  US 20190318538A1
• Time in Patent Office: 755 Days
• Field of Search: 345418
• CPC Class Codes: G06T 13/80 2D [Two Dimensional] animation, e.g. using sprites
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2219/028 Multiple view windows (top-side-front-sagittal-orthogonal)

Abstract

Techniques are described for, given two images taken at two viewing locations, generating and presenting an image transition sequence that simulates viewer movement between the viewing locations, such as without using information about relative distance between the viewing locations and other depth information to objects in an environment around the viewing locations. The start and end images for an image transition sequence may be part of panorama images acquired at the viewing locations, and generating of the image transition sequence may include selecting and using one of multiple types of transition sequences to perform rotating (e.g., from a start image'"'"'s viewing direction to face toward the viewing location for the end image), zooming (e.g., to simulate moving forward), and/or blending of the end image with a modified version of the start image, with the resulting image transition sequence played as a video or other animation.

Claims

1. A computer-implemented method comprising:

displaying, by a computing device and to a user, a first panorama image previously taken from a first location in a building, including modifying, in response to one or more user interactions, a current direction of view for the first panorama image to cause display of an area of the building that includes a second location from which a second panorama image was previously taken;

receiving, by the computing device and from an additional interaction of the user with the displayed first panorama image, a selection to change to the second panorama image;

generating, by the computing device, a visual transition sequence that simulates movement from the first location to the second location, including;

determining, by the computing device, and relative to a direct view to the second location in a straight line from the first location, an amount of an angle between the current direction of view for the first panorama image and the direct view, and that the determined amount of the angle exceeds a defined threshold for use of zooming without rotation in the visual transition sequence;

selecting, by the computing device, the current direction of view for the first panorama image as a beginning of the visual transition sequence;

selecting, by the computing device, and as an end of the visual transition sequence, a portion of the second panorama image that is in a direction of view away from the first location and corresponding to the direct view;

defining, by the computing device, and based at least in part on the determined amount of the angle exceeding the defined threshold, a first period of time for rotating and zooming that includes progressively rotating, from the beginning of the visual transition sequence at the current direction of view in the first panorama image, to the direct view to the second location and that simultaneously includes progressively increasing an amount of zoom in the first panorama image, resulting in a zoomed portion of the first panorama image centered on the direct view to the second location; and

defining, by the computing device, a second period of time for blending that begins with the zoomed portion of the first panorama image centered on the direct view to the second location and that includes progressively blending the selected portion of the second panorama image by increasing a relative proportion of the portion of the second panorama image in the direction of view corresponding to the direct view, resulting in the selected portion of the second panorama image at an end of the second period of time; and

displaying, by the computing device, and to the user over an amount of time that includes the first and second periods of time, the visual transition sequence to simulate the movement from the first location to the second location, and ending with a display of the selected portion of the second panorama image.

2. The computer-implemented method of claim 1 wherein the defining of the second period of time further includes performing further simultaneous rotating and zooming while progressively blending the selected portion of the second panorama image, wherein a first rate of zoom for the zooming during the first period of time is different from a second rate of zoom for the zooming during the second period of time, and wherein a first rate of rotation for the rotating during the first period of time is different from a second rate of rotation for the rotating during the second period of time.

3. The computer-implemented method of claim 2 wherein the generating of the visual transition sequence further includes selecting, by the computing device and based at least in part on the determined amount of the angle exceeding the defined threshold, to use a rotate-zoom type of transition that includes the rotating and zooming for the first period of time and the further rotating and zooming during the second period of time, and wherein the method further comprises, after the displaying of the visual transition sequence:

modifying, by the computing device and in response to one or more further user interactions, a direction of view for the second panorama image to cause display of an additional area of the building that includes a third location from which a third panorama image was previously taken; and

generating, by the computing device, a second visual transition sequence that simulates movement from the second location to the third location, including;

determining, by the computing device, and relative to a second direct view to the third location in a straight line from the second location, an amount of a second angle between the modified direction of view for the second panorama image and the second direct view, and that the determined amount of the second angle does not exceed the defined threshold for use of zooming without rotation in the second visual transition sequence;

selecting, by the computing device and based at least in part on the determined amount of the second angle not exceeding the defined threshold, a center-zoom type of transition to use for the second visual transition sequence that is different from the rotate-zoom type of transition used for the visual transition sequence;

selecting, by the computing device, the modified direction of view for the second panorama image as a beginning of the second visual transition sequence, and a portion of the third panorama image that is in a direction of view away from the second location as an end of the second visual transition sequence;

defining, by the computing device, and based at least in part on the selected center-zoom type of transition, a third period of time for zooming without rotating that includes progressively increasing an amount of zoom in the second panorama image, resulting in an intermediate image that includes a zoomed portion of second first panorama image centered on the second direct view to the third location; and

defining, by the computing device and based at least in part on the selected center-zoom type of transition, a fourth period of time for blending that begins with the intermediate image and that includes progressively blending the selected portion of the third panorama image over the fourth period of time while performing further zooming from a zoomed-out version of the selected portion of the third panorama image, resulting in the selected portion of the third panorama image at an end of the fourth period of time; and

displaying, by the computing device, and to the user over an amount of time that includes the third and fourth periods of time, an animation of the second visual transition sequence to simulate the movement from the second location to the third location, and ending with a display of the selected portion of the third panorama image.

4. A computer-implemented method comprising:

presenting, by a computing system and to a user, a first image taken from a first location in a first viewing direction away from the first location;

determining, by the computing system, to present a second image taken from a second location, wherein all of the second image is not visible in the presented first image in the first viewing direction away from the first location; and

presenting, by the computing system, and to the user over a period of time, a visual transition sequence that simulates movement from the first location to the second location, wherein the presenting of the visual transition sequence includes performing simultaneous rotating and zooming during a first subset of the period of time that begins with the first image and progressively increases an amount of zoom within the first image to result in an intermediate image that includes an enlarged portion of part of the first image, and includes performing blending during a second subset of the period of time that begins with the intermediate image and blends the second image with the intermediate image by progressively increasing a relative proportion of the second image that is shown until only the second image is shown, ending with a display of the second image.

5. The computer-implemented method of claim 4 wherein the presenting of the first image by the computing system includes transmitting, by the computing system and over one or more computer networks, the first image to a client computing device of the user for display on the client computing device, and wherein the determining to present the second image is based at least in part by an interaction of the user with the first image displayed on the client computing device.

6. The computer-implemented method of claim 5 wherein the first image displayed on the client computing device includes a user-selectable control displayed on the first image at a position corresponding to the second location, and wherein the interaction of the user with the first image displayed on the client computing device includes selecting the user-selectable control displayed on the first image.

7. The computer-implemented method of claim 5 wherein the first image is a first subset of a starting panorama image that is taken from the first location, and wherein the presenting of the first image by the computing system includes, before displaying of the first image on the client computing device:

displaying, on the client computing device, a third image from a second subset of the starting panorama image corresponding to a view direction that does not include a view of the second location;

receiving one or more initial interactions of the user with the displayed third image, including to change the view direction within the starting panorama image to the first viewing direction; and

selecting, in response to the one or more initial interactions and based on the changed view direction, a first subset of the starting panorama image to use as the first image, and initiating the displaying of the first image on the client computing device.

8. The computer-implemented method of claim 7 wherein the second image is a subset of an ending panorama image that is taken from the second location, and wherein the presenting of the visual transition sequence includes determining a view direction within the ending panorama image away from the first location, and selecting the subset of the ending panorama image to use as the second image based on the determined view direction.

9. The computer-implemented method of claim 4 wherein the presenting of the visual transition sequence includes selecting first parameters to use for the zooming during the first subset of the period of time that include at least one of a first rate of zoom for the zooming or a zoom-in scale to reach by an end of the first subset of the period of time, and includes selecting second parameters to use for additional zooming during the second subset of the period of time that begins with a zoomed-out image of the second image and ends with the second image, the second parameters including at least one of a second rate of zoom for the additional zooming or a zoom-out scale to use for the zoomed-out image at a start of the second subset of the period of time.

10. The computer-implemented method of claim 9 wherein the first image is part of a starting panorama image and the second image is part of an ending panorama image, wherein the method further comprises pre-computing and storing, before the presenting of the first image, the first and second parameters as part of analysis of the starting and ending panorama images to identify a link connecting the starting and ending panorama images, and wherein the selecting of the first and second parameters includes retrieving the stored pre-computed first and second parameters.

11. The computer-implemented method of claim 9 wherein the selecting of the first parameters includes selecting the zoom-in scale, wherein the selecting of the second parameters includes selecting the zoom-out scale to be a multiplicative inverse of the zoom-in scale, and wherein the presenting of the visual transition sequence further includes determining to use a received transition length as the period of time and includes computing the first and second rates of zoom based on the period of time, the zoom-in scale and the zoom-out scale.

12. A non-transitory computer-readable medium having stored contents that cause one or more computing systems to perform automated operations, the automated operations including at least:

presenting, by the one or more computing systems and to a user, a first panorama image in a first viewing direction from a first location, and a user-selectable visual indicator of a second location that is in a different second direction from the first location;

selecting, by the one or more computing systems in response to a selection by the user of the user-selectable visual indicator, and based at least in part on a determined amount of difference between the first viewing direction and the second direction, a transition sequence type, from multiple types of transition sequences, to use to simulate moving from the presented first panorama image in the first viewing direction to a second panorama image from the second location that is presented in an initial viewing direction away from the first location; and

presenting, by the one or more computing systems, and to the user over a period of time, an image transition sequence of the selected transition sequence type, wherein the presenting of the image transition sequence includes performing rotating and zooming during a first subset of the period of time that begins with the first panorama image in the first viewing direction and progressively increases an amount of zoom to reach an intermediate image that includes an enlarged portion of part of the first panorama image, and includes performing blending during a second subset of the period of time that begins with the intermediate image and changes to the second panorama image in the initial viewing direction by progressively increasing a relative proportion of the second panorama image that is shown.

13. The non-transitory computer-readable medium of claim 12 wherein the stored contents include software instructions that, when executed, perform further automated operations to determine the amount of difference and that the amount of difference satisfies a center-zoom transition threshold in which a current subset of the first panorama image corresponding to the first viewing direction includes all of a second subset of the second panorama image corresponding to the initial viewing direction, wherein the selected transition sequence type is a center-zoom transition such that the zooming performed during the first subset of the period of time occurs at a center of a first subset of the first panorama image that results after the rotating from the current subset of the first panorama image, and wherein the performing of the blending during the second subset of the period of time further includes performing additional zooming from a zoomed-out version of a second subset of the second panorama image in the initial viewing direction to reach, after the additional zooming, the second panorama image in the initial viewing direction.

14. The non-transitory computer-readable medium of claim 13 wherein the determining of the amount of difference includes measuring an angular difference between a straight line direction from the first location to the second location and the first viewing direction for the displayed first panorama image.

15. The non-transitory computer-readable medium of claim 13 wherein the software instructions further cause the one or more computing systems to generate the image transition sequence by retrieving and using pre-computed parameters for a link from the first panorama image to the second panorama image, the pre-computed parameters including at least one of a first rate of zoom for the zooming or a zoom-in scale to reach by an end of the first subset of the period of time, and further including at least one of a second rate of zoom for the additional zooming or a zoom-out scale to use for the zoomed-out version of the second subset of the second panorama image.

16. The non-transitory computer-readable medium of claim 12 wherein the stored contents include software instructions that, when executed, perform further automated operations to determine the amount of difference and that the amount of difference satisfies a rotate-zoom transition threshold in which a current subset of the first panorama image corresponding to the first viewing direction includes some of a second subset of the second panorama image corresponding to the initial viewing direction, wherein the selected transition sequence type is a rotate-zoom transition such that the zooming occurs at a location away from a center of the current subset of the first panorama image and is performed simultaneously with rotating from the current subset of the first panorama image toward a first subset of the first panorama image that results after the rotating from the current subset of the first panorama image, and wherein the performing of the blending during the second subset of the period of time further includes performing additional zooming from a zoomed-out version of a second subset of the second panorama image in the initial viewing direction to reach, after the additional zooming, the second panorama image in the initial viewing direction.

17. The non-transitory computer-readable medium of claim 16 wherein the determining of the amount of difference includes measuring an angular difference between a straight line direction from the first location to the second location and the first viewing direction for the displayed first panorama image.

18. The non-transitory computer-readable medium of claim 12 wherein the stored contents include software instructions that, when executed, further cause the one or more computing systems to present a second image transition sequence to an end image from a start image that does not include any of the end image, wherein the second image transition sequence is of a fade type of image transition sequence, and wherein presenting of the second image transition sequence includes fading from the start image to the end image without performing zooming or rotation.

19. The non-transitory computer-readable medium of claim 18 wherein the stored contents further cause the one or more computing systems to determine to use the fade type of image transition sequence from the multiple types of transition sequences based at least in part on a measured angular difference exceeding a rotate-zoom transition threshold, and wherein the measured angular difference is between a straight line direction from a location of the start image to a location of the end image and a current view direction for the start image.

20. The non-transitory computer-readable medium of claim 12 wherein the one or more computing systems are part of an image locations transition manager system, wherein the presenting of the first panorama image by the one or more computing systems includes transmitting the first panorama image over one or more computer networks to a client computing device of the user for display on the client computing device, and wherein the instruction from the user to present the second panorama image is based at least in part by an interaction of the user with a user-selectable control displayed on the first panorama image at a position corresponding to the second location.

21. A system comprising:

one or more hardware processors of one or more computing systems; and

one or more memories with stored instructions that, when executed by at least one of the one or more hardware processors, cause the one or more computing systems to perform automated operations to implement at least some functionality of an image locations transition manager system, including at least;

presenting, to a user, a subset of a first panorama image in a current view direction from a first location, wherein the first panorama image is taken from the first location;

determining, in response to an indication to present a second panorama image taken from a second location, an amount of difference between the current view direction of the first panorama image and an inter-panorama view direction from the first location of the first panorama image to the second location of the second panorama image;

selecting, based at least in part on the determined amount of difference, and from multiple types of transition sequences, a transition sequence type to use to simulate moving from the current view direction of the first panorama image to an initial view direction of the second panorama image away from the first location; and

presenting, to the user over a period of time, an image transition sequence of the selected transition sequence type, including performing rotating and zooming during a first subset of the period of time that begins with the current view direction of the first panorama image and progressively increases an amount of zoom to reach an intermediate image that includes an enlarged portion of part of the first panorama image, and including performing blending during a second subset of the period of time that begins with the intermediate image and changes to the initial view direction of the second panorama image by progressively increasing a relative proportion of the second panorama image that is shown.

22. The system of claim 21 wherein the software instructions further cause the one or more computing systems to perform automated operations to implement at least some functionality of an interior capture and analysis system, including to generate the first and second panorama images and to determine a link between the first and second panorama images in the inter-panorama view direction from the first panorama image to a subset of the first panorama image that includes a view of the second location.

23. The system of claim 21 further comprising a client computing device of the user, wherein the presenting of the subset of the first panorama image in the current view direction to the user includes receiving, by the client computing device, the first panorama image from the one or more computing systems and displaying an initial subset of the first panorama image in an initial view direction on the client computing device and changing, in response to interactions by the user, the displayed initial subset to a display of a different subset of the first panorama image in the current view direction, and wherein the presenting of the image transition sequence to the user includes receiving, by the client computing device, the image transition sequence from the one or more computing systems and displaying the image transition sequence on the client computing device over the period of time, including to display a plurality of images during the displaying of the image transition sequence.