Systems and methods for navigating a camera

• US 9,189,891 B2
• Filed: 03/19/2012
• Issued: 11/17/2015
• Est. Priority Date: 08/16/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Hernandez Esteban, Carlos
• Primary Examiner(s): Zimmerman, Mark
• Assistant Examiner(s): KAJI, SOHUM S
• Application Number:
  US13/424,313
• Publication Number:
  US 20130044139A1
• Time in Patent Office: 1,338 Days
• Field of Search: 345/473
• US Class Current: 1/1
• CPC Class Codes: G01C 21/3638 including 3D objects and buildings three dimensional [3D] modelling, e.g. data description of 3D objects G06T17/00; geographic models G06T17/05
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images

Abstract

Systems and methods for navigating a camera are provided. In some aspects, a method includes identifying an initial projection of a target on a viewport of the camera. The initial projection is associated with an initial camera pose of the camera. The method also includes determining a final projection of the target on the viewport. The final projection is associated with a final camera pose of the camera. The method also includes calculating intermediate camera poses of the camera along a navigational path between the initial camera pose and the final camera pose. Each of the intermediate camera poses is associated with a corresponding intermediate projection of the target on the viewport. The intermediate camera poses is calculated such that each subsequent one of the intermediate projections is closer to the final projection than is a previous one of the intermediate projections.

Claims

1. A computer-implemented method for navigating a camera, the method comprising:

identifying, using one or more computing devices, an initial projection of a target on a viewport of the camera, the initial projection being associated with an initial camera pose of the camera and an initial size of the target;

determining, using the one or more computing devices, a final projection of the target on the viewport, the final projection being associated with a final camera pose of the camera and a final size of the target, the final camera pose corresponding to a final camera image at an altitude lower than an altitude of an initial camera image corresponding to the initial camera pose;

calculating, using the one or more computing devices, a plurality of intermediate camera poses of the camera along a navigational path between the initial camera pose and the final camera pose; and

determining, using the one or more computing devices, a plurality of intermediate projections of the target correlating to the plurality of intermediate camera poses, each of the plurality of intermediate projections being associated with an intermediate size of the target and the plurality of intermediate positions is determined such that inverses of the plurality of intermediate sizes is linearly interpolated between an inverse of the initial size of the target and an inverse of the final size of the target,

wherein each subsequent one of the plurality of intermediate projections is closer to the final projection of the target than is a previous one of the plurality of intermediate projections.

2. The method of claim 1, further comprising determining the final camera pose of the camera based on the target, the final camera pose comprising at least one of a final position of the camera and a final orientation of the camera.

3. The method of claim 2, wherein the final camera image comprises a panorama image displaying the target, and wherein the final position is at a center of the panorama image.

4. The method of claim 3, wherein the panorama image is spherical, and wherein the final position is centered within the panorama image.

5. The method of claim 2, wherein determining the final camera pose comprises:

generating a billboard image displaying the target; and

determining the final position of the camera and the final orientation of the camera such that the viewport of the camera is aligned with the billboard image.

6. The method of claim 5, wherein the billboard image comprises a two-dimensional plane perpendicular to a ground plane.

7. The method of claim 5, further comprising removing the billboard image once the camera assumes the final camera pose.

8. The method of claim 5, wherein the final camera image comprises a panorama image displaying the target, and wherein the panorama image at least partially surrounds the billboard image.

9. The method of claim 5, wherein the billboard image is defined by a viewing position and a viewing orientation.

10. The method of claim 9, wherein determining the final position of the camera and the final orientation of the camera comprises matching the final position to the viewing position and matching the final orientation to the viewing orientation.

11. The method of claim 1, wherein each of the plurality of intermediate camera poses comprises at least one of an intermediate position of the camera and an intermediate orientation of the camera.

12. The method of claim 11, wherein calculating the plurality of intermediate camera poses comprises:

determining the plurality of intermediate orientations of the camera between the initial camera pose and the final camera pose; and

determining the plurality of intermediate positions between the initial camera pose and the final camera pose based on the plurality of intermediate orientations.

13. The method of claim 12, wherein determining the plurality of intermediate orientations comprises linearly interpolating the plurality of intermediate orientations between the initial camera pose and the final camera pose.

14. The method of claim 11, wherein calculating the plurality of intermediate camera poses comprises:

determining the plurality of intermediate positions of the camera between the initial camera pose and the final camera pose; and

determining the plurality of intermediate orientations between the initial camera pose and the final camera pose based on the plurality of intermediate positions.

15. The method of claim 14, wherein determining the plurality of intermediate positions comprises interpolating the plurality of intermediate positions between the initial camera pose and the final camera pose using a three-dimensional spline process.

16. The method of claim 1, wherein the initial projection and the initial camera pose are associated with an initial field-of-view (FOV) of the camera, wherein the final projection and the final camera pose are associated with a final FOV of the camera, and wherein the method further comprises calculating a plurality of intermediate FOVs of the camera along the navigational path between the initial camera pose and the final camera pose, each of the plurality of intermediate FOVs being associated with a corresponding intermediate projection of the target on the viewport.

17. The method of claim 16, wherein calculating the plurality of intermediate FOVs comprises linearly interpolating the plurality of intermediate FOVs between the initial FOV and the final FOV along the navigational path.

18. The method of claim 16, wherein each of the plurality of intermediate camera poses comprises an intermediate position of the camera, and wherein calculating the plurality of intermediate camera poses comprises determining the plurality of intermediate positions between the initial camera pose and the final camera pose based on the plurality of intermediate FOVs.

19. The method of claim 1, further comprising navigating the camera along the navigational path from the initial camera pose to the final camera pose in at least 0.4 seconds and at most 1 second.

20. The method of claim 1, wherein the plurality of intermediate projections is aligned in a straight line between the initial projection and the final projection.

21. The method of claim 1, wherein the plurality of intermediate camera poses is calculated such that each of the plurality of intermediate projections of a center of the target remains on the viewport.

22. The method of claim 1, wherein the target comprises a point of interest in a mapping application.

23. A system for navigating a camera, the system comprising:

a memory storing a plurality of projections; and

one or more processors coupled to the memory, the one or more processor being configured to;

identify an initial projection of a target on a viewport of the camera and to determine a final projection of the target on the viewport, the initial projection being associated with an initial camera pose of the camera and an initial size of the target, the final projection being associated with the final camera pose and a final size of the target, the final camera pose corresponding to a final camera image;

calculate a plurality of intermediate camera poses of the camera along a navigational path between the initial camera pose and the final camera pose; and

determine a plurality of intermediate projections of the target correlating to the plurality of intermediate camera poses, each of the plurality of intermediate projections being associated with an intermediate size of the target and the plurality of intermediate positions is determined such that inverses of the plurality of intermediate sizes is linearly interpolated between an inverse of the initial size of the target and an inverse of the final size of the target,

wherein each subsequent one of the plurality of intermediate projections is closer to the final projection than is a previous one of the plurality of intermediate projections.

24. The system of claim 23, wherein the initial projection and the initial camera pose are associated with an initial field-of-view (FOV) of the camera, wherein the final projection and the final camera pose are associated with a final FOV of the camera, and wherein the one or more processors are further configured to calculate a plurality of intermediate FOVs of the camera along the navigational path between the initial camera pose and the final camera pose, each of the plurality of intermediate FOVs being associated with a corresponding intermediate projection of the target on the viewport.

25. The system of claim 24, wherein each of the plurality of intermediate camera poses comprises an intermediate position of the camera, and wherein the one or more processors are further configured to determine the plurality of intermediate positions between the initial camera pose and the final camera pose based on the plurality of intermediate FOVs.

26. The system of claim 25, wherein the one or more processors are configured to determine the plurality of intermediate positions such that the initial projection, the plurality of intermediate projections, and the final projection remain substantially the same size in response to the plurality of intermediate FOVs.

27. A non-transitory machine-readable medium encoded with executable instructions for navigating a camera, the instructions comprising code for:

Identifying an initial projection of a target on a viewport of the camera, the initial projection being associated with an initial camera pose of the camera and an initial size of the target, the initial camera pose corresponding to an aerial image displaying the target;

determining, using the one or more processors, a final camera pose of the camera based on the target and a final size of the target, the final camera pose corresponding to a street-level panoramic image displaying the target;

determining a final projection of the target on the viewport, the final projection being associated with the final camera pose of the camera;

calculating a plurality of intermediate camera poses of the camera along a navigational path between the initial camera pose and the final camera pose; and

determining a plurality of intermediate projections of the target correlating to the plurality of intermediate camera poses, each of the plurality of intermediate projections being associated with an intermediate size of the target and the plurality of intermediate positions is determined such that inverses of the plurality of intermediate sizes is linearly interpolated between an inverse of the initial size of the target and an inverse of the final size of the target,

wherein each subsequent one of the plurality of intermediate projections is closer to the final projection than is a previous one of the plurality of intermediate projections, and

wherein the plurality of intermediate projections is aligned in a straight line between the initial projection and the final projection.

28. The non-transitory machine-readable medium of claim 27, wherein the initial projection and the initial camera pose are associated with an initial field-of-view (FOV) of the camera, wherein the final projection and the final camera pose are associated with a final FOV of the camera, and wherein the instructions further comprise code for calculating a plurality of intermediate FOVs of the camera along the navigational path between the initial camera pose and the final camera pose, each of the plurality of intermediate FOVs being associated with a corresponding intermediate projection of the target on the viewport.

29. The non-transitory machine-readable medium of claim 28, wherein calculating the plurality of intermediate FOVs comprises linearly interpolating the plurality of intermediate FOVs between the initial FOV and the final FOV along the navigational path.