3D Position Tracking for Panoramic Imagery Navigation

• US 20130083055A1
• Filed: 09/30/2011
• Published: 04/04/2013
• Est. Priority Date: 09/30/2011
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Piemonte, Patrick, Chen, Billy
• Granted Patent: US 9,121,724 B2
• Time in Patent Office: Days
• US Class Current: 345/629
• CPC Class Codes: G01C 21/3638 including 3D objects and buildings three dimensional [3D] modelling, e.g. data description of 3D objects G06T17/00; geographic models G06T17/05
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images

Abstract

Position tracking subsystems and onboard sensors enable a mobile device to navigate virtually a location in panoramic imagery. Physically moving the device through space provides translation data that can be used to move up or down a virtual street or other navigation actions. In some implementations, forward and backward translation enables the user to enter a structure (e.g., a commercial venue) or enter an intersection and navigate a turn onto another street at the intersection. In some implementations, information or an information layer can be displayed when translating. In some implementations, distance data can be used to move up or down a street a particular distance. Distance data can be obtained from motion and/or image sensors onboard the device. The distance data can be scaled to a virtual distance in the panoramic scene.

Claims

1. A method comprising:

displaying panoramic imagery on a display of a mobile device;

receiving input from a sensor onboard the mobile device;

determining a device translation using the input; and

navigating the imagery based on the device translation,

where the method is performed by one or more hardware processors.

2. The method of claim 1, further comprising:

displaying information or an information layer over the panoramic imagery.

3. The method of claim 1, where determining a device translation further comprises:

determining a translation direction and distance; and

navigating the imagery based on the translation direction and distance.

4. The method of claim 3, further comprising:

determining a context in the panoramic imagery; and

navigating the imagery based on the translation direction, translation distance and determined context.

5. The method of claim 3, where the sensor includes at least one of a motion sensor and an image sensor.

6. The method of claim 1, where navigating imagery further comprises:

automatically navigating a corner of a street intersection.

7. The method of claim 1, where navigating imagery further comprises:

continuously navigating imagery until a threshold translation distance is reached or exceeded.

8. The method of claim 1, further comprising:

determining if the translation has reached or exceeded a maximum or minimum translation distance; and

navigating imagery based on results of the determining.

9. The method of claim 1, where the device is a smart phone or electronic tablet.

10. The method of claim 1, where navigating the imagery based on the device translation further comprises:

mapping the translation direction to a navigation command based on a location in the panoramic imagery.

11. The method of claim 1, where displaying panoramic imagery on a display of a mobile device, further comprises displaying the panoramic imagery on two or more adjacent devices.

12. A system comprising:

one or more processors;

memory coupled to the one or more processors and configured to store instructions, which, when executed by the one or more processors, causes the one or more processors to perform operations comprising;

displaying panoramic imagery on a display of a mobile device;

receiving input from a sensor onboard the mobile device;

determining a device translation using the input; and

navigating the imagery based on the device translation.

13. The system of claim 12, where the instructions cause the one or more processors to perform operations comprising:

displaying information or an information layer over the panoramic imagery.

14. The system of claim 12, where determining a device translation further comprises:

determining a translation direction and distance; and

navigating the imagery based on the translation direction and distance.

15. The system of claim 14, where the instructions cause the one or more processors to perform operations comprising:

determining a context in the panoramic imagery; and

navigating the imagery based on the translation direction, translation distance and determined context.

16. The system of claim 14, where the sensor includes at least one of an accelerometer sensor and an image sensor.

17. The system of claim 12, where navigating imagery further comprises:

automatically navigating a corner of a street intersection.

18. The system of claim 12, where navigating imagery further comprises:

continuously navigating imagery until a threshold translation distance is reached or exceeded.

19. The system of claim 12, where the instructions cause the one or more processors to perform operations comprising:

determining if the translation has reached or exceeded a maximum or minimum translation distance; and

navigating imagery based on results of the determining.

20. The system of claim 12, where the device is a smart phone or electronic tablet.

21. The system of claim 12, where navigating the imagery based on the device translation further comprises:

mapping the translation direction to a navigation command based on a location in the panoramic imagery.

22. The system of claim 12, where displaying panoramic imagery on a display of a mobile device, further comprises displaying the panoramic imagery on two or more adjacent devices.