Image-aided positioning and navigation system

• US 8,509,488 B1
• Filed: 02/24/2010
• Issued: 08/13/2013
• Est. Priority Date: 02/24/2010
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Enge, Per K., Wang, Yi-Hsiu
• Primary Examiner(s): Newman, Michael A
• Application Number:
  US12/711,913
• Time in Patent Office: 1,266 Days
• Field of Search: 382/106, 382/113, 382/154, 382/181, 382/285, 701/448, 701/223
• US Class Current: 382/106
• CPC Class Codes: G01C 21/3623 using a camera or code reader, e.g. for optical or magnetic codes

Abstract

Satellite-based navigation techniques can help determine a user'"'"'s position on a map and can also determine directions to a destination. Functionality can be implemented to determine the user'"'"'s orientation and direction of motion based on a captured image taken in the direction of motion. The captured image may be preprocessed to compensate for a camera angle and to vary image properties before being transmitted to a server. The user'"'"'s orientation may be determined based, at least in part, on knowledge of a user'"'"'s position on a map and comparing the captured image to a set of other images. Walking directions can accordingly be tailored based on the user'"'"'s orientation.

Claims

1. A method comprising:

receiving, at a server, position information of a mobile device indicative of a current location of the mobile device;

determining whether one or more of a plurality of reference images, associated with the server, are associated with the position information of the mobile device;

determining a resolution of an image that should be received from the mobile device based, at least in part, on a number of the plurality of reference images that are associated with the position information of the mobile device;

transmitting, to the mobile device, an indication of the resolution of an image that should be provided from the mobile device;

receiving, at the server, a first image captured by the mobile device indicative of an orientation of the mobile device;

comparing the first image to the one or more of the plurality of reference images, if the one or more of the plurality of reference images are associated with the position information of the mobile device; and

determining an orientation of the mobile device based, at least in part, on results of said comparing the first image to the one or more of the plurality of reference images associated with the position information of the mobile device.

2. The method of claim 1, wherein:

said comparing the first image to the one or more of the plurality of reference images that are associated with the position information of the mobile device further comprises identifying a reference image that matches the first image; and

said determining the orientation of the mobile device further comprises determining the orientation based on the reference image that matches the first image.

3. The method of claim 2, wherein said determining the orientation based on the reference image that matches the first image further comprises determining location and orientation information associated with the reference image that matches the first image and associating the orientation information associated with the reference image that matches the first image with the mobile device.

4. The method of claim 2, wherein said identifying the reference image that matches the first image further comprises determining an image of the plurality of reference images associated with the position information of the mobile device that matches the first image at least by a predefined value.

5. The method of claim 1, wherein said comparing the first image to the one or more of the plurality of reference images further comprises comparing objects in the first image with objects in the one or more of the plurality of reference images associated with the position information of the mobile device to identify a reference image that comprises objects that match the objects in the first image.

6. The method of claim 1, wherein the position information of the mobile device comprises location coordinates of the mobile device.

7. The method of claim 1, wherein the plurality of reference images, associated with the server, comprise images, stored in the server, that are associated with various geographic locations, and images downloaded to the server from an external server.

8. The method of claim 1, further comprising:

receiving, at the server, a first distance between the mobile device and a focused object in the first image from the mobile device;

estimating position information of the focused object in the first image, indicative of a location of the focused object, based on comparing the first image to the one or more of the plurality of reference images associated with the position information of the mobile device and identifying the focused object;

determining a second distance between the mobile device and the focused object in the first image, based on said determining the position information of the focused object in the first image;

determining whether the first distance and the second distance are equal; and

re-estimating the position information of the mobile device to yield a new position information of the mobile device if it is determined that the first distance and the second distance are not equal.

9. The method of claim 1, further comprising:

determining an orientation of the mobile device cannot be determined based on said comparing the first image to the one or more of the plurality of reference images associated with the position information of the mobile device; and

transmitting, to the mobile device, a request for a second image of the current location of the mobile device to determine the orientation of the mobile device.

10. The method of claim 1, further comprising associating with the server at least one of the first image, the position information of the mobile device, and the orientation of the mobile device.

11. The method of claim 1, wherein said receiving the position information of the mobile device indicative of the current location of the mobile device comprises receiving the position information from at least one of a plurality of satellites, determining the position information using one or more Wi-Fi based positioning techniques, and determining the position information using one or more cellular-based positioning techniques.

12. A method comprising:

capturing a first image at a mobile device, wherein the first image is indicative of a current location and orientation of the mobile device;

estimating position information associated with the mobile device, wherein the position information is indicative of the current location of the mobile device;

transmitting the position information associated with the mobile device to a server;

receiving, from the server, image resolution information for the first image based, in part, on the position information of the mobile device, wherein the image resolution information is indicative of a first resolution;

processing the first image in response to said receiving the image resolution information to yield a processed first image having the first resolution;

transmitting the processed first image at the first resolution to the server; and

receiving, from the server, an orientation of the mobile device determined based on the first image and the position information.

13. The method of claim 12, wherein the mobile device comprises satellite-based navigation capabilities and wherein said estimating the position information associated with the mobile device further comprises estimating the position information based on a plurality of satellites.

14. The method of claim 13, further comprising determining routing information based, at least in part, on the orientation of the mobile device, wherein said determining the routing information comprises:

determining position information of a destination location; and

determining the routing information based, at least in part, on the orientation of the mobile device, the position information associated with the mobile device comprising satellite-based navigation capabilities, and the position information of the destination location.

15. The method of claim 12, further comprising:

determining an angle of inclination and a direction of inclination based on said capturing the first image at the mobile device;

determining that the angle of inclination of the first image is not zero; and

processing the first image to compensate for inclination by rotating the first image in a direction opposite to the direction of inclination by the angle of inclination.

16. The method of claim 12, further comprising:

capturing a second image at the mobile device, wherein the second image is indicative of the current location and orientation of the mobile device;

determining an angle of separation between the first image and the second image; and

transmitting, to the server, the first image, the second image, and the angle of separation between the first image and the second image.

17. A server comprising:

an image database unit configured to store a plurality of reference images of a plurality of geographic locations; and

an orientation calculation unit configured to;

receive position information, of a mobile device, indicative of a current location of the mobile device;

determine whether one or more of the plurality of reference images, stored in the image database unit, are associated with the position information of the mobile device;

determine a resolution of an image that should be received from the mobile device based, at least in part, on a number of the plurality of reference images that are associated with the position information of the mobile device;

transmit, to the mobile device, an indication of the resolution of an image that should be provided from the mobile device;

receiving, at the server, a first image captured by a mobile device indicative of an orientation of the mobile device;

compare the first image to the one or more of the plurality of reference images, if the one or more of the plurality of reference images are associated with the position information of the mobile device; and

determine an orientation of the mobile device based, at least in part, on results of the orientation calculation unit comparing the first image to the one or more of the plurality of reference images associated with the position information of the mobile device.

18. The server of claim 17, wherein:

the orientation calculation unit configured to compare the first image to the one or more of the plurality of reference images that are associated with the position information of the mobile device further comprises the orientation calculation unit configured to identify a reference image that matches the first image; and

the orientation calculation unit configured to determine the orientation of the mobile device further comprises the orientation calculation unit configured to determine the orientation based on the reference image that matches the first image.

19. The server of claim 18, wherein the orientation calculation unit configured to determine the orientation based on the reference image that matches the first image further comprises the orientation calculation unit configured to determine location and orientation information associated with the reference image that matches the first image and to associate the orientation information associated with the reference image that matches the first image with the mobile device.

20. The server of claim 17, wherein the orientation calculation unit configured to compare the first image to the one or more of the plurality of reference images further comprises the orientation calculation unit configured to compare objects in the first image with objects in the one or more of the plurality of reference images associated with the position information of the mobile device to identify a reference image that comprises objects that match the objects in the first image.

21. The server of claim 17, wherein the orientation calculation unit is further configured to:

receive, at the server, a first distance between the mobile device and a focused object in the first image from the mobile device;

estimate position information of the focused object in the first image, indicative of a location of the focused object, based on the orientation calculation unit comparing the first image to the one or more of the plurality of reference images associated with the position information of the mobile device and identifying the focused object;

determine a second distance between the mobile device and the focused object in the first image, based on the orientation calculation unit determining the position information of the focused object in the first image;

determine whether the first distance and the second distance are equal; and

re-estimate the position information of the mobile device to yield a new position information of the mobile device if it is determined that the first distance and the second distance are not equal.

22. A mobile device comprising:

an image processing unit configured to;

capture a first image, wherein the first image is indicative of a current location and orientation of the mobile device;

a positioning unit configured to;

estimate position information associated with the mobile device based on a plurality of satellites, wherein the position information is indicative of the current location of the mobile device;

transmit the position information associated with the mobile device to a server;

receive, from the server, image resolution information for the first image based, in part, on the position information of the mobile device, wherein the image resolution information is indicative of a first resolution;

process the first image in response to receiving the image resolution information to yield a processed first image having the first resolution;

transmit the processed first image at the first resolution to the server; and

receive, from the server, an orientation of the mobile device determined based on the first image and the position information.

23. The mobile device of claim 22, wherein the image processing unit is configured to:

determine an angle of inclination and a direction of inclination based on said the image processing unit capturing the first image at the mobile device;

determine that the angle of inclination of the first image is not zero; and

process the first image to compensate for inclination by rotating the first image in a direction opposite to the direction of inclination by the angle of inclination.

24. The mobile device of claim 22, wherein the image processing unit is configured to:

capture a second image at the mobile device, wherein the second image is indicative of the current location and orientation of the mobile device;

determine an angle of separation between the first image and the second image; and

transmit, to the server, the first image, the second image, and the angle between the first image and the second image.

25. A non-transitory machine-readable storage media having machine executable instructions stored therein, the machine executable instructions comprising instructions to:

receive, at a server, position information of a mobile device indicative of a current location of the mobile device;

determine whether one or more of a plurality of reference images, associated with the server, are associated with the position information of the mobile device;

determine a resolution of a first image that should be received from the mobile device based, at least in part, on a number of the plurality of reference images that are associated with the position information of the mobile device;

transmit, to the mobile device, an indication of the resolution of a first image that should be provided from the mobile device;

receive the first image captured by the mobile device indicative of an orientation of the mobile device;

compare the first image to the one or more of the plurality of reference images, if the one or more of the plurality of reference images are associated with the position information of the mobile device; and

determine an orientation of the mobile device based, at least in part, on results of said comparing the first image to the one or more of the plurality of reference images associated with the position information of the mobile device.

26. A non-transitory machine-readable storage media having machine executable instructions stored therein, the machine executable instructions comprising instructions to:

capture a first image at a mobile device, wherein the first image is indicative of a current location and orientation of the mobile device;

estimate position information associated with the mobile device, wherein the position information is indicative of the current location of the mobile device;

transmit the position information associated with the mobile device to a server;

receive, from the server, image resolution information for the first image based, in part, on the position information of the mobile device, wherein the image resolution information is indicative of a first resolution;

process the first image in response to said receiving the image resolution information to yield a processed first image having the first resolution;

transmit the processed first image at the first resolution to the server; and

receive, from the server, an orientation of the mobile device determined based on the first image and the position information.

27. A method comprising:

receiving, at a server, position information of a mobile device indicative of a current location of the mobile device;

determining whether one or more of a plurality of reference images, associated with the server, are associated with the position information of the mobile device;

if determined that the one or more of the plurality of reference images, associated with the server, are associated with the position information of the mobile device, comparing a first image captured by the mobile device to the one or more of the plurality of reference images, if the one or more of the plurality of reference images are associated with the position information of the mobile device, wherein the first image is indicative of an orientation of the mobile device;

determining an orientation of the mobile device based, at least in part, on results of said comparing the first image to the one or more of the plurality of reference images associated with the position information of the mobile device if determined that the one or more of the plurality of reference images, associated with the server, are not associated with the position information of the mobile device;

providing a notification to the mobile device indicating that the one or more of the plurality of reference images, associated with the server, are not associated with the position information of the mobile device, wherein the notification comprises instructions to move the mobile device;

determining one or more position estimates based on movement of the mobile device; and

estimating at least one of a direction of motion of the mobile device and the orientation of the mobile device based on said determining the one or more position estimates.

28. A method comprising:

receiving, at a server, position information of a mobile device indicative of a current location of the mobile device;

determining whether one or more of a plurality of reference images, associated with the server, are associated with the position information of the mobile device;

if determined that the one or more of the plurality of reference images, associated with the server, are associated with the position information of the mobile device, comparing a first image captured by the mobile device to the one or more of the plurality of reference images, if the one or more of the plurality of reference images are associated with the position information of the mobile device, wherein the first image is indicative of an orientation of the mobile device;

determining an orientation of the mobile device based, at least in part, on results of said comparing the first image to the one or more of the plurality of reference images associated with the position information of the mobile device if determined that the one or more of the plurality of reference images, associated with the server, are not associated with the position information of the mobile device;

providing a notification to the mobile device indicating that the one or more of the plurality of reference images, associated with the server, are not associated with the position information of the mobile device;

receive compass measurements from a compass on the mobile device; and

determining the orientation of the mobile device based on the compass measurements received from the compass on the mobile device.