Map generation using map features from user captured images

• US 9,467,660 B1
• Filed: 03/31/2014
• Issued: 10/11/2016
• Est. Priority Date: 03/31/2014
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Edelstein, David Adam
• Primary Examiner(s): Aghevli, Reza
• Application Number:
  US14/231,548
• Time in Patent Office: 925 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/30 Map- or contour-matching
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast

Abstract

Various embodiments enable a user to capture an image of a map, such as a map of a municipal or amusement park, mall, golf course, or the like, using a camera of a computing device to geocode features of the map that may not be available for display through a native mapping application. For example, a park map may show walking trails, vista points, restrooms, playgrounds, or other locations of interest, which may not be represented by the mapping application. Accordingly, once these previously unrepresented features are associated with a particular location, they can be represented within a map of a native mapping application of the user'"'"'s computing device, thereby, enabling the user to see their location within the map relative to these features and navigate through the park using the same.

Claims

1. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor, cause a computing device to:

capture, using a camera of the computing device, an image of a park map, the park map having at least one of a difference in scale or a difference in perspective relative to a portion of a world map available via a map application stored in memory of the computing device;

determine a region associated with the portion of the world map using a current location of the computing device;

analyze the image to determine a set of park features of the park map;

compare the set of park features to features of the portion of the world map for the region available via the map application;

match a first park feature with a first world map feature, wherein the first park feature is included in the set of park features and the first world map feature is included in the features of the portion of the world map;

match a second park feature with a second world map feature, wherein the second park feature is included in the set of park features and the second world map feature is included in the features of the portion of the world map;

generate a first anchor for the image based on the first park feature matching the first world map feature;

generate a second anchor for the image based on the second park feature matching the second world map feature;

determine an uncommon park map feature from the set of park features, the uncommon park map feature being omitted from the map application;

geocode the first anchor to have a first set of latitude and longitude coordinates;

geocode the second anchor to have a second set of latitude and longitude coordinates;

assign a third set of latitude and longitude coordinates to the uncommon park map feature, the third set of latitude and longitude coordinates being determined based at least partly on a scaling factor that compensates for spatial inconsistencies between the first world map feature and the second world map feature relative to the first park feature and the second park feature; and

display an indication of the current location of the computing device on the image of the park map based on the first set of latitude and longitude coordinates, the second set of latitude and longitude coordinates, and the third set of latitude and longitude coordinates.

2. The non-transitory computer-readable storage medium of claim 1, wherein at least a subset of the park map features and the features of the world map associated with the map application are at least one of physical features or map labels, and wherein comparing the set of park map features to the features of the world map includes at least one of:

comparing, using a feature recognition algorithm, shapes of physical features of the park map to the shapes of physical features of the world map associated with the map application for the current location to determine the first anchor and the second anchor;

or

recognizing, using an optical character recognition engine, text of park map labels from the image and map labels from the world map for the current location; and

comparing the recognized text of the park map labels from the image and the map labels from the world map for the current location to determine the first anchor and the second anchor.

3. The non-transitory computer-readable storage medium of claim 1, wherein analyzing the image to determine the set of park features includes identifying the set of park features of the park map using at least one of a Scale-Invariant Feature Transform (SIFT) algorithm or a Speeded Up Robust Features (SURF) algorithm.

4. The non-transitory computer-readable storage medium of claim 1, wherein adjusting the scaling factor of the uncommon park map feature includes locally warping the at least one of the uncommon park map feature using an elastic transformation algorithm, wherein locally warping includes at least one of stretching, elongating, enlarging, or shrinking the uncommon park map feature to align with the features of the world map based at least in part on the first anchor and the second anchor.

5. A computer-implemented method, comprising:

capturing an image of a first map using an image capture element of a computing device;

analyzing the image to determine a first feature of the first map, the first feature representing a first object at a first geographic location;

analyzing the image to determine a second feature of the first map, the second feature representing a second object at a second geographic location;

associating the first feature to a third feature of a stored second map, the third feature representing the first object;

associating the second feature to a fourth feature of the stored second map, the fourth feature representing the second object;

determining an uncommon feature that is present in the first map and absent from the stored second map;

associating, using a scaling factor, the uncommon feature with a geographic location; and

displaying, on a display screen of the computing device, an indication of a current location of the computing device in a third map, the third map including the uncommon feature displayed concurrently with features common to both the first map and the stored second map.

6. The computer-implemented method of claim 5, further comprising:

adjusting, using the at least one scaling factor, at least one of a scale or relative location of the uncommon feature of the first map to match the scale of the second map; and

generating, by a native map application of the computing device, the third map for display, the third map including the second map and the uncommon feature as an overlay to the second map.

7. The computer-implemented method of claim 5, further comprising:

adjusting, using the at least one scaling factor, a scale of the first map to match the scale of the second map; and

generating the third map for display, the third map comprising the image of the first map adjusted to match the scale of the second map.

8. The computer-implemented method of claim 5, further comprising:

adjusting, using the at least one scaling factor, a scale of the first map to match the scale of the second map; and

generating the third map for display, the third map comprising the second map adjusted to match the scale of the image of the first map.

9. The computer-implemented method of claim 5, further comprising:

comparing the set of first features to a set of second features of the second map, wherein comparing the set of first features to the set of second features includes comparing an associated definition of a color of a respective first feature to a corresponding second feature.

10. The computer-implemented method of claim 5, wherein the set of features of the first map are physical features, and wherein determining at least one uncommon feature that is absent from the second map includes:

comparing, using a shape recognition algorithm, shapes of physical features of the first map to the shapes of physical features of the second map for the current location to determine a set of anchors for displaying the at least one uncommon feature concurrently with the features common to both the first map and the second map.

11. The computer-implemented method of claim 10, wherein the physical features common to both the first map and the second map include at least one of roads, landmarks, or geographic features, and wherein the at least one uncommon feature is at least one of a trail, a recreational area, or a point of interests.

12. The computer-implemented method of claim 5, wherein set of features of the first map are map labels, and wherein determining at least one uncommon feature that is absent from the second map includes:

recognizing, using an optical character recognition engine, text of the map labels from the first map and text of map labels from the second map for the current location; and

comparing the recognized text of the map labels from the first map to the recognized text of the map labels from the second map to determine a set of anchors for displaying the at least one uncommon feature concurrently with the features common to both the first map and the second map.

13. The computer-implemented method of claim 5, further comprising:

supplementing a native map application on the computing device with map data for the at least one uncommon feature.

14. The computer-implemented method of claim 5, wherein analyzing the image to determine the set of features of the first map includes identifying the set of features using at least one of a Scale-Invariant Feature Transform (SIFT) algorithm or a Speeded Up Robust Features (SURF) algorithm.

15. A computing device, comprising:

a processor;

a display screen; and

memory including instructions that, when executed by the processor, cause the computing device to;

capture, using a camera of the computing device, an image of a park map;

analyze the image to determine a set of park features of the park map;

compare the set of park features to features of a world map stored in memory;

match a first park feature with a first world map feature, wherein the first park feature is included in the set of park features and the first world map feature is included in the features of the world map;

match a second park feature with a second world map feature, wherein the second park feature is included in the set of park features and the second world map feature is included in the features of the world map;

generate a first anchor for the image based on the first park feature matching the first world map feature;

generate a second anchor for the image based on the second park feature matching the second world map feature;

determine an uncommon park map feature, wherein the uncommon park map feature is absent from the world map;

geocode the first anchor to have a first set of latitude and longitude coordinates;

geocode the second anchor to have a second set of latitude and longitude coordinates;

assign a third set of latitude and longitude coordinates to the uncommon park map feature based at least partly on a scaling factor between the first world map feature and the second world map feature relative to the first park feature and the second park feature; and

display an indication of a current location of the computing device on at least one of the world map relative to the uncommon feature or the image of the park map based on the first set of latitude and longitude coordinates, the second set of latitude and longitude coordinates, and the third set of latitude and longitude coordinates.

16. The computing device of claim 15, wherein at least a subset of the set features of the first map and the features of the second map are map labels, and wherein comparing the set of features of the first map to the features of the second map includes:

recognizing, using an optical character recognition engine, text of map labels from the image and map labels from the map for the current location; and

comparing the recognized text of the map labels from the image and the map labels from the map for the current location to determine the first anchor and the second anchor.

17. The computing device of claim 15, wherein comparing the set of features to the features of the second map includes comparing color of a respective first feature to a corresponding second feature.

18. The computing device of claim 15, wherein the instructions that, when executed by the processor, further cause the computing device to:

supplement a native map application on the computing device with map data for the at least one uncommon feature.

19. The computing device of claim 15, wherein at least a subset of the features of the first map and the features of the second map are physical features, and wherein comparing the set of features of the first map to the features of the second map includes:

comparing, using a shape recognition algorithm, shapes of physical features of the first map to the shapes of physical features of the second map to determine the first anchor and the second anchor.

20. The computing device of claim 19, wherein the physical features of the first map and the second map include at least one of roads, landmarks, or geographic features, and wherein the uncommon park map feature is at least one of a trail, a recreational area, or a point of interests.