Correcting image positioning data

• US 9,218,789 B1
• Filed: 09/26/2011
• Issued: 12/22/2015
• Est. Priority Date: 05/02/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Lininger, Scott, Anguelov, Dragomir
• Primary Examiner(s): Johnson, M Good
• Application Number:
  US13/244,799
• Time in Patent Office: 1,548 Days
• Field of Search: 345/629
• US Class Current: 1/1
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
• CPC Class Codes: G09G 2340/125 wherein one of the images is motion video
• CPC Class Codes: G09G 2340/14 Solving problems related to the presentation of information to be displayed
• CPC Class Codes: G09G 2340/145 related to small screens
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T
• CPC Class Codes: G09G 5/14 Display of multiple viewports

Abstract

An image positioning system provides an interactive visualization that includes a representation of a geographic area and several camera pose indicators, each of which indicates a location within the geographic area at which a corresponding image was obtained. An operator may select one a pose indicators and adjust the position of the pose indicator relative to the representation of the geographic area. In response, the image positioning system may automatically generate a corrected location at which the image corresponding to the selected pose indicator was obtained. The corrected location then may be stored in a database and used for various applications that utilize image positioning data.

Claims

1. A computer-implemented method for correcting image pose data stored on a computer-readable medium, wherein the image pose data includes geographic location data for each of a plurality of images, the image pose data and the plurality of images obtained during a single pose run, the method comprising:

causing a representation of a geographic area to be displayed on a display device;

determining a position of each of a plurality of pose indicators relative to the representation of the geographic area based on the geographic location data, wherein the pose indicators correspond to the single pose run and each of the plurality pose indicators corresponds to one of the plurality of images;

causing the plurality of pose indicators corresponding to the single pose run to be displayed over the representation of the geographic area on the display in accordance with the determined positions;

receiving an indication of a modified position of a selected pose indicator within the displayed single pose run, wherein the modified position is modified relative to the representation of the geographic area on the display device;

determining corrected geographic location data for the image corresponding to the selected pose indicator based on the received indication of the modified position; and

modifying the image pose data in accordance with the corrected geographic location data;

wherein the single pose run describes a trajectory of a device that obtained the image pose data and the corresponding plurality of images.

2. The method of claim 1, wherein the representation of the geographic area includes satellite imagery.

3. The method of claim 1, wherein the image pose data includes global positioning service (GPS) coordinates to indicate the geographic locations.

4. The method of claim 1, wherein the image pose data further indicates an order in which the plurality of images were obtained within the single pose run.

5. The method of claim 4, wherein the image pose data includes one of a unique sequence number or a timestamp for each of the plurality of images to indicate the order in which the plurality of images were obtained within the single pose run.

6. The method of claim 4, further comprising causing a plurality of arrows to be displayed over the representation of the geographic area, wherein the plurality of arrows interconnect the plurality of pose indicators according to the order indicated in the image pose data within the single pose run.

7. The method of claim 1, wherein causing the plurality of pose indicators to be displayed includes:

causing the selected one of the plurality of pose indicators to be displayed as an indicator of a first type to indicate that the position of the selected one of the plurality of pose indicators relative to the representation of the geographic area can be adjusted; and

in response to receiving the indication of the modified position of the selected one of the plurality of pose indicators, causing N of the plurality of pose indicators to be displayed as indicators of a second type to indicate that the position of the corresponding pose indicators relative to the representation of the geographic area cannot be adjusted.

8. The method of claim 1, further comprising:

in response to an operator command, causing one of the plurality of images that corresponds to the selected one of the plurality of pose indicators to be displayed on the display device.

9. The method of claim 1, wherein the representation of the geographic area to be displayed on the display device includes road map data.

10. The method of claim 1, wherein the plurality of images were obtained during the single pose run using a vehicle on which at least one camera is mounted.

11. The method of claim 10, further comprising causing a plurality of arrows to be displayed over the representation of the geographic area, wherein:

each of the plurality of arrows corresponds to a respective one of the plurality of images; and

each of the plurality of arrows indicates an orientation of the vehicle at a time when the corresponding one of the plurality of images was obtained.

12. An image pose data correction system comprising:

a database to store a plurality of pose records, wherein each of the plurality of pose records includes an image and pose data, wherein the pose data includes geographic location data for a geographic location at which the image was obtained, and the image and geographic location data for each pose record were obtained during a single pose run;

a pose rendering engine communicatively coupled to the database and configured to;

generate a representation of a geographic area to be displayed at a client device,

determine a position of each of a plurality of pose indicators relative to the representation of the geographic area based on the geographic location data, wherein the pose indicators correspond to a single pose run and each of the pose indicators corresponds to an image, and

generate a representation of the plurality of pose indicators corresponding to the single pose run to be displayed over the representation of the geographic area at the client device in accordance with the determined positions; and

a pose calculation engine configured to;

in response to receiving a user-modified position of a selected pose indicator within the displayed single pose run, determine corrected geographic location data for the image corresponding to the selected pose indicator based on the modified position of the selected pose indicator, wherein the modified position is modified relative to the representation of the geographic area at the client device, and

modify the pose record in accordance with the corrected geographic location;

wherein the single pose run describes a trajectory of a device that obtained the image and pose data.

13. The image processing system of claim 12, further comprising:

a pose correction user interface module to be installed on the client device and configured to;

display the representation of the geographic area on a display device;

display the representation of the plurality of pose indicators on the display device within the single pose run; and

receive the modified position of the selected one of the plurality of pose indicators from an input device.

14. The image positioning system of claim 12, wherein:

the pose rendering engine operates in a front-end server, wherein the front-end server is coupled to the client device via a first network connection, and

the pose calculation engine operates in a back-end server communicatively coupled to the front-end server via a second network connection.

15. The image positioning system of claim 14, wherein the back-end server receives the modified position from a crowdsourcing server from one or more client devices.

16. The image positioning system of claim 12, wherein each of the plurality of pose records further includes an indication of a time at which the image was obtained.

17. The image positioning system of claim 12, wherein the representation of the geographic area includes at least one of satellite imagery and a street map.

18. The image positioning system of claim 12, wherein each of the plurality of pose records includes global positioning service (GPS) coordinates to indicate the geographic locations.

19. A tangible non-transitory computer-readable medium having instructions stored thereon that, when executed by one or more processors, cause the one or more processors to:

cause a representation of a geographic area to be displayed on a display device;

determine a position of each of a plurality of pose indicators relative to the representation of the geographic area based on the geographic location data, wherein the pose indicators correspond to a single pose run and each of the plurality pose indicators corresponds to one of the plurality of images;

cause the plurality of pose indicators corresponding to the single pose run to be displayed over the representation of the geographic area on the display in accordance with the determined positions;

receive a modified position of a selected pose indicator within the displayed single pose run, wherein the modified position is modified relative to the representation of the geographic area on the display device;

determine corrected geographic location data for the image corresponding to the selected pose indicator based on the received indication of the modified position; and

modify the image pose data in accordance with the corrected geographic location data;

wherein the single pose run describes a trajectory of a device that obtained the image pose data and the corresponding plurality of images.

20. The computer-readable medium of claim 19, wherein:

the image pose data further indicates an order in which the plurality of images were obtained within the single pose run; and

the instructions further cause the one or more processors to cause a plurality of arrows to be displayed over the representation of the geographic area, wherein the plurality of arrows interconnect the plurality of pose indicators according to the order indicated in the image pose data within the single pose run.

21. The computer-readable medium of claim 19, wherein the representation of the geographic area includes satellite imagery.