Path side imagery

• US 9,404,764 B2
• Filed: 12/30/2011
• Issued: 08/02/2016
• Est. Priority Date: 12/30/2011
• Status: Active Grant

• Current Assignee: HERE Global B.V. (HERE Holding Corporation)
• Original Assignee: HERE Global B.V. (HERE Holding Corporation)
• Inventors: Lynch, James D.
• Primary Examiner(s): McCulley, Ryan D
• Application Number:
  US13/340,923
• Publication Number:
  US 20130169668A1
• Time in Patent Office: 1,677 Days
• Field of Search: 345/419
• US Class Current: 1/1
• CPC Class Codes: G01C 21/3638 including 3D objects and buildings three dimensional [3D] modelling, e.g. data description of 3D objects G06T17/00; geographic models G06T17/05
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G01S 5/16 using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 7/4808 Evaluating distance, position or velocity data
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 3/08 Projecting images onto non-planar surfaces, e.g. geodetic screens

Abstract

One or more systems, devices, and/or methods for generating path side imagery are disclosed. For example, a method includes receiving optical data measured in three-dimensional geographic coordinates and image data from at least one image bubble. The optical data is projected onto at least one predefined two-dimensional plane. A pixel value for a point of the optical data on the two-dimensional plane is calculated according to the image data. The pixel value may correspond to color or another graphical attribute. The two-dimensional plane is stored or displayed as street side imagery or other path side imagery. An image file including the pixel value for each point of the optical data may be stored in memory or a map database.

Claims

1. A method comprising:

receiving optical data measured in three-dimensional geographic coordinates using an optical distancing system and collected along a path;

projecting, using a controller, the optical data onto three or more predefined two-dimensional planes, wherein the three or more predefined two-dimensional planes are defined by different horizontal distance ranges from the path;

receiving image data from at least one panoramic image collected along the path, the image data including spherical coordinates;

mapping, using the controller, the spherical coordinates of the image data to the geographical coordinate space of the optical data;

determining, using the controller, a pixel value for a point of the optical data on the three or more predefined two-dimensional planes by selecting a pixel in the mapped image data having a three-dimensional geographic coordinate corresponding to the three-dimensional geographic coordinate of the point of the optical data;

defining three or more images, wherein each image includes pixel values for one of the three or more predefined two-dimensional planes;

selecting two or more of the three or more images; and

providing the two or more images to a map application, wherein the map application overlays the two or more images to create a pathway view.

2. The method of claim 1, wherein

selecting the two or more images is based on a configuration setting including data indicative of the distance ranges for the predefined two-dimensional planes.

3. The method of claim 1, wherein

selecting the two or more images is based on an analysis of the optical data.

4. The method of claim 1, further comprising:

detecting a navigation feature in the optical data, wherein

selecting the two or more images is based on the navigation feature.

5. The method of claim 4, wherein the navigation feature is at least one of a road surface, a curb, a storefront, a door, a doorway, a window, or a building facade.

6. The method of claim 1, wherein determining the pixel value for the point of the optical data on the three or more predefined two-dimensional planes according to the image data further comprises:

selecting a plurality of pixel values in the image data based on a density of the optical data in an area surrounding the point of the optical data; and

averaging a plurality of pixel values in the image data to calculate the pixel value for the point of the optical data.

7. The method of claim 1, wherein determining the pixel value for the point of the optical data on the three or more predefined two-dimensional planes according to the image data further comprises:

averaging a plurality of pixel values across a plurality of panoramic images.

8. The method of claim 7, further comprising:

identifying an obstruction in at least one of the plurality of panoramic images corresponding with the point in the optical data; and

omitting the at least one of the plurality of panoramic images having the obstruction in the calculating of the pixel value for the point of the optical data.

9. The method of claim 1, wherein the optical distancing system is a light detection and ranging (LIDAR) system.

10. The method of claim 1, wherein as the map application scrolls along the pathway view, the selected two or more images scroll at different speeds.

11. An apparatus comprising:

a memory configured to store optical data measured in three-dimensional geographic coordinates and generated by an optical distancing system along a road and configured to store image data measured in spherical coordinates from at least one image bubble; and

a controller configured to project the optical data onto a plurality of predefined two-dimensional planes according to horizontal distance ranges from the road and configured to map the spherical coordinates of the image data to the geographic coordinates of the optical data using a vector based function, and configured to calculate a pixel value for a point of the optical data on the plurality of predefined two-dimensional planes by selecting a pixel in the mapped image data having a three-dimensional geographic coordinate corresponding to the three-dimensional geographic coordinate of the point of the optical data,

wherein each of the plurality of predefined two-dimensional planes corresponds to one of a plurality of layers of a street side imagery, and wherein a command further indicates which of the plurality of layers is to be displayed with the street side imagery, wherein as the street side imagery is scrolled, each layer of the plurality of layers scrolls at a different speed.

12. The apparatus of claim 11, wherein the controller is configured to associate an image file including the plurality of predefined two-dimensional planes with a navigation segment.

13. The apparatus of claim 12, further comprising:

an input device configured to receive the command, wherein the command is indicative of the navigation segment; and

a display configured to display the street side imagery from the image file in response to the command.

14. The apparatus of claim 11, wherein the plurality of layers includes a plurality of layers defined by a respective distance range determined from an analysis of the optical data.

15. The apparatus of claim 11, wherein the optical data is generated from a light detection and ranging (LIDAR) system or a system that emits structured infrared light.

16. A non-transitory computer readable medium having stored thereon a computer program, the computer program comprising instructions to cause a processor to:

receive optical data measured in three-dimensional geographic coordinates and generated by an optical distancing system along a road;

project the optical data onto three or more predefined two-dimensional planes defined by horizontal distance ranges from the road;

receive first image data from a first captured image bubble, the first image data containing spherical coordinates;

receive second image data from a second captured image bubble, the second image data containing spherical coordinates;

map the spherical coordinates of the first image data to the geographical coordinate space of the optical data;

map the spherical coordinates of the second image data to the geographical coordinate space of the optical data;

select pixel values for the optical data on the three or more predefined two-dimensional planes by selecting a pixel in the mapped image data having a three-dimensional geographic coordinate corresponding to a three-dimensional geographic coordinate of a point of the optical data, wherein when there is no mapped first or second image data for a pixel, a diffusion technique is used to select the pixel value;

define three or more images, wherein each image includes the pixel values for one of the three or more predefined two-dimensional planes;

select two or more images from the three or more images; and

provide the two or more images to a map application, wherein the map application overlays the two or more images to create a pathway view, wherein as the map application scrolls along the pathway view, the two or more images scroll at different speeds.

17. The non-transitory computer readable medium of claim 16, the program further comprising instructions to:

output a subset of the three or more predefined two-dimensional planes based on a statistical analysis of the optical data or a user selection.

18. The non-transitory computer readable medium of claim 16, the program further comprising instructions to:

average a plurality of pixel values across a plurality of image bubbles;

identify an obstruction in at least one of the plurality of image bubbles corresponding with the point in the optical data; and

omit the at least one of the plurality of image bubbles having the obstruction in calculation of the pixel value for the point of the optical data.