Path side image in map overlay

• US 10,235,787 B2
• Filed: 01/09/2017
• Issued: 03/19/2019
• Est. Priority Date: 12/30/2011
• Status: Active Grant

• Current Assignee: HERE Global B.V. (Nokia Corporation)
• Original Assignee: HERE Global B.V. (Nokia Corporation)
• Inventors: Lynch, James
• Primary Examiner(s): McCulley, Ryan
• Application Number:
  US15/401,873
• Publication Number:
  US 20170124749A1
• Time in Patent Office: 799 Days
• Field of Search: None
• CPC Class Codes: G01C 21/3635 Guidance using 3D or perspective road maps
• CPC Class Codes: G01C 21/3647 Guidance involving output of stored or live camera images or video streams
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 3/60 Rotation of whole images or parts thereof
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light

Abstract

One or more systems, devices, and/or methods for generating a map including path side data include storing path side data referenced to three-dimensional geographic coordinates. The path side data may be optical data or optical data modified based on one or more panoramic images. The path side data is combined with map data received from a map database. The map data includes nodes and segments. A processor rotates the path side data based on one of the segments. The rotation may be about the segment or about a featured identified in the optical data. The path side data overlaid on the map data is outputted to a display, a file, or another device.

Claims

1. A method comprising:

receiving, from a light detection device, optical distance data corresponding to a pathway;

receiving data for an image bubble including a view from the pathway;

aligning the image bubble with the optical distance data;

identifying a plurality of pixels from the image bubble aligned with the optical distance data;

calculating a rotation point (F) according to;

F=C+W*N,

wherein C represents a center road point, W represents a road width and N represents a unit vector normal to a direction of the pathway; and

rotating, about the rotation point (F) based on a road segment for the pathway, the plurality of pixels correlated with the optical distance data to a map plane using the optical distance data.

2. The method of claim 1, further comprising:

displaying a map and the plurality of pixels in the map plane.

3. The method of claim 1, further comprising:

dividing the optical distance data into a plurality of layers defined by distance to the road segment.

4. The method of claim 3, further comprising:

selecting at least one of the plurality of layers according to a user input; and

displaying the at least one of the plurality of layers and a portion of the map data.

5. The method of claim 1, further comprising:

accessing map data for the map plane from a map database, wherein the map data includes the road segment.

6. The method of claim 1, further comprising:

averaging pixel values for a section of the image bubble; and

defining a new value for the optical data based on the averaged pixel values for the section of the image bubble.

7. The method of claim 1, further comprising:

weighting pixel values according to distance; and

defining a new value for the optical data based on the weighted pixel values.

8. The method claim 1, wherein the optical distance data is collected by a light detection and ranging (LIDAR) system.

9. The method of claim 1, further comprising:

filtering the optical distance data according to reflectivity of the optical distance data.

10. An apparatus comprising:

at least one processor; and

at least one memory including computer program code for one or more programs;

the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least perform;

receive, from a light detection device, optical distance data corresponding to a road segment;

receive image data from at least one image bubble corresponding to a view from the road segment;

modify the optical distance data based on the at least one image bubble;

calculate a rotation point (F) according to;

F=C+W*N,

wherein C represents a center of the road segment, W represents a width of the road segment and N represents a unit vector normal to a direction of the road segment; and

rotate the optical distance data according to rotation point (F) to a map plane including the road segment.

11. The apparatus of claim 10, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to:

display a map and the image data in the map plane.

12. The apparatus of claim 10, wherein the modified optical distance data are rotated about the road segment.

13. The apparatus of claim 10, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to:

average pixel values from the image bubble; and

define colorized optical distance data based on the averaged pixel values.

14. The apparatus of claim 10, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to:

weight pixel values from the image bubble according to distance; and

define colorized optical distance data based on the weighted pixel value.

15. The apparatus of claim 14, wherein the colorized optical distance data is rotated using a rotation matrix.

16. The apparatus of claim 10, wherein the optical distance data is collected by a light detection and ranging (LIDAR) system including the light detection device.

17. The apparatus of claim 10, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to:

divide the optical distance data into a plurality of layers defined by distance to the road segment.

18. The apparatus of claim 17, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to:

select at least one of the plurality of layers according to a user input; and

display the at least one of the plurality of layers and a portion of the map data.

19. A non-transitory computer readable medium having stored thereon a computer program, the computer program comprising instructions to cause a processor to:

detect optical distance data corresponding to a pathway using a light detection device;

identify image data from at least one image bubble corresponding to a view from the pathway;

associate pixels of the at least one image bubble to the optical distance data;

calculate a rotation point (F) according to;

F=C+W*N,

wherein C represents a center of a road segment for the pathway, W represents a width for the road segment and N represents a unit vector normal to a direction of the road segment; and

rotate, using the rotation point (F), pixel values correlated with the optical distance data to a map plane.