Method, System, and Computer-Readable Data Storage Device for Creating and Displaying Three-Dimensional Features on an Electronic Map Display

• US 20120274625A1
• Filed: 04/26/2011
• Published: 11/01/2012
• Est. Priority Date: 04/26/2011
• Status: Active Grant

• Current Assignee: HERE Global B.V. (HERE Holding Corporation)
• Original Assignee: HERE Global B.V. (HERE Holding Corporation)
• Inventors: Lynch, James D.
• Granted Patent: US 9,196,086 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G01C 21/3638 including 3D objects and buildings three dimensional [3D] modelling, e.g. data description of 3D objects G06T17/00; geographic models G06T17/05
• CPC Class Codes: G01C 21/3852 Data derived from aerial or satellite images
• CPC Class Codes: G01C 21/3867 Geometry of map features, e.g. shape points, polygons or for simplified maps
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2210/61 Scene description
• CPC Class Codes: G09B 29/007 using computer methods
• CPC Class Codes: G09B 29/12 Relief maps relief models G09B25/06

Abstract

Methods, systems, and computer-readable data storage devices for generating and/or displaying a map with three-dimensional (3D) features are disclosed. For example, a method may comprise (i) defining a plurality of major three-dimensional regions (“major 3DRs”) and associating each major 3DR with a respective geographical area defined for a map stored in a computer-readable map database, and (ii) displaying, via a display device, one or more of the major 3DRs upon the map. Each major 3DR comprises a top, a bottom, and multiple sides. Each top, bottom, and side of each major 3DR comprises at least one surface. At least one surface of each major 3DR being displayed is textured with an image captured via an imaging device. The image textured onto each surface comprises an image captured by the imaging device when capturing images in a direction of that surface.

Claims

1. A method for displaying a map with three-dimensional features, the method comprising:

defining a plurality of major three-dimensional regions (major 3DRs) and associating each major 3DR with a respective geographical area defined for a map stored in a computer-readable map database, wherein each major 3DR comprises a top, a bottom, and multiple sides, and wherein each top, bottom, and side of each major 3DR comprises at least one surface; and

displaying, via a display device, one or more of the major 3DRs upon the map, wherein at least one surface of each major 3DR being displayed is textured with an image captured via an imaging device, and wherein the image textured onto each surface comprises an image captured by the imaging device when capturing images in a direction of that surface.

2. The method of claim 1, further comprising:

associating one or more sets of data points with a first major 3DR of the plurality of major 3DRs, wherein each set of data points associated with the first major 3DR indicates presence of one or more objects at points in space on or above the geographical area associated with the first major 3DR;

defining one or more minor three-dimensional regions (minor 3DRs) for the first major 3DR, wherein each minor 3DR defined for the first major 3DR pertains to a portion of the first major 3DR at which a density of data points of a set of data points associated with the first major 3DR equals or exceeds a density threshold, wherein each minor 3DR defined for the first major 3DR comprises at least a top, a bottom, and multiple sides; and

displaying, via the display device, the one or more minor 3DRs defined for the first major 3DR, wherein at least the top, the bottom, or one of the multiple sides of each minor 3DR, defined for the first major 3DR, being displayed has one or more images textured onto that top, bottom, or side, and wherein the one or more images textured onto each top, bottom, or side of a minor 3DR are images that are visible from a location where the image was captured when looking in the direction of that top, bottom, or side.

3. The method of claim 2, wherein at least a portion of the data points within the one or more sets of data points are generated by a LIDAR device, a three-dimensional stereo camera imaging device, or a radar device.

4. The method of claim 2, wherein each surface of any major 3DR being displayed via the display device and each surface of any minor 3DR being displayed via the display device is outlined with a wireframe.

5. The method of claim 1, wherein the top of at least one major 3DR of the plurality of major 3DRs comprises a top inner surface and a top outer surface, and each of the multiple sides of the at least one major 3DR comprises a respective inner side surface and a respective outer side surface.

6. The method of claim 5, further comprising:

receiving an input to trigger the display device from displaying at least one of the inner surfaces of the first major 3DR to displaying at least one of the outer surfaces of the first major 3DR,

wherein, while the display device displays the at least one of the inner surfaces of the first major 3DR, the display device does not display any of the outer surfaces of the first major 3DR, and while the display device displays the at least one of the outer surfaces of the first major 3DR, the display device does not display any of the inner surfaces of the first major 3DR.

7. The method of claim 5, further comprising:

receiving an input to trigger the display device from displaying at least one of the outer surfaces of the first major 3DR to displaying at least one of the inner surfaces of the first major 3DR,

wherein, while the display device displays the at least one of the inner surfaces of the first major 3DR, the display device does not display any of the outer surfaces of the first major 3DR, and while the display device displays the at least one of the outer surfaces of the first major 3DR, the display device does not display any of the inner surfaces of the first major 3DR.

8. The method of claim 1, wherein at least one major 3DR of the plurality of major 3DRs comprises a rectangular prism.

9. A computer-readable data storage device for use in creating a map with three-dimensional features, the data storage device comprising:

a set of object-detection data generated for each of one or more locations, wherein each set of object-detection data comprises a plurality of data points and each data point is associated with a particular location remote from the location for which the set of object-detection data was generated;

a plurality of images and location data that identifies a viewpoint and location from which each image was captured;

computer-readable program instructions executable by a processor to identify, from among the object-detection data, data points that indicate presence of a three-dimensional (3D) object at a particular location and to define a 3D region associated with the 3D object and the particular location, wherein the 3D region comprises a plurality of surfaces;

computer-readable program instructions executable by the processor to determine, for each surface of the 3D region and from among the plurality of images, a respective image to be associated with that surface; and

computer-readable program instructions executable by the processor to create, within the data storage device, association data that identifies each association between each surface of the 3D region and the respective image associated with that surface.

10. The computer-readable data storage device of claim 9,

wherein the data points that indicate presence of a 3D object are generated by a data point device located on a vehicle while the vehicle is at a first location,

wherein at least one image associated with a surface of the 3D region comprises an image captured by an image capturing device located on the vehicle while the vehicle is at a second location, and

wherein at least one other image associated with a surface of the 3D region comprises an image capture by the image capturing device while the vehicle is at a third location.

11. The computer-readable data storage device of claim 9,

wherein the data points that indicate presence of a 3D object are generated by a data point device located on a vehicle while the vehicle is at a first location,

wherein at least one image associated with a surface of the 3D region comprises an image captured by an image capturing device located on the vehicle while the vehicle is at the first location, and

wherein at least one other image associated with a surface of the 3D region comprises an image capture by the image capturing device while the vehicle is at a second location.

12. The computer-readable data storage device of claim 11,

wherein the data points that indicate presence of a 3D object comprise LIDAR data points.

13. The computer-readable data storage device of claim 9,

wherein the data points that indicate presence of a 3D object are generated by a data point device located on a vehicle while the vehicle is at a first location, and

wherein each image associated with a surface of the 3D region comprises an image captured by an image capturing device located on the vehicle while the vehicle is at the first location.

14. The computer-readable data storage device of claim 9,

wherein the data points that indicate presence of a 3D object are generated by a data point device located on a first vehicle, and

wherein at least one image associated with a surface of the 3D region comprises an image captured by an image capturing device located on a second vehicle.

15. The computer-readable data storage device of claim 9, wherein the stored images captured from any given location are arranged as an image bubble covering 360 degrees of viewpoints.

16. The computer-readable data storage device of claim 9, further comprising:

computer-readable program instructions executable by the processor to refer to the association data to identify the respective image associated with a surface of the 3D region when that 3D region is within an area to be displayed on the map, and

computer-readable program instructions that cause the surface of the 3D region to be textured with the respective image when that surface of the 3D region is displayed on the map.

17. A system for displaying a map with three-dimensional features, the system comprising:

a display device;

a user-input device;

a computer-readable data storage device; and

a processor that is operable to execute computer-readable program instructions stored at the computer-readable data storage device,

wherein the computer-readable data storage device comprises data that identifies a plurality of major three-dimensional regions (major 3DRs), each major 3DR comprising a top, a bottom, and multiple sides,

wherein the computer-readable data storage device comprises, for each identified major 3DR, data that identifies at least one respective surface for each side, top, and bottom of that major 3DR,

wherein the computer-readable data storage device comprises a plurality of images,

wherein at least a portion of each image is associated with a surface defined for a major 3DR,

wherein the computer-readable program instructions comprise program instructions executable to cause the display device to display at least one major 3DR in response to selection of a geographical location via the user-input device, and

wherein, when a given major 3DR is being displayed, each visible surface of the given major 3DR is textured with the image associated with that surface.

18. The system of claim 17,

wherein the computer-readable data storage device comprises data that identifies at least one minor three-dimensional region (minor 3DR), each minor 3DR comprising a top, a bottom, and multiple sides, and having at least one respective surface defined for the top, the bottom, and each side of that minor 3DR,

wherein the computer-readable data storage device comprises data that associates each defined minor 3DR with a major 3DR of the plurality of major 3DRs,

wherein the computer-readable program instructions comprise program instructions executable to cause the display device to transition from displaying the given major 3DR to displaying each minor 3DR that is associated with the given major 3DR, and

wherein, when a given minor 3DR is being displayed, each visible surface of the given minor 3DR is textured with the image associated with that surface.

19. The system of claim 18,

wherein the data that identifies at least one minor 3DR comprises data that identifies multiple levels of minor 3DRs for the given minor 3DR, and

wherein each level of minor 3DRs for the given minor 3DR that is lower in resolution than another level of minor 3DR for the given minor 3DR comprises fewer defined three-dimensional regions than a number of defined three-dimensional regions for that other level of minor 3DR for the given minor 3DR.

20. The system of claim 19,

wherein each three-dimensional region defined for any level of minor 3DRS for the given minor 3DR comprises a top, a bottom, and multiple sides, and has at least one surface defined for the top, the bottom, and each side of that three-dimensional region,

wherein the computer-readable program instructions comprise program instructions executable to cause the display device to transition from displaying the given minor 3DR to a higher-resolution level of the given minor 3DR, and

wherein, when a given minor 3DR is being displayed, each visible surface of the given minor 3DR is textured with the image associated with that surface.

21. The system of claim 18, further comprising:

a communications interface that is operable to receive data points associated with the given minor 3DR,

wherein the data points represent at least a portion of an object located at a location associated with the given minor 3DR,

22. The system of claim 21, wherein the data points associated with the given minor 3DR comprise Light Detection and Ranging (LIDAR) data points generated by a LIDAR device.

23. The system of claim 17,

wherein the plurality of images comprises satellite imagery captured by a satellite,

wherein the display device displays at least one satellite image, associated with the geographical location selection entered via the user-input device, as a base image, and

wherein the at least one major 3DR displayed via the display device covers at least a portion of the base image.

24. The system of claim 17, wherein the plurality of images comprises images arranged as an image bubble having 360 degrees of images captured at a given location.

25. A method for displaying a map with three-dimensional features, the method comprising:

storing, within a non-transitory computer-readable data storage device, data that identifies a major three-dimensional region (major 3DR) and data that identifies at least a portion of an image that is associated with at least one surface defined for the three-dimensional region, wherein the major 3DR comprises a top, a bottom, and multiple sides, and wherein each top, bottom, and side of each major 3DR comprises at least one surface;

receiving computer-readable data indicating a selection of a given geographical location and responsively executing computer-readable program instructions to determine that the major 3DR is associated with a geographic area that includes the given geographical location; and

providing data to display a map that depicts the geographical area and the major 3DR associated with the geographical area, wherein the at least one surface defined for the three-dimensional region is textured with the at least a portion of the image that is associated with that surface.