Reimaging based on depthmap information

• US 8,553,942 B2
• Filed: 10/21/2011
• Issued: 10/08/2013
• Est. Priority Date: 10/21/2011
• Status: Active Grant

• Current Assignee: HERE Global B.V. (Nokia Corporation)
• Original Assignee: Navteq B.V.
• Inventors: Lynch, James D.
• Primary Examiner(s): Lu, Tom Y
• Application Number:
  US13/278,499
• Publication Number:
  US 20130101175A1
• Time in Patent Office: 718 Days
• Field of Search: 382/106, 382/154
• US Class Current: 382/106
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G01C 21/3638 including 3D objects and buildings three dimensional [3D] modelling, e.g. data description of 3D objects G06T17/00; geographic models G06T17/05
• CPC Class Codes: G01C 3/00 Measuring distances in line of sight; Optical rangefinders tapes, chains or wheels for measuring length G01B3/00; active triangulation systems, i.e. using the transmission and reflection of electromagnetic waves other than radio waves, G01S17/48
• CPC Class Codes: G06T 17/05 Geographic models
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: H04N 23/698 for achieving an enlarged field of view, e.g. panoramic image capture

Abstract

One or more systems, devices, and/or methods for emphasizing objects in an image, such as a panoramic image, are disclosed. For example, a method includes receiving a depthmap generated from an optical distancing system, wherein the depthmap includes position data and depth data for each of a plurality of points. The optical distancing system measures physical data. The depthmap is overlaid on the panoramic image according to the position data. Data is received that indicates a location on the panoramic image and, accordingly, a first point of the plurality of points that is associated with the location. The depth data of the first point is compared to depth data of surrounding points to identify an area on the panoramic image corresponding to a subset of the surrounding points. The panoramic image is altered with a graphical effect that indicates the location.

Claims

1. A method for reimaging objects in a panoramic image, the method comprising:

receiving a depthmap generated from an optical distancing system, wherein the depthmap includes depth data for each of a plurality of points;

overlaying the depthmap on the panoramic image according to the plurality of points;

receiving data indicative of a location on the panoramic image and indicative of a first point of the plurality of points;

comparing depth data of the first point to depth data of surrounding points through the calculation of a difference between the depth data of each of the surrounding points and the depth data of the first point to determine a gradient quantity;

comparing the gradient quantity to a predetermined value;

identifying surrounding points for which the predetermined value exceeds the gradient quantity as an area on the panoramic image corresponding to a subset of the surrounding points; and

altering the panoramic image with a graphical effect at one or more of the surrounding points.

2. The method of claim 1, wherein each of the plurality of points comprises two-dimensional position data.

3. The method of claim 1, wherein the graphical effect blurs portions of the panoramic image, inverts portions of the panoramic image, outlines portions of the panoramic image, obscures portions of the panoramic image, fades portions of the panoramic image, cuts pixels from portions of the panoramic image, or removes portions of the panoramic image.

4. The method of claim 1, wherein receiving data indicative of a location on the panoramic image comprises receiving a user input of the location on the panoramic image.

5. The method of claim 1, wherein receiving data indicative of a location on the panoramic image comprises receiving a map object associated with a route, a point of interest, a destination, or a cue.

6. The method of claim 1, wherein the optical distancing system is a light detection and ranging (LIDAR) system.

7. The method of claim 1, wherein the optical distancing system is a two dimensional stereo camera that collects the panoramic image and the depthmap.

8. The method of claim 1, wherein the graphical effect is a simulated spotlight, wherein an intensity of each of the surrounding points is a function of a the depth data of the surrounding points and position data associated with the surrounding points.

9. The method of claim 1, wherein the optical distancing system emits structured infrared light.

10. An apparatus for reimaging objects in an image, the apparatus comprising:

a memory configured to store a depthmap overlaid on the image, wherein the depthmap is generated from an optical distancing system and includes depth data for each of plurality of points; and

a processor configured to receive a location on the image and configured to identify a focus area on the image corresponding to the location, wherein the focus area is defined according to a function of depth data of the location and depth data of points surrounding the location, wherein the image is altered with a graphical effect at the focus area,

wherein the function uses a rate of change of the depth data in a direction moving away from the location to define the focus area where the rate of change is less than a predetermined rate.

11. The apparatus of claim 10, wherein the graphical effect is an outline of a physical object, a blurring of the focus area, a color inversion of the focus area, a fading of the focus area, or a fading of the image outside of the focus area.

12. The apparatus of claim 10, wherein the optical distancing system is a light detection and ranging (LIDAR) system.

13. The apparatus of claim 10, wherein the location is based on a perspective that defines the image.

14. The apparatus of claim 10, wherein the function defines the focus area to include points surrounding the location that have depth data within a predetermined distance from the depth data of the location.

15. A non-transitory computer readable medium configured to store a computer program configured to organize spatial data, the program comprising instructions to:

receive a depthmap generated from an optical distancing system, wherein the depthmap includes depth data for each of a plurality of points and is overlaid on an image according to the plurality of points;

receive data indicative of a location on the image and a first point of the plurality of points; and

alter pixel values in the image at a subset of the surrounding points according to a function of the depth data,

wherein the function uses a rate of change of the depth data in a direction moving away from the first point to define the subset of surrounding points where the rate of change is less than a predetermined rate.

16. The non-transitory computer readable medium of claim 15, where in the function defines the subset of the surrounding points to include points surrounding the location that have depth data within a predetermined distance from the depth data of the first point.

17. The non-transitory computer readable medium of claim 15, wherein the optical distancing system is a light detection and ranging (LIDAR) system.

18. A method comprising:

receiving a depthmap generated from an optical distancing system, wherein the depthmap includes depth data for each of a plurality of points;

associating the depthmap on a panoramic image according to the plurality of points;

receiving data indicative of a location on the panoramic image and indicative of a first point of the plurality of points;

calculating a difference between a depth of the first point and depths of surrounding points;

comparing the differences between the depth of the first point and the depths of surrounding points to a threshold value;

identifying a subset of surrounding points for which the threshold value exceeds the differences between the depth of the first point and the depths of surrounding points; and

altering the panoramic image with a graphical effect at the subset of surrounding points.

19. The method of claim 18, wherein the difference between the depth of the first point and depths of surrounding points is a gradient quantity.

20. The method of claim 18, wherein the graphical effect highlights an object corresponding to the subset of surrounding points.