System and method for producing multi-angle views of an object-of-interest from images in an image dataset

• US 9,251,174 B2
• Filed: 07/30/2013
• Issued: 02/02/2016
• Est. Priority Date: 12/18/2007
• Status: Active Grant

• Current Assignee: HERE Global B.V. (Nokia Corporation)
• Original Assignee: HERE Global B.V. (Nokia Corporation)
• Inventors: Lynch, James, Gale, William
• Primary Examiner(s): BROOME, SAID A
• Application Number:
  US13/954,612
• Publication Number:
  US 20130318078A1
• Time in Patent Office: 917 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 23/63 by using electronic viewfinders

Abstract

Disclosed are a system and method for creating multi-angle views of an object-of-interest from images stored in a dataset. A user specifies the location of an object-of-interest. As the user virtually navigates through the locality represented by the image dataset, his current virtual position is determined. Using the user'"'"'s virtual position and the location of the object-of-interest, images in the image dataset are selected and interpolated or stitched together, if necessary, to present to the user a view from his current virtual position looking toward the object-of-interest. The object-of-interest remains in the view no matter where the user virtually travels. From the same image dataset, another user can select a different object-of-interest and virtually navigate in a similar manner, with his own object-of-interest always in view. The object-of-interest also can be “virtual,” added by computer-animation techniques to the image dataset. For some image datasets, the user can virtually navigate through time as well as through space.

Claims

1. A method comprising:

receiving image data from an image capture device, wherein the image data includes a first photographic image and a second photographic image captured after the first photographic image;

receiving distance data from a range-finding device;

associating a plurality of pixels of the image data with the distance data;

transforming, by a processor, the first photographic image using a portion of the image data from the second photographic image according to the received distance data;

wherein the second photographic image includes a virtual object, and wherein transforming the first photographic image comprises adding the virtual object from the second photographic image to test visibility of the virtual object from a perspective of the first photographic image; and

displaying the transformed image to a user to virtually navigate the image.

2. The method of claim 1, further comprising:

evaluating visual objects in the first photographic image; and

identifying an object of concern from the visual objects.

3. The method of claim 2, further comprising:

removing the object of concern from the first photographic image,

wherein transforming the first photographic image comprises adding the portion of the image data from the second photographic image to replace the object of concern in the first photographic image.

4. The method of claim 1, wherein the virtual object is a proposed street sign.

5. The method of claim 1, wherein the portion of the image data from the second photographic image includes an at least partially occluded object in the first photographic image.

6. A non-transitory computer-readable medium containing computer-executable instructions for performing a method comprising:

receiving a first viewpoint;

receiving a second viewpoint;

selecting image data including a first image in a perspective of the first viewpoint and a second image in a perspective of the second viewpoint, wherein the first image was captured before the second image;

receiving distance data associated with the image data; and

transforming the first image in the perspective of the first viewpoint and the second image in the perspective of the second viewpoint according to the received distance data associated with the image data;

wherein the second image includes a virtual object, and wherein transforming the first image comprises adding the virtual object from the second image to test visibility of the virtual object from a perspective of the first image; and

displaying the transformed image to a user to virtually navigate the image.

7. The non-transitory computer-readable medium containing computer-executable instructions of claim 6, the method comprising:

generating a third image including an object of interest included in the first image and included in the second image, wherein the third image is from a third viewpoint location.

8. The non-transitory computer-readable medium containing computer-executable instructions of claim 6, wherein the distance data is generated by a laser range finding device.