System and method for producing multi-angle views of an object-of-interest from images in an image dataset

• US 20090153549A1
• Filed: 12/18/2007
• Published: 06/18/2009
• Est. Priority Date: 12/18/2007
• Status: Active Grant

• Current Assignee: HERE Global B.V. (Nokia Corporation)
• Original Assignee: Navteq North America LLC (Nokia Corporation)
• Inventors: Gale, William, Lynch, James
• Granted Patent: US 8,531,449 B2
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/532 Query formulation, e.g. graphical querying
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2200/32 involving image mosaicing
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 23/63 by using electronic viewfinders

Abstract

Disclosed are a system and method for creating multi-angle views of an object-of-interest from images stored in a dataset. A user specifies the location of an object-of-interest. As the user virtually navigates through the locality represented by the image dataset, his current virtual position is determined. Using the user'"'"'s virtual position and the location of the object-of-interest, images in the image dataset are selected and interpolated or stitched together, if necessary, to present to the user a view from his current virtual position looking toward the object-of-interest. The object-of-interest remains in the view no matter where the user virtually travels. From the same image dataset, another user can select a different object-of-interest and virtually navigate in a similar manner, with his own object-of-interest always in view. The object-of-interest also can be “virtual,” added by computer-animation techniques to the image dataset. For some image datasets, the user can virtually navigate through time as well as through space.

Claims

1. A method for using images in an image dataset to produce multi-angle views of an object-of-interest, the image dataset stored on a first computer-readable medium, the method comprising:

specifying a location of the object-of-interest;

specifying a first viewpoint location;

based, at least in part, on images read from the image dataset, producing a first view of the object-of-interest as seen from the first viewpoint location;

presenting the first view of the object-of-interest;

specifying a second viewpoint location;

based, at least in part, on images read from the image dataset, producing a second view of the object-of-interest as seen from the second viewpoint location; and

presenting the second view of the object-of-interest.

2. The method of claim 1 wherein the image dataset comprises multiple images taken from multiple points of view, at least some of which images overlap.

3. The method of claim 1 wherein the image dataset represents a locality selected from the group consisting of:

a real-world locality and a virtual-world locality.

4. The method of claim 1 wherein the image dataset comprises 3D pixels.

5. The method of claim 1 wherein the object-of-interest is selected from the group consisting of:

a structure, a landmark, a location, and a virtual structure.

6. The method of claim 1 wherein specifying a location of the object-of-interest comprises specifying a location in 3D space.

7. The method of claim 1 wherein producing the first view of the object-of-interest comprises interpolating images in the image dataset.

8. The method of claim 1 wherein producing the first view of the object-of-interest comprises stitching together images in the image dataset.

9. The method of claim 8 wherein stitching together images comprises stitching together images taken on different image-capture passes.

10. The method of claim 1 wherein presenting the first view of the object-of-interest comprises writing the first view to a second computer-readable medium.

11. The method of claim 10 wherein the first and the second computer-readable media are the same medium.

12. The method of claim 1 wherein presenting the first view of the object-of-interest comprises displaying the first view on a user interface of a computing platform.

13. The method of claim 1 wherein the image dataset comprises a time coordinate for at least some images in the image dataset, the method further comprising:

specifying a first time associated with the first viewpoint location;

wherein producing a first view of the object-of-interest as seen from the first viewpoint location is based, at least in part, on images read from the image dataset whose time coordinates are correlated to the first time associated with the first viewpoint location.

14. The method of claim 13 further comprising:

specifying a second time associated with the second viewpoint location;

wherein producing a second view of the object-of-interest as seen from the second viewpoint location is based, at least in part, on images read from the image dataset whose time coordinates are correlated to the second time associated with the second viewpoint location.

15. The method of claim 1 wherein the image dataset comprises data representing a first object and a second object, the first object at least partially occluding the second object from at least one point of view, the method further comprising:

determining relative positions of the first and second objects; and

based, at least in part, on the relative positions of the first and second objects, transforming the image dataset to reveal the second object.

16. The method of claim 15 wherein transforming the image dataset comprises applying a technique selected from the group consisting of:

deleting at least some of the data representing the first object, flagging at least some of the data representing the first object, rendering at least some of the first object semi-transparent, interpolating 3D pixels in the second object, and interpolating a texture of an object adjacent to the second object.

17. A computer-readable medium containing computer-executable instructions for performing a method for using images in an image dataset to produce multi-angle views of an object-of-interest, the image dataset stored on a first computer-readable medium, the method comprising:

specifying a location of the object-of-interest;

specifying a first viewpoint location;

based, at least in part, on images read from the image dataset, producing a first view of the object-of-interest as seen from the first viewpoint location;

presenting the first view of the object-of-interest;

specifying a second viewpoint location;

based, at least in part, on images read from the image dataset, producing a second view of the object-of-interest as seen from the second viewpoint location; and

presenting the second view of the object-of-interest.

18. A system for using images in an image dataset to produce multi-angle views of an object-of-interest, the system comprising:

a first computer-readable medium containing the image dataset;

a program for specifying a location of an object-of-interest;

a program for specifying first and second viewpoint locations;

a program for producing, based, at least in part, on images read from the image dataset, first and second views of the object-of-interest as seen, respectively, from the first and second viewpoint locations; and

a program for presenting the first and second views of the object-of-interest.

19. The system of claim 18 wherein the image dataset comprises multiple images taken from multiple points of view, at least some of which images overlap.

20. The system of claim 18 wherein the image dataset represents a locality selected from the group consisting of:

a real-world locality and a virtual-world locality.

21. The system of claim 18 wherein the image dataset comprises 3D pixels.

22. The system of claim 18 wherein producing first and second views of the object-of-interest comprises interpolating images in the image dataset.

23. The system of claim 18 wherein producing first and second views of the object-of-interest comprises stitching together images in the image dataset.

24. The system of claim 23 wherein stitching together images comprises stitching together images taken on different image-capture passes.

25. The system of claim 18 further comprising a location-finder for determining the first and second viewpoint locations.

26. The system of claim 25 further comprising a program associated with the location-finder, the associated program selected from the group consisting of:

an image-based navigation application, a route calculator, a route guidance application, a destination selector, electronic yellow pages, a vehicle-positioning application, and a map display application.

27. The system of claim 18 further comprising a second computer-readable medium for containing the produced first and second views of the object-of-interest.

28. The system of claim 18 further comprising a user interface of a computing platform for displaying the first and second views of the object-of-interest.

29. The system of claim 18 wherein the image dataset comprises a time coordinate for at least some images in the image dataset, the system further comprising:

a program for specifying a first time associated with the first viewpoint location;

wherein producing a first view of the object-of-interest as seen from the first viewpoint location is based, at least in part, on images read from the image dataset whose time coordinates are correlated to the first time associated with the first viewpoint location.

30. The system of claim 18 further comprising:

a program for determining relative positions of objects represented by data in the image dataset;

a program for determining whether a first object in the image dataset at least partially occludes a second object from at least one point of view; and

a program for transforming the image dataset to reveal the at least partially occluded object.