LOCATION OF IMAGE CAPTURE DEVICE AND OBJECT FEATURES IN A CAPTURED IMAGE

• US 20130243250A1
• Filed: 03/14/2013
• Published: 09/19/2013
• Est. Priority Date: 09/14/2009
• Status: Active Grant

• Current Assignee: Trimble Navigation Limited (Trimble Inc.)
• Original Assignee: Trimble Navigation Limited (Trimble Inc.)
• Inventors: France, Peter G., Weisenburger, Shawn D.
• Granted Patent: US 9,324,003 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G01C 15/00 Surveying instruments or accessories not provided for in groups G01C1/00 - G01C13/00
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59

Abstract

A method for matching a region on an object of interest with a geolocation in a coordinate system is disclosed. In one embodiment, an image of a region on an object of interest is captured on an image capture device. The image is processed to detect a located feature using a feature detection algorithm. Further processing of the located feature is performed to derive a first feature descriptor using a feature descriptor extraction algorithm. The feature descriptor is stored in a memory. A database of feature descriptors having geolocation information associated with the feature descriptors is searched for a match to the first feature descriptor. The geolocation information is then made available for access.

Claims

1. A method for matching a region on an object of interest with a geolocation in a coordinate system, said method comprising:

capturing an image of a region of an object of interest using an image capture device at a first location;

processing said image to detect a located feature using a feature detection algorithm;

further processing said located feature to derive a first feature descriptor using a feature descriptor extraction algorithm;

storing said feature descriptor in a memory;

searching a database of feature descriptors having geolocation information associated with the feature descriptors, for a match to said first feature descriptor; and

making said geolocation information available for access.

2. The method of claim 1 further comprising:

sending said first feature descriptor from said image capture device to an image database; and

matching said feature descriptor with at least one feature descriptor stored at said image database, said at least one feature descriptor stored at said database having geolocation information associated therewith.

3. The method of claim 2 further comprising:

determining by said image database a geographic position of said image capture device based upon the geolocation information associated with said at least one feature descriptor;

determining by said image database an orientation of said image capture device based upon the geolocation information associated with said at least one feature descriptor; and

conveying by said image database said geographic position and said orientation to said image capture device.

4. The method of claim 2 further comprising:

sending by said image database to said image capture device the geolocation information associated with said at least one feature descriptor;

storing the geolocation information associated with said at least one feature descriptor upon a database disposed upon said image capture device;

determining by said image capture device a geographic position of said image capture device based upon the geolocation information associated with said at least one feature descriptor; and

determining by said image capture device an orientation of said image capture device based upon the geolocation information associated with said at least one feature descriptor.

5. The method of claim 2 further comprising:

sending an estimated location of said image capture device to said image database; and

selecting by said image database a subset of feature descriptors to match with said feature descriptor sent by said image capture device based upon said estimated location.

6. The method of claim 5 further comprising:

determining by said image database a geographic position of said image capture device based upon the geolocation information associated with at least one of said subset of feature descriptors;

determining by said image database an orientation of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors; and

conveying by said image database said geographic position and said orientation to said image capture device.

7. The method of claim 5 further comprising:

sending by said image database to said image capture device the geolocation information associated with at least one of said subset of feature descriptors;

storing the geolocation information associated with said at least one of said subset of feature descriptors upon a database disposed upon said image capture device;

determining by said image capture device a geographic position of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors; and

determining by said image capture device an orientation of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors.

8. The method of claim 5 further comprising:

sending an estimated orientation of said image capture device to said database; and

selecting by said image database a subset of feature descriptors to match with said feature descriptor sent by said image capture device based upon said estimated orientation, each of said subset of feature descriptors having geolocation information associated therewith.

9. The method of claim 8 further comprising:

matching said feature descriptor with at least one of said subset of feature descriptors;

determining by said image database a geographic position of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors;

determining by said image database an orientation of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors; and

conveying by said image database said geographic position and said orientation to said image capture device.

10. The method of claim 8 further comprising:

sending by said image database to said image capture device the geolocation information associated with at least one of said subset of feature descriptors;

storing the geolocation information associated with said at least one of said subset of feature descriptors upon a database disposed upon said image capture device;

determining by said image capture device a geographic position of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors; and

determining by said image capture device an orientation of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors.

11. The method of claim 1 further comprising:

sending an estimated location of said image capture device to an image database; and

selecting by said image database a subset of feature descriptors to match with said feature descriptor sent by said image capture device based upon said estimated location, each of said subset of feature descriptors having geolocation information associated therewith.

12. The method of claim 11 further comprising:

determining by said image database a geographic position of said image capture device based upon the geolocation information associated with at least one of said subset of feature descriptors;

determining by said image database an orientation of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors; and

conveying by said image database said geographic position and said orientation to said image capture device.

13. The method of claim 11 further comprising:

sending by said image database to said image capture device the geolocation information associated with at least one of said subset of feature descriptors;

storing the geolocation information associated with said at least one of said subset of feature descriptors upon a database disposed upon said image capture device;

determining by said image capture device a geographic position of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors; and

determining by said image capture device an orientation of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors.

14. The method of claim 11 further comprising:

sending an estimated orientation of said image capture device to said database; and

selecting by said image database a subset of feature descriptors to match with said feature descriptor sent by said image capture device based upon said estimated orientation, each of said subset of feature descriptors having geolocation information associated therewith.

15. The method of claim 14 further comprising:

matching said feature descriptor with at least one of said subset of feature descriptors;

determining by said image database a geographic position of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors;

determining by said image database an orientation of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors; and

conveying by said image database said geographic position and said orientation to said image capture device.

16. The method of claim 14 further comprising:

sending by said image database to said image capture device the geolocation information associated with at least one of said subset of feature descriptors;

storing the geolocation information associated with said at least one of said subset of feature descriptors upon a database disposed upon said image capture device;

determining by said image capture device a geographic position of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors; and

determining by said image capture device an orientation of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors.

17. The method of claim 1, wherein said image further comprises a second object of interest, said method further comprising:

capturing a second image of said object of interest and a second object of interest at a second location;

processing said second image to detect said located feature using said feature detection algorithm;

further processing said located feature of said second image to derive said first feature descriptor using said feature descriptor extraction algorithm;

determining a geographic position and an orientation of said image capture device at said first location and a second geographic position and a second orientation of an image capture device at said second location; and

using a bundle adjustment algorithm to calculate a geographic position of said second object of interest based upon said geographic position and said orientation of said image capture device at said first location and said second geographic position and said second orientation of an image capture device at said second location.

18. The method of claim 1 further comprising:

using said geolocation information for a plurality of points of interest to derive geolocation information for a second object of interest which is co-planar with said plurality of points of interest.

19. A non-transitory computer-readable storage medium comprising computer executable code for directing a processor to executed a method for matching a region on an object of interest with a geolocation in a coordinate system, said method comprising:

capturing an image of a region of an object of interest using an image capture device at a first location;

processing said image to detect a located feature using a feature detection algorithm;

further processing said located feature to derive a first feature descriptor using a feature descriptor extraction algorithm;

storing said feature descriptor in a memory;

searching a database of feature descriptors having geolocation information associated with the feature descriptors, for a match to said first feature descriptor; and

making said geolocation information available for access.

20. The non-transitory computer-readable storage medium of claim 19, wherein said method further comprises:

sending said first feature descriptor from said image capture device to an image database; and

matching said feature descriptor with at least one feature descriptor stored at said image database, said at least one feature descriptor stored at said database having geolocation information associated therewith.

21. The non-transitory computer-readable storage medium of claim 20, wherein said method further comprises:

determining by said image database a geographic position of said image capture device based upon the geolocation information associated with said at least one feature descriptor;

determining by said image database an orientation of said image capture device based upon the geolocation information associated with said at least one feature descriptor; and

conveying by said image database said geographic position and said orientation to said image capture device.

22. The non-transitory computer-readable storage medium of claim 20, wherein said method further comprises:

sending by said image database to said image capture device the geolocation information associated with said at least one feature descriptor;

storing the geolocation information associated with said at least one feature descriptor upon a database disposed upon said image capture device;

determining by said image capture device a geographic position of said image capture device based upon the geolocation information associated with said at least one feature descriptor; and

determining by said image capture device an orientation of said image capture device based upon the geolocation information associated with said at least one feature descriptor.

23. The non-transitory computer-readable storage medium of claim 20, wherein said method further comprises:

sending an estimated location of said image capture device to said image database; and

selecting by said image database a subset of feature descriptors to match with said feature descriptor sent by said image capture device based upon said estimated location.

24. The non-transitory computer-readable storage medium of claim 23, wherein said method further comprises:

determining by said image database a geographic position of said image capture device based upon the geolocation information associated with at least one of said subset of feature descriptors;

determining by said image database an orientation of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors; and

conveying by said image database said geographic position and said orientation to said image capture device.

25. The non-transitory computer-readable storage medium of claim 23, wherein said method further comprises:

sending by said image database to said image capture device the geolocation information associated with at least one of said subset of feature descriptors;

storing the geolocation information associated with said at least one of said subset of feature descriptors upon a database disposed upon said image capture device;

determining by said image capture device a geographic position of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors; and

determining by said image capture device an orientation of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors.

26. The non-transitory computer-readable storage medium of claim 23, wherein said method further comprises:

sending an estimated orientation of said image capture device to said database; and

selecting by said image database a subset of feature descriptors to match with said feature descriptor sent by said image capture device based upon said estimated orientation, each of said subset of feature descriptors having geolocation information associated therewith.

27. The non-transitory computer-readable storage medium of claim 26, wherein said method further comprises:

matching said feature descriptor with at least one of said subset of feature descriptors;

determining by said image database a geographic position of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors;

determining by said image database an orientation of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors; and

conveying by said image database said geographic position and said orientation to said image capture device.

28. The non-transitory computer-readable storage medium of claim 26, wherein said method further comprises:

sending by said image database to said image capture device the geolocation information associated with at least one of said subset of feature descriptors;

storing the geolocation information associated with said at least one of said subset of feature descriptors upon a database disposed upon said image capture device;

determining by said image capture device a geographic position of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors; and

determining by said image capture device an orientation of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors.

29. The non-transitory computer-readable storage medium of claim 19, wherein said method further comprises:

sending an estimated location of said image capture device to an image database; and

selecting by said image database a subset of feature descriptors to match with said feature descriptor sent by said image capture device based upon said estimated location, each of said subset of feature descriptors having geolocation information associated therewith.

30. The non-transitory computer-readable storage medium of claim 29, wherein said method further comprises:

determining by said image database a geographic position of said image capture device based upon the geolocation information associated with at least one of said subset of feature descriptors;

determining by said image database an orientation of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors; and

conveying by said image database said geographic position and said orientation to said image capture device.

31. The non-transitory computer-readable storage medium of claim 29, wherein said method further comprises:

sending by said image database to said image capture device the geolocation information associated with at least one of said subset of feature descriptors;

storing the geolocation information associated with said at least one of said subset of feature descriptors upon a database disposed upon said image capture device;

determining by said image capture device a geographic position of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors; and

determining by said image capture device an orientation of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors.

32. The non-transitory computer-readable storage medium of claim 29, wherein said method further comprises:

sending an estimated orientation of said image capture device to said database; and

selecting by said image database a subset of feature descriptors to match with said feature descriptor sent by said image capture device based upon said estimated orientation, each of said subset of feature descriptors having geolocation information associated therewith.

33. The non-transitory computer-readable storage medium of claim 32, wherein said method further comprises:

matching said feature descriptor with at least one of said subset of feature descriptors;

determining by said image database a geographic position of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors;

determining by said image database an orientation of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors; and

conveying by said image database said geographic position and said orientation to said image capture device.

34. The non-transitory computer-readable storage medium of claim 32, wherein said method further comprises:

sending by said image database to said image capture device the geolocation information associated with at least one of said subset of feature descriptors;

storing the geolocation information associated with said at least one of said subset of feature descriptors upon a database disposed upon said image capture device;

determining by said image capture device a geographic position of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors; and

determining by said image capture device an orientation of said image capture device based upon the geolocation information associated with said at least one of said subset of feature descriptors.

35. The non-transitory computer-readable storage medium of claim 19, wherein said image further comprises a second object of interest, said method further comprising:

capturing a second image of said object of interest and a second object of interest at a second location;

processing said second image to detect said located feature using said feature detection algorithm;

further processing said located feature of said second image to derive said first feature descriptor using said feature descriptor extraction algorithm;

determining a geographic position and an orientation of said image capture device at said first location and a second geographic position and a second orientation of an image capture device at said second location; and

using a bundle adjustment algorithm to calculate a geographic position of said second object of interest based upon said geographic position and said orientation of said image capture device at said first location and said second geographic position and said second orientation of an image capture device at said second location.

36. The non-transitory computer-readable storage medium of claim 19, wherein said method further comprises:

using said geolocation information for a plurality of points of interest to derive geolocation information for a second object of interest which is co-planar with said plurality of points of interest.

37. A system for determining the location of an image capture device, said system comprising:

an image capture device comprising;

a processor coupled with a bus;

a memory coupled with a bus; and

a camera; and

an image database comprising;

a memory coupled with a bus; and

a processor coupled with said bus, said processor configured to access a feature descriptor of an image received from said image capture device and match said feature descriptor with a corresponding feature descriptor having geolocation information associated therewith, said processor further configured to make said geolocation information available for access as a result of matching said feature descriptor with said corresponding feature descriptor.

38. The system of claim 37, wherein said processor receives said feature descriptor from said image capture device.

39. The system of claim 37, wherein said processor is further configured to generate a geographic position and an orientation of said image capture device based upon said geolocation information and convey said geographic position and said orientation to said image capture device.

40. The system of claim 37 wherein said processor is further configured to convey said corresponding feature descriptor and geolocation information to said image capture device.

41. The system of claim 37 wherein said processor is further configured to identify a subset of a plurality of feature descriptors having geolocation information associated therewith based upon an estimated location of said image capture device.