Location determination for an object using visual data

• US 9,036,865 B2
• Filed: 09/12/2012
• Issued: 05/19/2015
• Est. Priority Date: 09/12/2012
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Haas, Norman, Li, Ying, Pankanti, Sharathchandra, Otto, Charles A., Fujiji, Yuichi, Trinh, Hoang
• Primary Examiner(s): Mariam, Daniel
• Assistant Examiner(s): JAMES, THOMAS AIDEN
• Application Number:
  US13/612,345
• Publication Number:
  US 20140072169A1
• Time in Patent Office: 979 Days
• Field of Search: None
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/74 involving reference images or patches

Abstract

A global position of an observed object is determined by obtaining a first global position of an observed object with at least one positioning device. A determination is made as to whether a set of stored visual characteristic information of at least one landmark matches a visual characteristic information set obtained from at least one captured image comprising a scene associated with the observed object. In response to the set of stored visual characteristic information matching the obtained visual characteristic information set, a second global position of the observed object is determined based on a set of stored location information associated with the at least one landmark and the first global position.

Claims

1. A method for determining a global position of an observed object, the method comprising:

obtaining a first global position of an observed object with at least one positioning device;

determining if a set of stored visual characteristic information of at least one landmark matches a visual characteristic information set obtained from at least one captured image comprising a scene associated with the observed object; and

determining, based on the set of stored visual characteristic information matching the obtained visual characteristic information set, a second global position of the observed object based on a set of stored location information associated with the at least one landmark and the first global position.

2. The method of claim 1, wherein the first global position comprises a first degree of accuracy and the second global position comprises a second degree of accuracy that is greater than the first degree of accuracy.

3. The method of claim 1, further comprising:

determining a geographical vicinity associated with the observed object based on the first global position; and

selecting the at least one landmark from a plurality of landmarks based on a set of location information associated with the at least one landmark corresponding to the geographical vicinity that has been determined.

4. The method of claim 3, wherein the geographical vicinity includes an area within a given distance from the first global position.

5. The method of claim 1, wherein at least the set of location information associated with the landmark comprises one or more of:

longitude data and latitude data;

distance travelled from a given reference point; and

a number of railroad ties counted from a given reference point.

6. The method of claim 1, further comprising:

determining that the set of stored visual characteristic information of the at least one landmark fails to match the visual characteristic information set obtained from the at least one captured image;

identifying at least one visual characteristic of the at least one landmark that has changed based on the set of stored visual characteristic information of the at least one landmark and the visual characteristic information set obtained from the at least one captured image; and

updating, based on the identifying, the set of stored visual characteristic information associated with the at least one landmark to include the at least one visual characteristic that has changed.

7. The method of claim 1, further comprising:

determining that the set of stored visual characteristic information of the at least one landmark fails to match the visual characteristic information set obtained from the at least one captured image;

determining, based on the set of stored visual characteristic information failing to match the visual characteristic information set, that the at least one landmark has been removed; and

updating at least one database to remove the set of stored visual characteristic information and the set of location information associated with the at least one landmark to indicate that at least one landmark has been removed.

8. The method of claim 1, wherein determining if the set of stored visual characteristic information of the at least one landmark matches the visual characteristic information obtained from the at least one captured image comprises:

comparing the set of stored visual characteristic information associated with the at least one landmark to the visual characteristic information set obtained from the at least one captured image; and

determining that the set of stored visual characteristic information associated with the at least one landmark matches the visual characteristic information set within a given visual appearance threshold.

9. The method of claim 7, wherein determining that the set of stored visual characteristic information matches the visual characteristic information set within a given visual appearance threshold comprises:

determining, based on the comparing, that a set of differences between the set of stored visual characteristic information associated with the at least one landmark and the visual characteristic information set obtained from the at least one captured image is within the given visual appearance threshold.

10. The method of claim 1, wherein the second global position is further determined based on at least one of a railroad tie count and a distance travelled from a given reference point.

11. The method of claim 1, wherein the observed object is at least one of a missing railroad track component and a damaged railroad track component.

12. A method for maintaining a database of target objects for inspection, the method comprising:

analyzing at least one image comprising a scene, wherein the image is captured by at least one image capturing device situated on a vehicle;

identifying, based on the analyzing, a first set of visual characteristic information associated with the scene;

comparing the first set of visual characteristic information with a second set of visual characteristic information associated with at least one target object;

determining, based on the comparing, if the first set of visual characteristic information and the second set of visual characteristic information match each other within a given threshold;

based on the first set of visual characteristic information and the second set of visual characteristic information matching each other,

obtaining a set of positioning information associated with the vehicle; and

updating a database record comprising the second set of visual characteristic information and associated with the at least one target object, the updating adding the set of positioning information to the database record;

based on the first set of visual characteristic information and the second set of visual characteristic information failing to match each other,

identifying at least one visual characteristic of the at least one target object that has changed based on the first set of visual characteristic information and the second set of visual characteristic information;

obtaining a set of positioning information associated with the vehicle; and

updating, based on the identifying, the database record comprising the second set of visual characteristic information and associated with the at least one target object, the updating adding the set of positioning information and the at least one visual characteristic that has changed to the database record.

13. An information processing system for determining a global position of an observed object, the information processing system comprising:

a memory;

a processor communicatively coupled to the memory; and

a location determining system communicatively coupled to the memory and the processor, wherein the location determining system is configured to perform a method comprising;

obtaining a first global position of an observed object with at least one positioning device;

determining if a set of stored visual characteristic information of at least one landmark matches a visual characteristic information set obtained from at least one captured image comprising a scene associated with the observed object; and

determining, based on the set of stored visual characteristic information matching the obtained visual characteristic information set, a second global position of the observed object based on a set of stored location information associated with the at least one landmark and the first global position.

14. The information processing system of claim 13, wherein the first global position comprises a first degree of accuracy and the second global position comprises a second degree of accuracy that is greater than the first degree of accuracy.

15. The information processing system of claim 13, the method further comprising:

determining a geographical vicinity associated with the observed object based on the first global position; and

selecting the at least one landmark from a plurality of landmarks based on a set of location information associated with the at least one landmark corresponding to the geographical vicinity that has been determined.

16. The information processing system of claim 15, wherein the geographical vicinity includes an area within a given distance from the first global position.

17. The information processing system of claim 13, wherein the method further comprises:

determining that the set of stored visual characteristic information of the at least one landmark fails to match the visual characteristic information set obtained from the at least one captured image;

identifying at least one visual characteristic of the at least one landmark that has changed based on the set of stored visual characteristic information of the at least one landmark and the visual characteristic information set obtained from the at least one captured image; and

updating, based on the identifying, the set of stored visual characteristic information associated with the at least one landmark to include the at least one visual characteristic that has changed.

18. The information processing system of claim 13, wherein the method further comprises:

determining that the set of stored visual characteristic information of the at least one landmark fails to match the visual characteristic information set obtained from the at least one captured image;

determining, based on the set of stored visual characteristic information failing to match the visual characteristic information set, that the at least one landmark has been removed; and

updating at least one database to remove the set of stored visual characteristic information and the set of location information associated with the at least one landmark to indicate that at least one landmark has been removed.

19. The information processing system of claim 13, wherein determining if the set of stored visual characteristic information of the at least one landmark matches the visual characteristic information obtained from the at least one captured image comprises:

comparing the set of stored visual characteristic information associated with the at least one landmark to the visual characteristic information set obtained from the at least one captured image; and

determining that the set of stored visual characteristic information associated with the at least one landmark matches the visual characteristic information set within a given visual appearance threshold.

20. The information processing system of claim 19, wherein determining that the set of stored visual characteristic information matches the visual characteristic information set within a given visual appearance threshold comprises:

determining, based on the comparing, that a set of differences between the set of stored visual characteristic information associated with the at least one landmark and the visual characteristic information set obtained from the at least one captured image is within the given visual appearance threshold.