METHOD AND SYSTEM FOR MAINTAINING A DATABASE OF REFERENCE IMAGES

• US 20110282897A1
• Filed: 06/05/2009
• Published: 11/17/2011
• Est. Priority Date: 06/06/2008
• Status: Abandoned Application

• Current Assignee: Agency For Science Technology and Research
• Original Assignee: Agency For Science Technology and Research
• Inventors: Li, Yiqun, Goh, Hanlin, Lim, Joo Hwee
• Application Number:
  US12/996,494
• Publication Number:
  US 20110282897A1
• Time in Patent Office: Days
• US Class Current: 707/769
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 16/58 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 18/211 Selection of the most significant subset of features
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]
• CPC Class Codes: G06V 10/771 Feature selection, e.g. selecting representative features from a multi-dimensional feature space

Abstract

A method and system for maintaining a database of reference images, the database including a plurality of sets of images, each set associated with one location or object. The method comprises the steps of identifying local features of each set of images; determining distances between each local feature of each set and the local features of all other sets; identifying discriminative features of each set of images by removing local features based on the determined distances; and storing the discriminative features of each set of images.

Claims

1. A method of maintaining a database of reference images, the database including a plurality of sets of images, each set associated with one location or object;

the method comprising the steps of;

identifying local features of each set of images;

determining distances between each local feature of each set and the local features of all other sets;

identifying discriminative features of each set of images by removing local features based on the determined distances; and

storing the discriminative features of each set of images.

2. The method as claimed in claim 1, wherein identifying the local features comprises:

identifying key points; and

extracting features from the key points.

3. The method as claimed in claim 2, further comprising reducing a number of key points prior to extracting the features.

4. The method as claimed in claim 3, wherein reducing the number of key points comprises a region-based key point reduction.

5. The method as claimed in claim 4, wherein the region-based key point reduction comprises choosing one of the key points in a region having a highest radius.

6. The method as claimed in claim 2, further comprising reducing a number of extracted features.

7. The method as claimed in claim 6, wherein reducing the number of extracted features comprises a hierarchical feature clustering.

8. The method as claimed in claim 1, wherein removing local features based on the determined distances comprises removing the local features having distances to any local feature of the other sets lower than a first threshold.

9. The method as claimed in claim 1, wherein removing local features based on the determined distances comprises:

calculating respective discriminative values for each local feature of said set based on the determined distances; and

removing the local features having discriminative values lower than a second threshold.

10. A method for image based mobile information retrieval, the method comprising the steps of:

maintaining a dedicated database of reference images as claimed in claim 1;

taking a query image of a location or object by a user using a mobile device;

transmitting the query image to a information server;

comparing the image with reference images in the dedicated database coupled to the information server;

identifying the location or object based on a matched reference image; and

transmitting information based on the identified location or object to the user.

11. The method as claimed in claim 10, wherein comparing the image with reference images comprises a nearest neighbour matching.

12. The method as claimed in claim 11, wherein nearest neighbour matching comprises:

determining a minimum distance between each feature vector of the query image and feature vectors of reference images of each location or object; and

calculating a number of matches for each location or object, wherein a match comprises the minimum distance being smaller than a third threshold.

13. The method as claimed in claim 12, wherein the third threshold is equal to the first threshold.

14. The method as claimed in claim 12, further comprising calculating a vote based on the number of matches and an average matching distance, wherein the highest vote comprises the nearest neighbour.

15. The method as claimed in claim 10, wherein the identifying of the location or object comprises a multi query user verification.

16. The method as claimed in claim 15, further comprising transmitting a sample photo of the identified location or object to the user.

17. The method as claimed in claim 15, wherein the multi query user verification comprises taking a new query image of the location or object by the user using the mobile device and transmitting the new query image to an information server.

18. The method as claimed in claim 17, further comprising calculating a confidence level of the identified location or object based on results of one or more previous query images and the new query image.

19. The method as claimed in claim 18, further comprising transmitting a new query image recommendation to the user if the confidence level of the identified location or object is below a fourth threshold.

24. A system for image based mobile information retrieval, the system comprising:

means for maintaining a dedicated database of reference images as claimed in claim 1;

means for receiving a query image of a location or object taken by a user using a mobile device;

means for comparing the image with reference images in the dedicated database;

means for identifying the location or object based on a matched reference image; and

means for transmitting information based on the identified location or object to the user.

20. A system for maintaining a database of reference images, the database including a plurality of sets of images, each set associated with one location or object;

the system comprising;

means for identifying local features of each set of images;

means for determining distances between each local feature of each set and the local features of all other sets;

means for identifying discriminative features of each set of images by removing local features based on the determined distances; and

means for storing the discriminative features of each set of images.

21. The system as claimed in claim 20, wherein the means for identifying the discriminative features removes the local features having distances to any local feature of the other sets lower than a first threshold.

22. The system as claimed in claim 20, wherein the means for identifying the discriminative features calculates respective discriminative values for each local feature of said set based on the determined distances, and removes the local features having discriminative values lower than a second threshold.

23. A data storage medium comprising code means for instructing a computing device to exercise a method of maintaining a database of reference images, the database including a plurality of sets of images, each set associated with one location or object;

the method comprising the steps of;

identifying local features of each set of images;

determining distances between each local feature of each set and the local features of all other sets;

identifying discriminative features of each set of images by removing local features based on the determined distances; and

storing the discriminative features of each set of images.

25. A data storage medium comprising code means for instructing a computing device to exercise a method for image based mobile information retrieval, the method comprising the steps of:

receiving a query image of a location or object taken by a user using a mobile device;

comparing the image with reference images in the dedicated database;

identifying the location or object based on a matched reference image; and

transmitting information based on the identified location or object to the user.