ORGANISING AND STORING DOCUMENTS

• US 20090327276A1
• Filed: 07/05/2007
• Published: 12/31/2009
• Est. Priority Date: 07/06/2006
• Status: Active Grant

• Current Assignee: British Telecommunications PLC (BT Group PLC)
• Original Assignee: British Telecommunications PLC (BT Group PLC)
• Inventors: Thurlow, Ian, Thurlow, Richard, Davies, Nicholas J.
• Granted Patent: US 9,165,063 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/38 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/93 Document management systems

Abstract

A data handling device has access to a store of existing metadata pertaining to existing documents having associated metadata terms. It analyses the metadata to generate statistical data as to the co-occurrence of pairs of terms in the metadata of one and the same document. When a fresh document is received, it is analysed to assign to it a set of terms and determine for each a measure of their strength of association with the document. Then, for each term of the set, a score is generated that is a monotonically increasing function of (a) the strength of association with the document and of (b) the relative frequency of co-occurrence of that term and another term that occurs in the set; metadata for the fresh document are then selected as the subset of the terms in the set having the highest scores.

Claims

1. A data handling device for organising and storing documents for subsequent retrieval, the documents having associated metadata terms, the device comprising:

means providing access to a store of existing metadata;

means operable to analyse the existing metadata to generate statistical data as to the co-occurrence of pairs of terms in the metadata of one and the same document;

means for analysing a fresh document to assign to it a set of terms and determine for each a measure of their strength of association with the document;

means operable to determine for each term of the set a score that is a monotonically increasing function of (a) the strength of association with the document and of (b) the relative frequency of co-occurrence, in the existing metadata, of that term and another term that occurs in the set;

means operable to select, as metadata for the fresh document, a subset of the terms in the set having the highest scores.

2. A data handling device according to claim 1, in which the terms are terms of a predetermined set of terms.

3. A data handling device according, to claim 1, in which the analysis of the metadata generates for each term a set of weights, each weight Indicating the number of documents that have been assigned both the term in question and a respective other term, divided by the total number of documents to which the term in question has been assigned.

4. A data handling device according to claim 3 in which each set of weights is further processed by:

for each term having a nonzero weight, comparing the set of weights for that term with the set of weights under consideration to determine a rating equal to

the number of terms appearing in both vectors; and

deleting low-rating terms from the set.

5. A data handling device according to claim 1, in which the terms are words or phrases and the strength of association determined by the document analysis for each term is the number of occurrences of that term in the document.

6. A data handling device according to claim 1, in which the score for a term is proportional to the strength of association.

7. A data handling device according to claim 1, in which the score for a term is proportional to the square root of the number of terms which have been assigned to the document and also have nonzero weights in the set for that term.

8. A data handling device according to claim 1, in which the score for a term is also a function of the weights in the set for that term.

9. A data handling device according to claim 8, in which the score for a term is a function of the sum of the square roots of the weights in the set for that term.

10. A method of organising and storing documents for subsequent retrieval, the documents having associated metadata terms, the method comprising:

providing access to a store of existing metadata;

analysing the existing metadata to generate statistical data as to the co-occurrence of pairs of terms in the metadata of one and the same document;

analysing a fresh document to assign to it a set of terms and determine for each a measure of their strength of association with the document;

determining for each term of the set a score that is a monotonically increasing function of (a) the strength of association with the document and of (b) the relative frequency of co-occurrence, in the existing metadata, of that term and another term that occurs in the set; and

selecting, as metadata for the fresh document, a subset of the terms in the set having the highest scores.

11. A data handling device according to claim 10, in which the terms are terms of a predetermined set of terms.

12. A method according to claim 10, in which the analysis of the metadata generates for each term a set of weights, each weight indicating the number of documents that have been assigned both the term in question and a respective other term, divided by the total number of documents to which the term in question has been assigned.

13. A method according to claim 12 in which each set of weights is further processed by:

for each term having a nonzero weight, comparing the set of weights for that term with the set of weights under consideration to determine a rating equal to the number of terms appearing in both vectors; and

deleting low-rating terms from the set.

14. A method according to claim 10, in which the terms are words or phrases and the strength of association determined by the document analysis for each term is the number of occurrences of that term in the document.

15. A method according to claim 10, in which the score for a term is proportional to the strength of association.

16. A method according to claim 10, in which the score for a term is proportional to the square root of the number of terms which have been assigned to the document and also have nonzero weights in the set for that term.

17. A method according to claim 10, in which the score for a term is also a function of the weights in the set for that term.

18. A method according to claim 17, in which the score for a term is a function of the sum of the square roots of the weights in the set for that term.