Organising and storing documents

• US 9,165,063 B2
• Filed: 07/05/2007
• Issued: 10/20/2015
• Est. Priority Date: 07/06/2006
• Status: Active Grant

• Current Assignee: British Telecommunications PLC (BT Group PLC)
• Original Assignee: British Telecommunications PLC (BT Group PLC)
• Inventors: Thurlow, Ian, Thurlow, Richard, Davies, Nicholas J
• Primary Examiner(s): Choi, Yuk Ting
• Application Number:
  US12/307,622
• Publication Number:
  US 20090327276A1
• Time in Patent Office: 3,029 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 16/313 Selection or weighting of terms for indexing
• CPC Class Codes: G06F 16/38 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/93 Document management systems

Abstract

A data handling device has access to a store of existing metadata pertaining to existing documents having associated metadata terms. It analyses the metadata to generate statistical data as to the co-occurrence of pairs of terms in the metadata of one and the same document. When a fresh document is received, it is analysed to assign to it a set of terms and determine for each a measure of their strength of association with the document. Then, for each term of the set, a score is generated that is a monotonically increasing function of (a) the strength of association with the document and of (b) the relative frequency of co-occurrence of that term and another term that occurs in the set; metadata for the fresh document are then selected as the subset of the terms in the set having the highest scores.

Claims

1. A data handling device for organising and storing documents for subsequent retrieval, the documents having associated metadata terms, the device comprising:

means configured to provide access to a store of existing metadata;

means configured to analyse the existing metadata to generate statistical data as to co-occurrence of pairs of terms in the metadata of a single document;

means configured to analyse a fresh document to assign to the fresh document a set of terms and configured to determine a measure of a strength of association of each term with the document;

means configured to determine for each term of the set of terms a score that is a monotonically increasing function of (a) the strength of association with the document and of (b) a relative frequency of co-occurrence, in the existing metadata, of the term and another term that occurs in the set of terms;

means configured to select, as metadata for the fresh document, a subset of the terms in the set of terms having highest scores.

2. The data handling device according to claim 1, in which the terms are terms of a predetermined set of terms.

3. The data handling device according, to claim 1, in which analysis of the metadata generates for each term a set of weights, each weight indicating a number representing how many documents have been assigned both the term in question and a respective other term, divided by a total number of documents to which the term in question has been assigned.

4. The data handling device according to claim 3 in which each set of weights is further processed by:

for each term for which a nonzero weight has been generated, comparing the set of weights for the term with the set of weights under consideration to determine a rating equal to the number of terms appearing in both sets of weights; and

deleting low-rating terms from the set of terms.

5. The data handling device according to claim 1, in which the terms are words or phrases and the strength of association determined by document analysis for each term is a number representing how many times the term occurs in the document.

6. The data handling device according to claim 1, in which the score for a term is proportional to the strength of association.

7. The data handling device according to claim 1, in which the score for a term is proportional to the square root of a number representing how many terms have been assigned to the document and also have nonzero weights in the set of weights for that term.

8. The data handling device according to claim 1, in which the score for a term is also a function of the weights in the set of weights for that term.

9. The data handling device according to claim 8, in which the score for a term is a function of the sum of the square roots of the weights in the set of weights for that term.

10. A method of organising and storing documents in a computer system for subsequent retrieval, the documents having associated metadata terms, the method comprising:

providing access to a store of existing metadata in the computer system;

analysing the existing metadata to generate statistical data as to co-occurrence of pairs of terms in the metadata of a single document;

analysing a fresh document to assign to the fresh document a set of terms and determine for each term of the set a measure of a strength of association of the term with the document;

determining for each term of the set a score that is a monotonically increasing function of (a) the strength of association with the document and of (b) a relative frequency of co-ocurrence, in the existing metadata, of the term and another term that occurs in the set; and

selecting, as metadata for the fresh document, a subset of the terms in the set having highest scores.

11. The method according to claim 10, in which the terms are terms of a predetermined set of terms.

12. The method according to claim 10, in which analysis of the metadata generates for each term a set of weights, each weight indicating a number representing how many documents have been assigned both the term in question and a respective other term, divided by a total number of documents to which the term in question has been assigned.

13. The method according to claim 12 in which each set of weights is further processed by:

for each term for which a nonzero weight has been generated, comparing the set of weights for the term with the set of weights under consideration to determine a rating equal to the number of terms appearing in both sets of weight; and

deleting low-rating terms from the set of terms.

14. The method according to claim 10 in which the terms are words or phrases and the strength of association determined by document analysis for each term is a number representing how many times the term occurs in the document.

15. The method according to claim 10, in which the score for a term is proportional to the strength of association.

16. The method according to claim 10, in which the score for a term is proportional to the square root of a number a number representing how many terms have been assigned to the document and also have nonzero weights in the set of weights for that term.

17. The method according to claim 10, in which the score for a term is also a function of the weights in the set of weights for that term.

18. The method according to claim 17, in which the score for a term is a function of the sum of the square roots of the weights in the set of weights for that term.