INFORMATION EXTRACTION APPARATUS AND METHODS

• US 20110099184A1
• Filed: 10/10/2008
• Published: 04/28/2011
• Est. Priority Date: 10/10/2007
• Status: Active Grant

• Current Assignee: ITI Scotland Limited (Scottish Government)
• Original Assignee: ITI Scotland Limited (Scottish Government)
• Inventors: Symington, Beatrice
• Granted Patent: US 8,495,042 B2
• Time in Patent Office: Days
• US Class Current: 707/755
• CPC Class Codes: G06F 16/34 Browsing; Visualisation therefor

Abstract

Automatic information extraction apparatus for extracting data for review by a human curator from digital representations of documents comprising natural language text, the automatic information extraction apparatus having a plurality of selectable operating modes in which the automatic information extraction apparatus is operable to extract different data for review by a human curator. In the different operating modes, the information extraction apparatus may extract data with a different balance between recall and precision.

Claims

1. Automatic information extraction apparatus for extracting data for review by a human curator from digital representations of documents comprising natural language text, the automatic information extraction apparatus having a plurality of selectable operating modes in which the automatic information extraction apparatus is operable to extract different data for review by a human curator.

2. Automatic information extraction apparatus according to claim 1, wherein, in at least two of the plurality of selectable operating modes, the automatic information extraction apparatus is operable to extract data with different balances between precision and recall.

3. Automatic information extraction apparatus according to claim 1 or claim 2, wherein the automatic information extraction apparatus comprises a plurality of alternative information extraction modules which are operable to cause different data to be extracted for review.

4. Automatic information extraction apparatus according to claim 3, wherein alternative information extraction modules comprise the same computer program module but different parameter data which specifies the functionality of the computer program module.

5. Automatic information extraction apparatus according to claim 3 or claim 4, wherein the alternative information extraction modules are named entity recognition modules.

6. Automatic information extraction apparatus according to claim 5, wherein the alternative named entity recognition modules comprise the same software modules which implements a named entity recognition algorithm and different parameter data which determines the function of the named entity recognition algorithm.

7. Automatic information extraction apparatus according to any one preceding claim, wherein the automatic information extraction apparatus is operable to identify individual instances of entities in a digital representation of a document.

8. Automatic information extraction apparatus according to claim 7, wherein the automatic information extraction apparatus is operable to ensure that each individual instance of a character string which is identified as an entity is consistently identified as the same entity within the same digital representation of a document.

9. Automatic information extraction apparatus according to claim 7 or claim 8, wherein the automatic information extraction apparatus is operable to ensure that each individual instance of a character string which is identified as a particular type of entity is consistently identified as the same type of entity, respectively, within the same digital representation of a document.

10. Automatic information extraction apparatus according to any one preceding claim, which is operable to apply different algorithms to force consistent extraction of data concerning entities represented by identical character strings within a document in two different information extraction modes.

11. Automatic information extraction apparatus according to any one preceding claim, wherein the automatic information extraction apparatus is operable to receive data concerning the human curator who will review data extracted from a particular digital representation of a document, or preferences associated with the human curator who will review the data extracted from a particular digital representation of a document, and to select an operating mode from amongst the plurality of operating modes for the extraction of data from that particular digital representation of a document.

12. Information extraction and curation apparatus comprising automatic information extraction apparatus according to any one preceding claim and a curation computer-user interface which is operable to present extracted data to a human curator for review and possible amendment.

13. Information extraction and curation apparatus according to claim 12, operable to analyse the interaction between one or more human curators and the curation computer-user interface in order to allocate an operating mode for subsequent information extraction by a particular human curator, or to modify an information extraction mode for subsequent information extraction, or to measure the performance of an information extraction mode.

14. Information extraction and curation apparatus according to claim 12 or claim 13, operable to measure one or more metrics related to the speed of a curator.

15. Information extraction and curation apparatus according to any one of claims 12 to 14, wherein the curation computer-user interface is operable to display a digital representation of a document to a user with identified entities and/or relations highlighted in the display digital representation of a document.

16. A method of extracting data for review by a human curator from digital representations of documents comprising natural language text, comprising providing automatic information extraction apparatus according to any one of claims 1 to 11 or information extraction and curation apparatus according to any one of claims 12 to 15, selecting an operating mode from amongst the plurality of operating modes and causing the automatic information extraction apparatus to function in the selected operating mode to extract data for review by a human curator.

17. A method of extracting data according to claim 16, wherein the operating mode is selected dependent on the human curator, from a group of human curators, who is to review the extracted data.

18. A method of extracting data according to claim 17, comprising carrying out information extraction in a plurality of operating modes to extract data for review by a plurality of human curators.

19. A method of extraction data according to any one of claims 16 to 18, comprising receiving instructions as to the information extraction mode to be employed to extract data from a specific batch of digital representations of documents and then extracting data from the specified batch of digital representations of documents using the specific information extraction module.

25. A computer readable carrier storing computer program instructions which, when executed on computing apparatus, cause the computing apparatus to function as the automatic information extraction apparatus of any one of claims 1 to 11, or the information extraction and curation apparatus of any one of claims 12 to 15, or which cause the computing apparatus to carry out the methods of any one of claims 17 to 24.

20. A method of optimising automatic information extraction apparatus which is operable to extract data for review by a human curator from digital representations of documents comprising natural language text, comprising the steps of:

(i) extracting data from at least one digital representation of a document comprising natural language text, using an information extraction module;

(ii) providing a computer-user interface which presents the extracted data to a human curator for review and which analyses the interactions between the human curator and the computer-user interface; and

(iii) modifying the information extraction module responsive to the analysed interactions to facilitate an improvement in the subsequent performance of a or the human curator using the information extraction module and computer-user interface to review extracted data.

21. A method according to claim 20, wherein the analysis of the interactions between the human curator and the computer-user interface comprises measuring one or more metrics related to the speed of a curator.

22. A method according to claim 20 or claim 21, wherein the analysis of the interactions between the human curator and the computer-user interface comprises measuring a metric related to the number of searches made by a curator using a search engine which is operable to find, edit or check extracted data.

23. A method according to any one of claims 20 to 22, wherein the information extraction module is operable to identify entities and/or relations in a digital representation of a document and the computer-user interface is operable to display at least a part of the digital representation of a document with identified entities and/or relations highlighted therein.

24. A method according to any one of claims 20 to 23, wherein the modification to the information extraction module comprises a modification, such as an optimisation, which changes the balance between precision and recall of the information extraction module.