System And Method For Propagating Classification Decisions

• US 20110196879A1
• Filed: 02/04/2011
• Published: 08/11/2011
• Est. Priority Date: 02/05/2010
• Status: Active Grant

• Current Assignee: Nuix North America Inc. (Nuix Ltd.)
• Original Assignee: FTI Consulting Incorporated
• Inventors: Gabriel, Manfred J., Robinson, Eric Michael
• Granted Patent: US 8,296,290 B2
• Time in Patent Office: Days
• US Class Current: 707/749
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/3334 Selection or weighting of terms from queries, including natural language queries
• CPC Class Codes: G06F 16/334 Query execution G06F16/335 takes precedence
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: G06Q 50/18 Legal services

Abstract

A system and method for propagating classification decisions is provided. Text marked within one or more unclassified documents that is determined to be responsive to a predetermined issue is received from a user. The unclassified documents are selected from a corpus. A search query is generated from the responsive text. Same result documents are identified by applying inclusive search parameters to the query, applying the search query to the corpus, and identifying the documents that satisfy the query. Similar result documents are identified by adjusting a breadth of the query by applying less inclusive search parameters and identifying documents from the corpus that satisfy the query. A responsive classification code is automatically assigned to each same result document for classification as responsive documents. The similar documents are provided to the user. A responsive classification decision is received form the user for classification as the responsive documents.

Claims

1. A system for propagating classification decisions, comprising:

text marked within one or more unclassified documents that is determined to be responsive to a predetermined issue, wherein the one or more unclassified documents are selected from a corpus of unclassified documents;

a query generator to generate a search query from the responsive text and to identify result documents, comprising;

a same search module to identify same result documents by applying inclusive search parameters to the query, by applying the search query to the corpus, and by identifying the documents that satisfy the query as the same result documents; and

a similar search module to identify similar result documents from the corpus by adjusting a breath of the query by applying less inclusive search parameters and by identifying documents from the corpus that satisfy the query as the similar result documents;

a propagator to automatically assigning a responsive classification code to the same result documents for classification as responsive documents; and

a document feeder to provide the similar documents to the user and to receive a responsive classification decision from the user for at least one of the similar documents for classification as the responsive documents.

2. A system according to claim 1, wherein the document feeder further provides the one or more documents to the user for review via at least one of randomly selecting each of the one or more documents, selecting each of the one or more documents based on a similarity of that document to the responsive text, and randomly selecting a first set of documents from the one or more documents and selecting the remaining documents from the one or more documents based on the similarity of that document to the responsive text.

3. A system according to claim 2, further comprising:

a quantity module to determine a quantity of the one or more documents to be provided randomly to the user in accordance with the equation;

4. A system according to claim 1, further comprising:

a document calculation module to determine a total expected number of the responsive documents in the corpus and to provide the unclassified documents to the user for review until the total expected number of responsive documents is satisfied.

5. A system according to claim 4, wherein the total expected number of responsive documents is calculated based on a probability that the document includes text that is responsive to the predetermined issue and a total number of documents in the corpus.

6. A system according to claim 1, wherein the marked text is identified by a marking indicator comprising at least one of a text box, highlighting, underlining, font style, and font size.

7. A system according to claim 1, wherein the generator generates at least one further search query based on a further text selection that is responsive to the predetermined issue and applies the further search query to the corpus to identify further responsive documents as those that satisfy the further search query.

8. A system according to claim 1, further comprising:

a validator to select a number of the unclassified documents remaining in the corpus, to analyze the selected unclassified documents for responsive text, and to transmit further unclassified documents to the user for review when at least one of the selected unclassified documents includes responsive text.

9. A system according to claim 1, further comprising:

a document calculation module to calculate the number of unclassified documents in accordance with the equation;

10. A system according to claim 1, further comprising:

a quality control module to edit the search query based on feedback from the user.

14. A method according to claim 1 further comprising:

determining a total expected number of the responsive documents in the corpus; and

providing the unclassified documents to the user for review until the total expected number of responsive documents is satisfied.

15. A method according to claim 14, wherein the total expected number of responsive documents is calculated based on a probability that the document includes text that is responsive to the predetermined issue and a total number of documents in the corpus.

18. A method according to claim 1 further comprising:

selecting a number of the unclassified documents remaining in the corpus;

analyzing the selected unclassified documents for responsive text; and

transmitting further unclassified documents to the user for review when at least one of the selected unclassified documents includes responsive text.

19. A method according to claim 1 further comprising:

calculating the number of unclassified documents in accordance with the equation;

11. A method for propagating classification decisions, comprising:

receiving from a user, text marked within one or more unclassified documents that is determined to be responsive to a predetermined issue, wherein the one or more unclassified documents are selected from a corpus of unclassified documents;

generating a search query from the responsive text;

identifying same result documents, comprising;

applying inclusive search parameters to the query;

applying the search query to the corpus; and

identifying the documents that satisfy the query as the same result documents;

identifying similar result documents from the corpus, comprising;

adjusting a breath of the query by applying less inclusive search parameters; and

identifying documents from the corpus that satisfy the query as the similar result documents;

automatically assigning a responsive classification code to the same result documents for classification as responsive documents; and

providing the similar documents to the user and receiving a responsive classification decision from the user for at least one of the similar documents for classification as the responsive documents.

12. A method according to claim 11, further comprising:

providing the one or more documents to the user for review, comprising at least one of;

randomly selecting each of the one or more documents;

selecting each of the one or more documents based on a similarity of that document to the responsive text; and

randomly selecting a first set of documents from the one or more documents and selecting the remaining documents from the one or more documents based on the similarity of that document to the responsive text.

13. A method according to claim 12, further comprising:

determining a quantity of the one or more documents to be provided randomly to the user in accordance with the equation;

16. A method according to claim 11, wherein the marked text is identified by a marking indicator comprising at least one of a text box, highlighting, underlining, font style, and font size.

17. A method according to claim 11, further comprising:

identifying further responsive documents for the predetermined issue, comprising;

generating at least one further search query based on a further text selection that is responsive to the predetermined issue; and

applying the further search query to the corpus to identify the further responsive documents as those that satisfy the further search query.

20. A method according to claim 11, further comprising:

editing the search query based on feedback from the user.