Systems and methods for organizing data sets

• US 8,321,477 B2
• Filed: 06/29/2010
• Issued: 11/27/2012
• Est. Priority Date: 03/05/2008
• Status: Active Grant

• Current Assignee: Kofax Incorporated
• Original Assignee: Kofax Incorporated
• Inventors: Schmidtler, Mauritius A. R., Amtrup, Jan W., Thompson, Stephen Michael, Sarah, Anthony
• Primary Examiner(s): Beausoliel, Jr., Robert
• Assistant Examiner(s): Arjomandi, Noosha
• Application Number:
  US12/826,536
• Publication Number:
  US 20100262571A1
• Time in Patent Office: 882 Days
• Field of Search: 707/807
• US Class Current: 707/807
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]

Abstract

A method is provided for organizing data sets. In use, an automatic decision system is created or updated for determining whether data elements fit a predefined organization or not, where the decision system is based on a set of preorganized data elements. A plurality of data elements is organized using the decision system. At least one organized data element is selected for output to a user based on a score or confidence from the decision system for the at least one organized data element. Additionally, at least a portion of the at least one organized data element is output to the user. A response is received from the user comprising at least one of a confirmation, modification, and a negation of the organization of the at least one organized data element. The automatic decision system is recreated or updated based on the user response. Other embodiments are also presented.

Claims

1. A method for detecting limits to the quality in a dynamic organization process, comprising:

receiving user input affirming, negating, or modifying organization of a plurality of data elements during or after an organization process;

determining a quality of the organization based on the user input;

determining a stability in decisions made during the organization of the plurality of data elements;

monitoring the quality and the stability;

determining that the organization has reached a limit concerning at least one of the quality and stability of the organization by evaluating the at least one of the quality and stability of the organization over a period of time or number of cycles; and

outputting an indication of the determination to at least one of a user, a system and another process,

wherein the determination is that the quality of the organization is not likely to significantly change when the at least one of the quality and stability do not change by more than a predetermined amount over a given amount of time.

3. The method of claim 1, further comprising outputting at least portions of the data elements to the user during the organization process, wherein the at least one of the quality and the stability is updated each time user input is received.

4. The method of claim 1, further comprising changing an input or its representation used during the organization upon determining that the organization process has reached a limit.

5. The method of claim 1, further comprising implementing different decision functions than currently being used for organization upon determining that the organization process has reached a limit.

9. The method of claim 1, where the data elements are documents and the organization is based on an assignment of documents to categories based on the content of the documents.

10. The method of claim 1, where the data elements are elements within a document, and the organization is based on an assignment of labels to said elements based on the function of said elements in said document.

2. A method for detecting limits to the quality in a dynamic organization process, comprising:

receiving user input affirming, negating, or modifying organization of a plurality of data elements during or after an organization process;

determining a quality of the organization based on the user input;

determining a stability in decisions made during the organization of the plurality of data elements;

monitoring the quality and the stability;

determining that the organization has reached a limit concerning at least one of the quality and stability of the organization by evaluating the at least one of the quality and stability of the organization over a period of time or number of cycles; and

outputting an indication of the determination to at least one of a user, a system and another process,

wherein the determination is that the organization process is having difficulty in handling the data when the at least one of the quality and stability oscillate over time.

6. A method for detecting limits to the quality in a dynamic organization process, comprising:

receiving user input affirming, negating, or modifying organization of a plurality of data elements during or after an organization process;

determining a quality of the organization based on the user input;

determining a stability in decisions made during the organization of the plurality of data elements;

monitoring the quality and the stability;

determining that the organization has reached a limit concerning at least one of the quality and stability of the organization by evaluating the at least one of the quality and stability of the organization over a period of time or number of cycles;

outputting an indication of the determination to at least one of a user, a system and another process; and

altering the organization of data elements upon determining that the organization process has reached a limit.

7. The method of claim 6, wherein altering the organization of data elements includes combining data elements from at least two aspects of the organization into a new single aspect.

8. The method of claim 7, further comprising, in a second organization process, separating the data elements in the new single aspect back into the at least two aspects.

11. A computer program product comprising a non-transitory computer readable medium having computer code thereon, which when executed on a computer causes the computer to:

receive user input affirming, negating, or modifying organization of a plurality of data elements during or after an organization process;

determine a quality of the organization based on the user input;

determine a stability in decisions made during the organization of the plurality of data elements;

monitor the quality and the stability;

determine that the organization has reached a limit concerning at least one of the quality and stability of the organization by evaluating the at least one of the quality and stability of the organization over a period of time or number of cycles; and

output an indication of the determination to at least one of a user, a system and another process,

wherein the determination is that the quality of the organization is not likely to significantly change when the at least one of the quality and stability do not change by more than a predetermined amount over a given amount of time.

12. A system for detecting limits to the quality in a dynamic organization process, comprising:

an adapter configured to receive user input affirming, negating, or modifying organization of a plurality of data elements during or after an organization process;

a processor configured to determine a quality of the organization based on the user input;

logic configured to determine a stability in decisions made during the organization of the plurality of data elements;

logic configured to monitor the quality and the stability;

logic configured to determine that the organization has reached a limit concerning at least one of the quality and stability of the organization by evaluating the at least one of the quality and stability of the organization over a period of time or number of cycles; and

an adapter configured to output an indication of the determination to at least one of a user, a system and another process,

wherein logic configured to determine that the organization has reached a limit is also configured to determine that the quality of the organization is not likely to significantly change when the at least one of the quality and stability do not change by more than a predetermined amount over a given amount of time.

14. The system of claim 12, further comprising logic for outputting at least portions of the data elements to the user during the organization process, wherein the at least one of the quality and the stability is updated each time user input is received.

15. The system of claim 12, further comprising logic for changing an input or its representation used during the organization upon determining that the organization process has reached a limit.

16. The system of claim 12, further comprising logic for implementing different decision functions than currently being used for organization upon determining that the organization process has reached a limit.

18. The system of claim 12, where the data elements are documents and the organization is based on an assignment of documents to categories based on the content of the documents.

19. The system of claim 12, where the data elements are elements within a document, and the organization is based on an assignment of labels to said elements based on the function of said elements in said document.

13. A system for detecting limits to the quality in a dynamic organization process, comprising:

an adapter configured to receive user input affirming, negating, or modifying organization of a plurality of data elements during or after an organization process;

a processor configured to determine a quality of the organization based on the user input;

logic configured to determine a stability in decisions made during the organization of the plurality of data elements;

logic configured to monitor the quality and the stability;

logic configured to determine that the organization has reached a limit concerning at least one of the quality and stability of the organization by evaluating the at least one of the quality and stability of the organization over a period of time or number of cycles; and

an adapter configured to output an indication of the determination to at least one of a user, a system and another process,

wherein the determination is that the organization process is having difficulty in handling the data when the at least one of the quality and stability oscillate over time.

17. A system for detecting limits to the quality in a dynamic organization process, comprising:

an adapter configured to receive user input affirming, negating, or modifying organization of a plurality of data elements during or after an organization process;

a processor configured to determine a quality of the organization based on the user input;

logic configured to determine a stability in decisions made during the organization of the plurality of data elements;

logic configured to monitor the quality and the stability;

logic configured to determine that the organization has reached a limit concerning at least one of the quality and stability of the organization by evaluating the at least one of the quality and stability of the organization over a period of time or number of cycles; and

an adapter configured to output an indication of the determination to at least one of a user, a system and another process; and

logic for altering the organization of data elements upon determining that the organization process has reached a limit.