Method and system for ongoing performance monitoring of a character recognition system

• US 20050025362A1
• Filed: 07/30/2003
• Published: 02/03/2005
• Est. Priority Date: 07/30/2003
• Status: Active Grant

• Current Assignee: Toshiba Global Commerce Solutions Holdings Corporation (Toshiba Corporation)
• Original Assignee: International Business Machines Corporation
• Inventors: Blair, Brian E., Bui, Tuyen Q.
• Granted Patent: US 7,280,684 B2
• Time in Patent Office: Days
• US Class Current: 382/182
• CPC Class Codes: G06V 10/993 Evaluation of the quality of the acquired pattern
• CPC Class Codes: Y10S 707/99935 Query augmenting and refining, e.g. inexact access
• CPC Class Codes: Y10S 707/99936 Pattern matching access

Abstract

A method and system for monitoring the performance of a character recognition system is disclosed. According to a preferred embodiment, the method comprises utilizing an average confidence score for a plurality of characters for ongoing performance monitoring of the character recognition system, wherein a confidence score indicates a level of confidence that a character is accurately recognized.

Claims

1. A method for monitoring the performance of a character recognition system, comprising the step of:

a) utilizing an average confidence score for a plurality of characters for ongoing performance monitoring of the character recognition system, wherein a confidence'"'"'score indicates a level of confidence that a character is accurately recognized.

2. The method of claim 1 further comprising the step of:

b) generating the average confidence score from the plurality of characters recorded on at least one end user document prior to the utilizing step (a).

3. The method of claim 2, wherein generating step (b) further comprises

(b1) reading a character recorded on an end user document;

(b2) calculating the confidence score for the character;

(b3) repeating steps (b1) and (b2) for each of a plurality of characters recorded on the end user document; and

(b4) averaging the confidence scores for the characters with confidence scores of all characters read over a last set number of end user documents.

4. The method of claim 1, wherein the utilizing step (a) further comprising the step of:

(a1) determining a threshold confidence score, wherein the threshold confidence score represents a value greater than a minimum level of acceptable performance for the character recognition system and less than an expected level of performance for a new character recognition system.

5. The method of claim 4, wherein the utilizing step (a) further comprises:

(a2) comparing the average confidence score to the threshold confidence score.

6. The method of claim 5, wherein the utilizing step (a) further comprising the step of:

(a3) indicating that the character recognition system is in need of service if the average confidence score falls below the threshold confidence score.

7. The method of claim 6 further comprising the steps of:

(c) servicing the character recognition system; and

(d) resetting the average confidence score calculation.

8. The method of claim 2, wherein the at least one end user document is a standardized document.

9. The method of claim 8, wherein the standardized document is a bank note or personal check.

10. The method of claim 9, wherein at least one character is a magnetic ink character recognition (MICR) character.

11. The method of claim 1, wherein the character recognition system is one of a magnetic ink character recognition system, an optical character recognition system, or a combined magnetic ink and optical character recognition system.

12. A computer readable medium containing programming instructions for monitoring the performance of a character recognition system, comprising the instruction for:

a) utilizing an average confidence score for a plurality of characters for ongoing performance monitoring of the character recognition system, wherein a confidence score indicates a level of confidence that a character is accurately recognized.

13. The computer readable medium of claim 12 further comprising the instruction for:

b) generating the average confidence score from the plurality of characters recorded on at least one end user document prior to the utilizing instruction (a).

14. The computer readable medium of claim 13, wherein generating instruction (b) further comprises instructions for:

(b1) reading a character recorded on an end user document;

(b2) calculating the confidence score for the character;

(b3) repeating instructions (b1) and (b2) for each of a plurality of characters recorded on the end user document; and

(b4) averaging the confidence scores for the characters with confidence scores of all characters read over a last set number of end user documents.

15. The computer readable medium of claim 12, wherein the utilizing instruction (a) further comprising the instruction for:

(a1) determining a threshold confidence score, wherein the threshold confidence score represents a value greater than a minimum level of acceptable performance for the character recognition system and less than an expected level of performance for a new character recognition system.

16. The computer readable medium of claim 15, wherein the utilizing instruction (a) further comprises:

(a2) comparing the average confidence score to the threshold confidence score.

17. The computer readable medium of claim 16, wherein the utilizing instruction (a) further comprising the instruction for:

(a3) indicating that the character recognition system is in need of service if the average confidence score falls below the threshold confidence score.

18. The computer readable medium of claim 17 further comprising the instructions for:

(c) servicing the character recognition system; and

(d) resetting the average confidence score calculation.

20. The system of claim 17, wherein the means for reading the plurality of characters comprises one of an optical scanner, a magnetic read head, or a combined optical scanner and magnetic read head.

21. The system of claim 17 further comprising a decoding module for calculating the confidence score for each character and wherein the performance monitoring module further comprises a means for averaging the confidence scores of all characters read over a last set number of end user documents.

22. The system of claim 17 further comprising means for determining a threshold confidence score, wherein the threshold confidence score represents a value greater than a minimum level of acceptable performance for the character recognition system and less than an expected level of performance for a new character recognition system.

23. The system of claim 20, wherein the performance monitoring module further comprises a comparator for comparing the average confidence score to the threshold confidence score.

24. The system of claim 21, wherein the performance monitoring module further comprises an alarm indicating that the character recognition system is in need of service if the average confidence score falls below the threshold confidence score and a means for resetting the average confidence score calculation after the system has been serviced.

19. A system for monitoring the performance of a character recognition system, comprising:

means for reading a plurality of characters recorded on at least one end user document; and

a performance monitoring module for generating an average confidence score for the plurality of characters, wherein a confidence score indicates a level of confidence that a character is accurately recognized, and for utilizing the average confidence score for ongoing performance monitoring of the character recognition system.

25. A method for monitoring the performance of a character recognition system, comprising the step of:

a) utilizing a median confidence score for a plurality of characters for ongoing performance monitoring of the character recognition system, wherein a confidence score indicates a level of confidence that a character is accurately recognized.

26. The method of claim 25 further comprising the step of:

b) generating the median confidence score from the plurality of characters recorded on at least one end user document prior to the utilizing step (a).

27. The method of claim 26, wherein generating step (b) further comprises

(b1) reading a character recorded on an end user document;

(b2) calculating the confidence score for the character;

(b3) repeating steps (b1) and (b2) for each of a plurality of characters recorded on the end user document; and

(b4) determining the median confidence score from the confidence score of the characters and the confidence scores of all characters read over a last set number of end user documents.

28. The method of claim 25, wherein the utilizing step (a) further comprising the step of:

(a1) determining a threshold confidence score, wherein the threshold confidence score represents a value greater than a minimum level of acceptable performance for the character recognition system and less than an expected level of performance for a new character recognition system.

29. The method of claim 28, wherein the utilizing step (a) further comprises:

(a2) comparing the median confidence score to the threshold confidence score.

30. The method of claim 29, wherein the utilizing step (a) further comprising the step of:

(a3) indicating that the character recognition system is in need of service if the median confidence score falls below the threshold confidence score.