Method and apparatus for computerized recognition

• US 5,592,566 A
• Filed: 06/01/1995
• Issued: 01/07/1997
• Est. Priority Date: 05/27/1992
• Status: Expired due to Term

• Current Assignee: Apple Inc.
• Original Assignee: Apple Computer Incorporated (Apple Inc.)
• Inventors: Capps, Stephen P., Pagallo, Giulia, Beernink, Ernest H., Tchao, Michael C.
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Tadayon, Bijan
• Application Number:
  US08/456,554
• Time in Patent Office: 586 Days
• Field of Search: 382/100, 382/209, 382/218, 382/229, 382/187, 382/282, 382/309, 382/317, 382/119, 382/135, 382/137, 382/138, 382/176, 382/177, 382/178, 382/179, 382/185, 382/186, 382/188 , 382/189, 382/190, 382/192, 382/193, 382/194, 382/197, 382/198, 382/199, 382/200, 382/201, 382/202, 382/203, 382/204, 382/205, 382/217, 382/224, 382/227, 382/228 , 382/312, 382/313, 382/314, 382/321, 358/505, 358/530, 358/540, 358/400, 358/401, 358/403, 358/424, 358/453, 358/462, 358/470
• US Class Current: 382/187
• CPC Class Codes: G06V 30/373 using a special pattern or subpattern alphabet

Abstract

A recognition system for a pen-based computer system characterized by: a tablet for inputting strokes from a stylus; a low-level recognizer responsive to the strokes and operative to perform low-level recognition on the strokes; a high-level recognizer responsive to at least some of the strokes and operative to perform high-level recognition on those strokes; and a recognition interface permitting communication between the recognizers and an application program. The high-level recognizer includes a controller capable of receiving units to be recognized; a number of recognition domains coupled to the controller, where each of the recognition domains can receive one or more units from the controller and is capable of providing an interpretation of the meaning of the one or more units to the controller; and an arbiter coupled to the controller which is capable of resolving conflicts between multiple interpretations developed by the recognition domains. A method implemented by the high-level recognition system includes the steps of creating a stroke object from a stroke generated by the stylus on the tablet; grouping the stroke object with at least one other stroke object, if possible; classifying the stroke object by passing the stroke object to a number of recognition domains; and arbitrating between conflicting classifications generated by the recognition domains. This permits unstructured, free-form inputs to a pen-based computer system to be recognized and acted upon by the system.

Claims

1. A recognition system for a pen-based computer system having a stylus, a processing means capable of running an application program, and a display suitable for displaying an output directed by said application program, the recognition system comprising:

tablet means for inputting strokes from a stylus;

low level recognizer means responsive to said strokes inputted through said tablet means and operative to perform low level recognition on said strokes, including the recognition of a plurality of gestures, the low level recognizer means being capable of running on said processing means;

high level recognizer means independent of the low level recognizer means and responsive to stroke objects that are not recognized by said low level recognizer and operative to perform high level recognition on strokes that are not recognized by said low level recognizer, the high level recognizer means being capable of running on said processing means; and

a recognition interface permitting independent communication between said low level recognizer means and an application program, and permitting independent communication between said high level recognizer and said application program to permit the application program to utilize recognized information to direct the output to said display;

whereby processing efficiency is increased by permitting the low level recognizer to recognize certain stroke objects and not requiring stroke objects recognized by the low level recognizer to be handled by the high level recognizer and thereby avoiding at least some preprocessing that is required by the high level recognizer.

2. A recognition system as recited in claim 1 wherein said low level recognizer means recognizes gestures made with said stylus on said tablet means.

3. A recognition system as recited in claim 1 wherein said high level recognizer means comprises:

controller means; and

a plurality of recognition domains coupled to said controller means.

4. A recognition system as recited in claim 3 wherein said high level recognizer further includes arbitration means coupled to said controller means for resolving multiple interpretations provided by said plurality of recognition domains.

5. A recognition system as recited in claim 4 wherein said high-level recognizer further includes database means coupled to said controller means.

6. A recognition system as recited in claim 5 wherein said high-level recognizer further includes recognition area means coupled to said controller means.

7. A recognition system as recited in claim 4 further comprising stroke unit forming means coupled to said tablet means, wherein said controller means is responsive to stroke units formed by said stroke unit forming means.

8. A recognition system as recited in claim 7 wherein said recognition domains produce recognized units.

9. A recognition system as recited in claim 8 wherein said recognition interface is receptive to stroke units and recognized units from said high-level recognizer means.

10. A recognition system as recited in claim 9 wherein said high-level recognizer means is receptive to returned units and to area registration information from said recognition interface.

11. A recognition system for use in a computer system having a processor arrangement capable of running an application program, output means for outputting symbols directed by the application program, and input means for receiving input signals to be recognized, the recognition system comprising:

a controller capable of receiving units to be recognized, the units for recognition being based at least in part on received input signals;

a plurality of independent recognition domain means, each having a data portion and a procedures portion wherein the procedures portion communicates with the controller and has access to the associated data portion, and wherein each of said recognition domain means can receive one or more units from said controller and is capable of providing an interpretation of said one or more received units to said controller; and

arbitration means coupled to said controller, the arbitration means being capable of resolving conflicts between multiple interpretations developed by the independent recognition domain means and selecting a preferred interpretation and providing an indication of the preferred interpretation to the controller, and wherein the controller is arranged to output a preferred interpretation to an application program thereby permitting the application program to utilize recognized information to direct the output of said output means.

12. A recognition system as recited in claim 11 wherein:

the controller is capable of determining whether the received units need grouping;

said recognition system further includes database means coupled to said controller, said data base means including means for grouping and means for classifying received units when the controller determines that the received units need grouping, grouping the received units and passing the grouped units through the independent recognition domains and then placing the grouped units into a classification pool means.

13. A recognition system as recited in claim 11 wherein said recognition system further includes recognition area means coupled to said controller and said recognition area means representing physical areas on a tablet means identified for recognition, and wherein overlapped recognition areas define merged recognition areas capable of recognizing information in each of the overlapped recognition areas.

14. A recognition system as recited in claim 11 Wherein said controller means is receptive to stroke units derived from a tablet means.

15. A recognition system as recited in claim 14 wherein said recognition domains produce recognized units.

16. A recognition system as recited in claim 15 wherein said arbitration means is operative to pass stroke units and recognized units.

17. A recognition system as recited in claim 15 wherein arbitration means is receptive to returned units from an application program, and wherein said controller means is receptive to area registration information from said application program.

18. A computer implemented method for recognizing strokes generated by a stylus on a tablet in a computer system having a processor that runs an application program, the method comprising the steps of:

creating a stroke object from a stroke generated by a stylus on a tablet;

attempting to group said stroke object with at least one other stroke object, where in the step of attempting to group said stroke object is accomplished by passing said stroke object to through a plurality of independent recognition domains, and each independent recognition domain having a data portion and a procedures portion;

recognizing said stroke object by passing said stroke object to said plurality of independent recognition domains, wherein said independent recognition domains each return an associated recognition result;

arbitrating between conflicting recognition results generated by said independent recognition domains to select a particular one of the recognition results as a preferred recognition result; and

passing the preferred recognition result to the application program for use by the application program in directing an output to a display.

19. A method as recited in claim 18 wherein said step of attempting to group said stroke object is accomplished by passing said stroke object to said plurality of independent recognition domains in order to recognize said stroke object comprising a plurality of alphanumeric characters made with said stylus on said display means.

20. A method as recited in claim 19 wherein said step of arbitrating is accomplished using a decision theory analysis.

21. A method as recited in claim 20 wherein said decision theory analysis is an expert system analysis.

24. A computer implemented method as recited in claim 18, wherein said method further includes the step of defining a plurality of recognition areas being coupled to said processor, and said plurality of recognition areas representing physical areas on the tablet adapted to receive stroke objects, and wherein overlapped recognition areas define merged recognition areas capable of recognizing stroke objects in each of the overlapped recognition areas.

22. A computer system comprising:

a processor capable of running an application program;

a display for displaying symbols generated by the application program;

a stylus for inputting handwritten strokes on the display means;

a stroke object generator for digitizing strokes input by the stylus onto the display;

low level recognizer means responsive to said stroke objects and operative to perform low level recognition on said stroke objects wherein said low level recognizer means recognizes a plurality of gestures made with said stylus on the display means, the low level recognizer means being arranged to run on said processing means;

high level recognizer means independent of the low level recognizer means and responsive to stroke objects that are not recognized by said low level recognizer and operative to perform high level recognition on strokes that are not recognized by said low level recognizer, wherein the high level recognizer means recognizes a plurality of alphanumeric characters made with said stylus on said display means, the high level recognizer means being arranged to run on said processing means and including,

controller means capable of receiving units to be recognized, the units for recognition being based at least in part on the stroke objects,

a plurality of recognition domain means coupled to said controller means, each of the recognition domain means being arranged to receive one or more units from said controller means and being capable of providing an interpretation of said one or more units to said controller means, and

arbitration means coupled to said controller means for resolving conflicts between multiple interpretations developed by distinct ones of said recognition domain means and selecting a preferred interpretation;

a recognition interface that facilitates independent communication between the low level recognizer means and the application program, and facilitates independent communication between the high level recognizer and the application program to permit the application program to utilize recognized information to direct an output to said display; and

whereby processing efficiency is increased by permitting the low level recognizer to recognize certain stroke objects and not requiring stroke objects recognized by the low level recognizer to be handled by the high level recognizer and thereby avoiding at least some preprocessing that is required by the high level recognizer.

23. A computer implemented method for recognizing strokes in a pen based computer system that are generated by a stylus on a display in the computer system, the computer system further having a processor that runs an application program, the method comprising the steps of:

creating a stroke object from a stroke generated by the stylus on the display;

determining whether the stroke object represents a gesture input by the stylus and passing an indication of the determined gesture input to the application program when a gesture is determined;

passing a stroke unit that is based at least in part on the stroke object to a plurality of distinct recognition domains when the stroke object is not determined to be a gesture, wherein said distinct recognition domains each return an associated recognition result when passed a stroke unit for recognition;

arbitrating between conflicting recognition results generated by said recognition domains when a conflict occurs therebetween to select a particular one of the recognition results as a preferred recognition result;

passing the preferred recognition result to the application program; and

utilizing the application program to direct an output to the display based at least in part on the identified gesture when the stroke object is determined to be a gesture, and based at least in part on the preferred recognition result when a preferred recognition result is passed to the application program.