System and method for superimposed handwriting recognition technology

• US 9,524,440 B2
• Filed: 04/04/2014
• Issued: 12/20/2016
• Est. Priority Date: 04/04/2014
• Status: Active Grant

• Current Assignee: Myscript
• Original Assignee: Myscript
• Inventors: Wimmer, Zsolt, Perraud, Freddy, Lallican, Pierre-Michel
• Primary Examiner(s): Akhavannik, Hadi
• Application Number:
  US14/245,601
• Publication Number:
  US 20150286886A1
• Time in Patent Office: 991 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06V 30/2268 using stroke segmentation
• CPC Class Codes: G06V 30/274 Syntactic or semantic context, e.g. balancing
• CPC Class Codes: G06V 30/333 Preprocessing; Feature extraction
• CPC Class Codes: G06V 30/36 Matching; Classification

Abstract

A system and method that is able to recognize a user'"'"'s natural superimposed handwriting without any explicit separation between characters. The system and method is able to process single-stroke and multi-stroke characters. It can also process cursive handwriting. Further, the method and system can determine the boundaries of input words either by the use of a specific user input gesture or by detecting the word boundaries based on language characteristics and properties. The system and method analyzes the handwriting input through the processes of segmentation, character recognition, and language modeling. These three processes occur concurrently through the use of dynamic programming.

Claims

1. A non-transitory computer readable medium having a computer readable program code embodied therein, said computer readable program code adapted to be executed to implement a method for providing handwriting recognition for superimposed stroke input, said method comprising:

creating a segmentation graph based on a plurality of input strokes, at least two of the strokes being at least partially superimposed on one another, wherein the segmentation graph consists of nodes and paths corresponding to character hypotheses formed by segmenting the input strokes to take into account the at least partially superimposed strokes;

assigning a recognition score to each node of the segmentation graph based on language recognition information;

generating linguistic meaning of the input strokes by optimizing the recognition scores of the node paths of the segmentation graph based on a language model; and

providing an output based on the simultaneous analysis of the segmentation graph, the recognition score, and the language model.

2. The non-transitory computer readable medium according to claim 1, wherein the input stroke is preprocessed, wherein the preprocessing includes at least the normalization and smoothing of the input stroke.

3. The non-transitory computer readable medium according to claim 2, wherein the segmentation graph is based on continuous input strokes that have been broken into constituting segments.

4. The non-transitory computer readable medium according to claim 2, wherein the assigning of a recognition score comprises a feature extraction stage and a classification of features extracted by neural networks.

5. The non-transitory computer readable medium according to claim 4, wherein the feature extraction stage comprises at least one dynamic feature and at least one static feature.

6. The non-transitory computer readable medium according to claim 4, wherein the neural networks are multilayer perceptrons.

18. The non-transitory computer readable medium according to claim 1, wherein the language model includes linguistic information specific to one or more languages.

21. The non-transitory computer readable medium according to claim 1, wherein the generation of the linguistic meaning includes recognizing word boundaries in the superimposed input based on the language model.

7. A method for providing handwriting recognition for a superimposed input stroke, said method comprising:

creating a segmentation graph based on a plurality of input strokes, at least two of the strokes being at least partially superimposed on one another, wherein the segmentation graph consists of nodes and paths corresponding to character hypotheses formed by segmenting the input strokes to take into account the at least partially superimposed strokes;

assigning a recognition score to each node of the segmentation graph based on language recognition information;

generating linguistic meaning of the input strokes by optimizing the recognition scores of the node paths of the segmentation graph based on a language model; and

providing an output based on the simultaneous analysis of the segmentation graph, the recognition score, and the language model.

8. A method according to claim 7, wherein the input stroke is preprocessed, wherein the preprocessing includes at least the normalization and smoothing of the input stroke.

9. A method according to claim 8, wherein the segmentation graph is based on continuous input strokes that have been broken into constituting segments.

10. A method according to claim 8, wherein the assigning of a recognition score comprises a feature extraction stage and a classification of features extracted by neural networks.

11. A method according to claim 10, wherein the feature extraction stage comprises at least one dynamic feature and at least one static feature.

12. A method according to claim 10, wherein the neural networks are multilayer perceptrons.

19. A method according to claim 7, wherein the language model includes linguistic information specific to one or more languages.

22. A method according to claim 7, wherein the generation of the linguistic meaning includes recognizing word boundaries in the superimposed input based on the language model.

13. A system for providing handwriting recognition for a superimposed stroke input to a computing device, the computing device comprising a processor and at least one non-transitory computer readable medium for recognizing the input under control of the processor, said at least one program configured to:

create a segmentation graph based on a plurality of input strokes, at least two of the strokes being at least partially superimposed on one another, wherein the segmentation graph consists of nodes and paths corresponding to character hypotheses formed by segmenting the input strokes to take into account the at least partially superimposed strokes;

assign a recognition score to each node of the segmentation graph based on language recognition information;

generate linguistic meaning of the input strokes by optimizing the recognition scores of the node paths of the segmentation graph based on a language model; and

provide an output based on the simultaneous analysis of the segmentation graph, the recognition score, and the language model.

14. A system according to claim 13, wherein the segmentation graph is based on continuous input strokes that have been broken into constituting segments.

15. A system according to claim 13, wherein the assigning of a recognition score comprises a feature extraction stage and a classification of features extracted by neural networks.

16. A system according to claim 15, wherein the feature extraction stage comprises at least one dynamic feature and at least one static feature.

17. A system according to claim 15, wherein the neural networks are multilayer perceptrons.

20. A system according to claim 13, wherein the language model includes linguistic information specific to one or more languages.

23. A system according to claim 13, wherein the generation of the linguistic meaning includes recognizing word boundaries in the superimposed input based on the language model.