System and method for superimposed handwriting recognition technology

• US 9,911,052 B2
• Filed: 11/16/2016
• Issued: 03/06/2018
• Est. Priority Date: 04/04/2014
• Status: Active Grant

• Current Assignee: Myscript
• Original Assignee: Myscript
• Inventors: Wimmer, Zsolt, Perraud, Freddy, Lallican, Pierre-Michel
• Primary Examiner(s): Akhavannik, Hadi
• Application Number:
  US15/353,214
• Publication Number:
  US 20170061223A1
• Time in Patent Office: 475 Days
• Field of Search: None
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06T 5/70 Denoising; Smoothing
• CPC Class Codes: G06T 5/80 Geometric correction
• CPC Class Codes: G06T 7/10 Segmentation; Edge detection motion-based segmentation G06T7/215
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06V 30/2268 using stroke segmentation
• CPC Class Codes: G06V 30/274 Syntactic or semantic context, e.g. balancing
• CPC Class Codes: G06V 30/333 Preprocessing; Feature extraction
• CPC Class Codes: G06V 30/36 Matching; Classification

Abstract

A system and method is able to recognize a user'"'"'s natural superimposed handwriting without any explicit separation between characters. The system and method is able to process single-stroke and multi-stroke characters. The system and method can also process cursive handwriting. Further, the system and method can determine the boundaries of input words either by the use of a specific user input gesture or by detecting the word boundaries based on language characteristics and properties. The system and method analyzes the handwriting input through the processes of segmentation, character recognition, and language modeling. These three processes occur concurrently through the use of dynamic programming.

Claims

1. A non-transitory computer readable medium having a computer readable program code embodied therein, said computer readable program code adapted to be executed to implement a method for providing handwriting recognition for superimposed stroke input, said method comprising:

creating, with a segmentation expert, a segmentation graph based on a plurality of input strokes, at least two of the strokes being at least partially superimposed on one another, wherein the segmentation graph consists of nodes and paths corresponding to character hypotheses formed by segmenting the input strokes to take into account the at least partially superimposed strokes;

assigning, with a recognition expert, a recognition score to each node of the segmentation graph based on language recognition information;

generating, with a language expert, linguistic meaning of the input strokes by optimizing the recognition scores of the node paths of the segmentation graph based on a language model; and

providing an output based on the collaborative analysis of the segmentation graph, the recognition score, and the language model by the segmentation, recognition and language experts.

2. A non-transitory computer readable medium according to claim 1, wherein the input stroke is preprocessed, wherein the preprocessing includes at least the normalization and smoothing of the input stroke.

3. A non-transitory computer readable medium according to claim 2, wherein the segmentation graph is based on continuous input strokes that have been broken into constituting segments.

4. A non-transitory computer readable medium according to claim 2, wherein the assigning of a recognition score comprises a feature extraction stage and a classification of features extracted by neural networks.

5. A non-transitory computer readable medium according to claim 4, wherein the feature extraction stage comprises at least one dynamic feature and at least one static feature.

6. A non-transitory computer readable medium according to claim 4, wherein the neural networks are multilayer perceptrons.

7. A non-transitory computer readable medium according to claim 1, wherein the language model includes linguistic information specific to one or more languages.

8. A non-transitory computer readable medium according to claim 1, wherein the generation of the linguistic meaning includes recognizing word boundaries in the superimposed input based on the language model.

9. A method for providing handwriting recognition for a superimposed input stroke, said method comprising:

creating, with a segmentation expert, a segmentation graph based on a plurality of input strokes, at least two of the strokes being at least partially superimposed on one another, wherein the segmentation graph consists of nodes and paths corresponding to character hypotheses formed by segmenting the input strokes to take into account the at least partially superimposed strokes;

assigning, with a recognition expert, a recognition score to each node of the segmentation graph based on language recognition information;

generating, with a language expert, linguistic meaning of the input strokes by optimizing the recognition scores of the node paths of the segmentation graph based on a language model; and

providing an output based on the collaborative analysis of the segmentation graph, the recognition score, and the language model by the segmentation, recognition and language experts.

10. A method according to claim 9, wherein the input stroke is preprocessed, wherein the preprocessing includes at least the normalization and smoothing of the input stroke.

11. A method according to claim 10, wherein the segmentation graph is based on continuous input strokes that have been broken into constituting segments.

12. A method according to claim 10, wherein the assigning of a recognition score comprises a feature extraction stage and a classification of features extracted by neural networks.

13. A method according to claim 12, wherein the feature extraction stage comprises at least one dynamic feature and at least one static feature.

14. A method according to claim 12, wherein the neural networks are multilayer perceptrons.

15. A method according to claim 9, wherein the language model includes linguistic information specific to one or more languages.

16. A method according to claim 9, wherein the generation of the linguistic meaning includes recognizing word boundaries in the superimposed input based on the language model.

17. A system for providing handwriting recognition for a superimposed stroke input to a computing device, the computing device comprising a processor and at least one computer readable program for recognizing the input under control of the processor, said at least one program configured to:

create, with a segmentation expert, a segmentation graph based on a plurality of input strokes, at least two of the strokes being at least partially superimposed on one another, wherein the segmentation graph consists of nodes and paths corresponding to character hypotheses formed by segmenting the input strokes to take into account the at least partially superimposed strokes;

assign, with a recognition expert, a recognition score to each node of the segmentation graph based on language recognition information;

generate, with a language expert, linguistic meaning of the input strokes by optimizing the recognition scores of the node paths of the segmentation graph based on a language model; and

provide an output based on the collaborative analysis of the segmentation graph, the recognition score, and the language model by the segmentation, recognition and language experts.

18. A system according to claim 17, wherein the segmentation graph is based on continuous input strokes that have been broken into constituting segments.

19. A system according to claim 17, wherein the assigning of a recognition score comprises a feature extraction stage and a classification of features extracted by neural networks.

20. A system according to claim 19, wherein the feature extraction stage comprises at least one dynamic feature and at least one static feature.

21. A system according to claim 19, wherein the neural networks are multilayer perceptrons.

22. A system according to claim 17, wherein the language model includes linguistic information specific to one or more languages.

23. A system according to claim 17, wherein the generation of the linguistic meaning includes recognizing word boundaries in the superimposed input based on the language model.