SYSTEM AND METHOD OF HANDWRITING RECOGNITION IN DIAGRAMS

• US 20170109578A1
• Filed: 12/01/2015
• Published: 04/20/2017
• Est. Priority Date: 10/19/2015
• Status: Active Grant

• Current Assignee: Myscript
• Original Assignee: Myscript
• Inventors: HEBERT, David, VALENTE, Fabio, COUSIN, Gregory, NAGOT, Mal, BEDNAROWICZ, Romain, MLINAND, Robin, SIDOLI, Claire, RIC, Fabien, ELAGOUNI, Khaoula, CEROVIC, Cyril
• Granted Patent: US 10,643,067 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/353 into predefined classes
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06F 40/205 Parsing
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: G06V 30/32 Digital ink
• CPC Class Codes: G06V 30/347 Sampling; Contour coding; Stroke extraction
• CPC Class Codes: G06V 30/387 using human interaction, e.g. selection of the best displayed recognition candidate
• CPC Class Codes: G06V 30/422 Technical drawings; Geographical maps

Abstract

A system, method and computer program product for hand-drawing diagrams including text and non-text elements on a computing device are provided. The computing device has a processor and a non-transitory computer readable medium for detecting and recognizing hand-drawing diagram element input under control of the processor. Display of input diagram elements in interactive digital ink is performed on a display device associated with the computing device. One or more of the diagram elements are associated with one or more other of the diagram elements in accordance with a class and type of each diagram element. The diagram elements are re-displayed based on one or more interactions with the digital ink received and in accordance with the one or more associations.

Claims

1. A system for hand-drawing diagrams including text and non-text elements on a computing device, the computing device comprising a processor and at least one non-transitory computer readable medium for detecting and recognizing hand-drawing diagram element input under control of the processor, the at least one non-transitory computer readable medium configured to:

cause display of a plurality of input diagram elements in interactive digital ink on a display device associated with the computing device;

associate one or more of the diagram elements with one or more other of the diagram elements in accordance with a class and type of each diagram element; and

cause re-display of the diagram elements based on one or more interactions with the digital ink received and in accordance with the one or more associations.

2. A system as claimed in claim 1, the at least one non-transitory computer readable medium configured to:

identify the class of each diagram element by classifying strokes of the hand-drawn input.

3. A system as claimed in claim 2, wherein the strokes are grouped based on spatial and temporal information of the input.

4. A system as claimed in claim 2, wherein the strokes are classified by building and testing element type probability hypotheses for the groups of strokes.

5. A system as claimed in claim 1, wherein the identified types include text and non-text.

6. A system as claimed in claim 1, the at least one non-transitory computer readable medium configured to:

parse the classified strokes to a handwriting recognition system for recognition of the diagram elements of the classified strokes.

7. A system as claimed in claim 6, the at least one non-transitory computer readable medium configured to:

identify the type of each diagram element based on the recognized diagram elements and positional relationships between the diagram elements.

8. A method for hand-drawing diagrams including text and non-text elements on a computing device, the computing device comprising a processor and at least one non-transitory computer readable medium for detecting and recognizing hand-drawing diagram element input under control of the processor, the method comprising:

displaying a plurality of input diagram elements in interactive digital ink on a display device associated with the computing device;

associating one or more of the diagram elements with one or more other of the diagram elements in accordance with a class and type of each diagram element; and

re-displaying the diagram elements based on one or more interactions with the digital ink received and in accordance with the one or more associations.

9. A method as claimed in claim 8, comprising identifying the class of each diagram element by classifying strokes of the hand-drawn input.

10. A method as claimed in claim 9, wherein the strokes are grouped based on spatial and temporal information of the input.

11. A method as claimed in claim 9, wherein the strokes are classified by building and testing element type probability hypotheses for the groups of strokes.

12. A method as claimed in claim 8, wherein the identified types include text and non-text.

13. A method as claimed in claim 8, comprising parsing the classified strokes to a handwriting recognition system for recognition of the diagram elements of the classified strokes.

14. A method as claimed in claim 13, comprising identifying the type of each diagram element based on the recognized diagram elements and positional relationships between the diagram elements.

15. A non-transitory computer readable medium having a computer readable program code embodied therein, said computer readable program code adapted to be executed to implement a method for hand-drawing diagrams including text and non-text elements on a computing device, the computing device comprising a processor and at least one non-transitory computer readable medium for detecting and recognizing hand-drawing diagram element input under control of the processor, the method comprising:

displaying a plurality of input diagram elements in interactive digital ink on a display device associated with the computing device;

associating one or more of the diagram elements with one or more other of the diagram elements in accordance with a class and type of each diagram element; and

re-displaying the diagram elements based on one or more interactions with the digital ink received and in accordance with the one or more associations.

16. A non-transitory computer readable medium as claimed in claim 15, the method comprising identifying the class of each diagram element by classifying strokes of the hand-drawn input.

17. A non-transitory computer readable medium as claimed in claim 16, wherein the strokes are grouped based on spatial and temporal information of the input.

18. A non-transitory computer readable medium as claimed in claim 16, wherein the strokes are classified by building and testing element type probability hypotheses for the groups of strokes.

19. A non-transitory computer readable medium as claimed in claim 15, wherein the identified types include text and non-text.

20. A non-transitory computer readable medium as claimed in claim 15, the method comprising parsing the classified strokes to a handwriting recognition system for recognition of the diagram elements of the classified strokes.

21. A non-transitory computer readable medium as claimed in claim 20, the method comprising identifying the type of each diagram element based on the recognized diagram elements and positional relationships between the diagram elements.