STROKE EXTRACTION IN FREE SPACE

• US 20170154230A1
• Filed: 11/30/2015
• Published: 06/01/2017
• Est. Priority Date: 11/30/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Dow, Eli M., Fitzsimmons, Thomas D., Garrett, Tynan J., Metruck, Emily M.
• Granted Patent: US 10,169,670 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 40/126 Character encoding
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/2276 with probabilistic networks, e.g. hidden Markov models
• CPC Class Codes: G06V 30/36 Matching; Classification

Abstract

An approach for stroke extraction in free space utilizing a paired ring device is provided. The approach receives one or more images transmitted from the paired ring device, wherein the one or more images are transcribed sequentially from data related to one or more movements recorded by the paired ring device, and wherein the one or more images include one or more of a plurality of vector points, a plurality of coordinates, and a plurality of dots interconnected by a plurality of lines. The approach inputs the one or more images into a character training model. The approach maps the one or more images into one or more characters. The approach transcribes the one or more characters into a digital document.

Claims

1. A method for stroke extraction in free space utilizing a paired ring device, the method comprising:

receiving, by one or more computer processors, one or more images transmitted from the paired ring device, wherein the one or more images are transcribed sequentially from data related to one or more movements recorded by the paired ring device, and wherein the one or more images include one or more of a plurality of vector points, a plurality of coordinates, and a plurality of dots interconnected by a plurality of lines;

inputting, by one or more computer processors, the one or more images into a character training model;

mapping, by one or more computer processors, the one or more images into one or more characters; and

transcribing, by one or more computer processors, the one or more characters into a digital document.

2. The method of claim 1, further comprises:

wherein the paired ring device includes an accelerometer and a gyroscope for monitoring the data related to the one or more movements; and

wherein the data related to the one or more movements includes one or more of a change in a magnitude value and a direction value of proper acceleration of the paired ring device, an orientation of the paired ring device, a position of the paired ring device, a location of the paired ring device, a change in the orientation of the paired ring device, a change in the position of the paired ring device, and a change in the location of the paired ring device, in free space.

3. The method of claim 1, wherein inputting the one or more images into a character training model, further comprises:

wherein the character training model is a character recognition model specifically trained utilizing a neural network to recognize which of the one or more movements map to which of the one or more characters.

4. The method of claim 3, wherein the character training model may be a character recognition model initially trained utilizing handwritten character recognition (ICR).

5. The method of claim 1, wherein mapping the one or more images into one or more characters, further comprises:

determining, by one or more computer processors, whether the one or more images indicate a written language, a specific character, or a gesture output utilizing one or more changes in data recorded on the ring device.

6. The method of claim 5, wherein determining whether the one or more images indicate a written language, a specific character, or a gesture output utilizing one or more changes in data recorded on the ring device, further comprises:

responsive to determining that the one or more images indicate a written language, a specific character, or a gesture output, accumulating, by one or more computer processors, the one or more characters into one or more character streams, wherein the one or more character streams include at least one of one or more sentences, the one or more characters, one or more paragraphs, and one or more keywords; and

associating, by one or more computer processors, the one or more character streams, based, at least in part, on character recognition results from the character training model, with a time stamp and identifiers to indicate a textual representation of a written language, a specific character, or a gesture output.

7. The method of claim 1, wherein transcribing the one or more characters into a digital document on the one or more paired devices, further comprises:

determining, by one or more computer processors, a representation of the one or more characters as a written language or a gesture output derived from a series of strokes in free space; and

encoding, by one or more computer processors, the one or more characters into a visual augmentation.

8. The method of claim 1, further comprises:

displaying, by one or more computer processors, the digital document on the one or more paired devices utilizing an interactive user interface, wherein the interactive user interface provides the capability to view the one or more characters in real time.

9. A computer program product for stroke extraction in free space utilizing a paired ring device, the computer program product comprising:

one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions comprising;

program instructions to receive one or more images transmitted from the paired ring device, wherein the one or more images are transcribed sequentially from data related to one or more movements recorded by the paired ring device, and wherein the one or more images include one or more of a plurality of vector points, a plurality of coordinates, and a plurality of dots interconnected by a plurality of lines;

program instructions to input the one or more images into a character training model;

program instructions to map the one or more images into one or more characters; and

program instructions to transcribe the one or more characters into a digital document.

10. The computer program product of claim 9, further comprises:

wherein the paired ring device includes an accelerometer and a gyroscope for monitoring the data related to the one or more movements; and

wherein the data related to the one or more movements includes one or more of a change in a magnitude value and a direction value of proper acceleration of the paired ring device, an orientation of the paired ring device, a position of the paired ring device, a location of the paired ring device, a change in the orientation of the paired ring device, a change in the position of the paired ring device, and a change in the location of the paired ring device, in free space.

11. The computer program product of claim 9, wherein program instructions to input the one or more images into a character training model, further comprises:

wherein the character training model is a character recognition model specifically trained utilizing a neural network to recognize which of the one or more movements map to which of the one or more characters.

12. The computer program product of claim 11, wherein the character training model may be a character recognition model initially trained utilizing handwritten character recognition (ICR).

13. The computer program product of claim 9, wherein program instructions to map the one or more images into one or more characters, further comprises:

program instructions to determine whether the one or more images indicate a written language, a specific character, or a gesture output utilizing one or more changes in data recorded on the ring device;

14. The computer program product of claim 13, wherein program instructions to determine whether the one or more images indicate a written language, a specific character, or a gesture output utilizing one or more changes in data recorded on the ring device, further comprises:

responsive to determining that the one or more images indicate a written language, a specific character, or a gesture output, program instructions to accumulate the one or more characters into one or more character streams, wherein the one or more character streams include at least one of one or more sentences, the one or more characters, one or more paragraphs, and one or more keywords; and

program instructions to associate the one or more character streams, based, at least in part, on character recognition results from the character training model, with a time stamp and identifiers to indicate a textual representation of a written language, a specific character, or a gesture output.

15. The computer program product of claim 9, wherein program instructions to transcribe the one or more characters into a digital document on the one or more paired devices, further comprises:

program instructions to determine a representation of the one or more characters as a written language or a gesture output derived from a series of strokes in free space; and

program instructions to encode the one or more characters into a visual augmentation.

16. The computer program product of claim 9, further comprises:

program instructions to display the digital document on the one or more paired devices utilizing an interactive user interface, wherein the interactive user interface provides the capability to view the one or more characters in real time.

17. A computer system for stroke extraction in free space utilizing a paired ring device, the computer system comprising:

one or more computer readable storage media;

program instructions stored on at least one of the one or more computer readable storage media for extraction by at least one of the one or more computer processors, the program instructions comprising;

program instructions to receive one or more images transmitted from the paired ring device, wherein the one or more images are transcribed sequentially from data related to one or more movements recorded by the paired ring device, and wherein the one or more images include one or more of a plurality of vector points, a plurality of coordinates, and a plurality of dots interconnected by a plurality of lines;

program instructions to input the one or more images into a character training model;

program instructions to map the one or more images into one or more characters; and

program instructions to transcribe the one or more characters into a digital document.

18. The computer system of claim 17, further comprises:

wherein the paired ring device includes an accelerometer and a gyroscope for monitoring the data related to the one or more movements; and

wherein the data related to the one or more movements includes one or more of a change in a magnitude value and a direction value of proper acceleration of the paired ring device, an orientation of the paired ring device, a position of the paired ring device, a location of the paired ring device, a change in the orientation of the paired ring device, a change in the position of the paired ring device, and a change in the location of the paired ring device, in free space.

19. The computer system of claim 17, wherein program instructions to input the one or more images into a character training model, further comprises:

wherein the character training model is a character recognition model specifically trained utilizing a neural network to recognize which of the one or more movements map to which of the one or more characters.

20. The computer system of claim 19, wherein the character training model may be a character recognition model initially trained utilizing handwritten character recognition (ICR).