Method and apparatus for on-top writing

• US 8,884,905 B2
• Filed: 10/12/2012
• Issued: 11/11/2014
• Est. Priority Date: 05/13/2010
• Status: Active Grant

• Current Assignee: Cerence Operating Company (Cerence Inc.)
• Original Assignee: Nuance Communications, Inc. (Microsoft Corporation)
• Inventors: Andersson, Jonas, Morwig, Jonas
• Primary Examiner(s): SHERMAN, STEPHEN G
• Application Number:
  US13/651,177
• Publication Number:
  US 20130034303A1
• Time in Patent Office: 760 Days
• Field of Search: 345156-179, 178 1801- 1809, 178/18.11, 178 1901- 1907
• US Class Current: 345/173
• CPC Class Codes: G06F 3/0237 using prediction or retrieval techniques
• CPC Class Codes: G06F 3/0354 with detection of 2D relative movements between the device, or an operating part thereof, and a plane or surface, e.g. 2D mice, trackballs, pens or pucks
• CPC Class Codes: G06F 3/04883 for inputting data by handwriting, e.g. gesture or text
• CPC Class Codes: G06V 10/426 Graphical representations
• CPC Class Codes: G06V 30/1423 the instrument generating sequences of position coordinates corresponding to handwriting preprocessing or recognising digital ink G06V30/32
• CPC Class Codes: G06V 30/32 Digital ink
• CPC Class Codes: G06V 30/387 using human interaction, e.g. selection of the best displayed recognition candidate

Abstract

A handwriting recognition apparatus facilitates user entry of strokes one on top of another. The apparatus, which includes a processor and a display integrated with a touch sensitive screen, receives a series of strokes via the screen. Each stroke is defined by contact, trace, and lift occurrences. Each stroke appears on the display until occurrence of a prescribed event, and then disappears. The apparatus accumulates strokes into a buffer and interprets all accumulated strokes collectively against a character database and optionally a linguistic database, to identify multiple candidate strings that could be represented by the accumulated strokes. The apparatus displays candidate strings for user selection after all strokes have faded, or after receiving a user submitted delimiter, or after a given delay has elapsed following user entry of the latest stroke. Alternatively, candidate strings are displayed after each stroke without waiting for timeout or explicit delimiter.

Claims

1. A handwriting recognition apparatus, comprising:

an electrically driven display integrated with a touch sensitive screen;

machine-readable storage containing any of;

a character database representing predefined character shape data, the character database and a linguistic database representing predefined character strings; and

a processor coupled to the display and storage, the processor programmed to perform operations comprising;

receiving a series of multiple strokes via the touch sensitive screen, each stroke defined by a contact, trace, and lift of a stylus or finger or other member;

presenting each stroke on the display until occurrence of a prescribed event after which the displayed stroke disappears;

accumulating strokes into a buffer and interpreting all accumulated strokes collectively against one of the following to identify multiple candidate strings that could potentially be represented by the accumulated strokes;

the character database alone, the character database and the linguistic database;

where the operation of interpreting all accumulated strokes comprises;

combining input strokes in different ways to produce alternative configurations;

for each alternative configuration, comparing stroke input with the character database to produce multiple character q candidates and a character distance for each candidate, wherein q is a positive integer greater than or equal to two; and

concatenating said character candidates in different combinations to produce complete candidate strings each comprising a potential representation of user-intended strokes from start to end; and

producing a ranked list of the complete candidate strings according to a score of each candidate string, where scores are produced by weighing the distances of the concatenated characters and string likeliness according to said linguistic database such that strings that are more likely to represent the accumulated strokes are ranked higher and strings that are less likely to represent the accumulated strokes or absent from the linguistic database are ranked lower;

and

operating the display to present candidate strings for user selection, the candidate strings being displayed upon one of the following basis;

(1) only after all strokes have disappeared, (2) only after receiving a predefined user submitted delimiter, (3) only after a predetermined delay has elapsed following user entry of a most recent stroke.

7. The invention of any of claims 1 or 2 or 3 or 4 or 5 or 6, where said ranking is dependent upon string length such that long strings that have a given linguistic likelihood are promoted more than short strings of the given linguistic likelihood.

8. The invention of any of claims 1 or 2 or 3 or 4 or 5 or 6, where the prescribed event comprises expiration of a prescribed time period.

9. The invention of claim 8, responsive to user entry of multiple strokes within a length of time less than or equal to the prescribed time period, concurrently presenting all of the strokes in various stages of fading.

10. The invention of claim 8, where the predetermined delay is different than the prescribed time period.

11. The invention of any of claims 1 or 2 or 3 or 4 or 5 or 6, where the operation of presenting each stroke on the display until occurrence of a prescribed event after which the displayed stroke disappears comprises:

making an area where the touch screen received the stroke visually distinct; and

after a prescribed time period from entry of the stroke, fading and eventually removing the visually distinct area.

12. The invention of any of claims 1 or 2 or 3 or 4 or 5 or 6, where the prescribed event comprises user entry of a prescribed number of subsequent strokes.

13. The invention of any of claims 1 or 2 or 3 or 4 or 5 or 6, where the operation of presenting each stroke on the display until occurrence of a prescribed event after which the displayed stroke disappears comprises:

responsive to user entry of a new stroke, incrementally fading each currently displayed stroke by a predetermined amount.

14. The invention of any of claims 1 or 2 or 3 or 4 or 5 or 6, where the received strokes form multiple characters and at least some of the characters substantially overlap each other.

2. A handwriting recognition apparatus, comprising:

first means for displaying graphics and text;

integrated with the first means, second means for receiving touch input;

third means for storing digital data including any of;

a character database representing predefined character shape data, the character database and a linguistic database representing predefined character strings; and

fourth means for processing digital data including performing operations comprising;

receiving a series of multiple strokes via the second means, each stroke defined by a contact, trace, and lift of a stylus or finger or other member;

presenting each stroke on the first means until occurrence of a prescribed event after which the displayed stroke disappears;

accumulating strokes into a buffer and interpreting all accumulated strokes collectively against one of the following to identify multiple candidate strings that could potentially be represented by the accumulated strokes;

the character database alone, the character database and the linguistic database;

where the operation of interpreting all accumulated strokes comprises;

combining input strokes in different ways to produce alternative configurations;

for each alternative configuration, comparing stroke input with the character database to produce multiple character q candidates and a character distance for each candidate, wherein q is a positive integer greater than or equal to two; and

concatenating said character candidates in different combinations to produce complete candidate strings each comprising a potential representation of user-intended strokes from start to end; and

producing a ranked list of the complete candidate strings according to a score of each candidate string, where scores are produced by weighing the distances of the concatenated characters and string likeliness according to said linguistic database such that strings that are more likely to represent the accumulated strokes are ranked higher and strings that are less likely to represent the accumulated strokes or absent from the linguistic database are ranked lower;

and

operating the first means to present candidate strings for user selection, the candidate strings being displayed upon one of the following basis;

(1) only after all strokes have disappeared, (2) only after receiving a predefined user submitted delimiter, (3) only after a predetermined delay has elapsed following user entry of a most recent stroke.

3. A method for operating a handwriting recognition apparatus that includes an electrically driven display integrated with a touch sensitive screen, a machine-readable storage containing a character database representing predefined character shape data or the character database and a linguistic database representing predefined character strings, and a processor coupled to the display and storage, the operations comprising:

receiving a series of multiple strokes via the touch sensitive screen, each stroke defined by a contact, trace, and lift of a stylus or finger or other member;

presenting each stroke on the display until occurrence of a prescribed event after which the displayed stroke disappears;

accumulating strokes into a buffer and interpreting all accumulated strokes collectively against one of the following to identify multiple candidate strings that could potentially be represented by the accumulated strokes;

the character database alone, the character database and the linguistic database;

where the operation of interpreting all accumulated strokes comprises;

combining input strokes in different ways to produce alternative configurations;

for each alternative configuration, comparing stroke input with the character database to produce multiple character q candidates and a character distance for each candidate, wherein q is a positive integer greater than or equal to two; and

concatenating said character candidates in different combinations to produce complete candidate strings each comprising a potential representation of user-intended strokes from start to end; and

producing a ranked list of the complete candidate strings according to a score of each candidate string, where scores are produced by weighing the distances of the concatenated characters and string likeliness according to said linguistic database such that strings that are more likely to represent the accumulated strokes are ranked higher and strings that are less likely to represent the accumulated strokes or absent from the linguistic database are ranked lower;

and

operating the display to present candidate strings for user selection, the candidate strings being displayed upon one of the following basis;

(1) only after all strokes have disappeared, (2) only after receiving a predefined user submitted delimiter, (3) only after a predetermined delay has elapsed following user entry of a most recent stroke.

4. A method for operating a handwriting recognition apparatus that includes an electrically driven display integrated with a touch sensitive screen, a machine-readable storage containing a character database representing predefined character shape data or the character database and a linguistic database representing predefined character strings, and a processor coupled to the display and storage, the operations comprising:

a step for receiving a series of multiple strokes via the touch sensitive screen, each stroke defined by a contact, trace, and lift of a stylus or finger or other member;

a step for presenting each stroke on the display until occurrence of a prescribed event after which the displayed stroke disappears;

a step for accumulating strokes into a buffer and interpreting all accumulated strokes collectively against one of the following to identify multiple candidate strings that could potentially be represented by the accumulated strokes;

the character database alone, the character database and the linguistic database;

where the operation of interpreting all accumulated strokes comprises;

combining input strokes in different ways to produce alternative configurations;

for each alternative configuration, comparing stroke input with the character database to produce multiple character q candidates and a character distance for each candidate, wherein q is a positive integer greater than or equal to two; and

concatenating said character candidates in different combinations to produce complete candidate strings each comprising a potential representation of user-intended strokes from start to end; and

producing a ranked list of the complete candidate strings according to a score of each candidate string, where scores are produced by weighing the distances of the concatenated characters and string likeliness according to said linguistic database such that strings that are more likely to represent the accumulated strokes are ranked higher and strings that are less likely to represent the accumulated strokes or absent from the linguistic database are ranked lower;

and

a step for operating the display to present candidate strings for user selection, the candidate strings being displayed upon one of the following basis;

(1) only after all strokes have disappeared, (2) only after receiving a predefined user submitted delimiter, (3) only after a predetermined delay has elapsed following user entry of a most recent stroke.

5. A computer readable storage medium having stored thereon a non-transitory program of machine-readable instructions executable by a digital processing apparatus to perform operations to manage a handwriting recognition apparatus that includes an electrically driven display integrated with a touch sensitive screen, a machine-readable storage containing a character database representing predefined character shape data or the character database and a linguistic database representing predefined character strings, and a processor coupled to the display and storage, the operations comprising:

receiving a series of multiple strokes via the touch sensitive screen, each stroke defined by a contact, trace, and lift of a stylus or finger or other member;

presenting each stroke on the display until occurrence of a prescribed event after which the displayed stroke disappears;

accumulating strokes into a buffer and interpreting all accumulated strokes collectively against one of the following to identify multiple candidate strings that could potentially be represented by the accumulated strokes;

the character database alone, the character database and the linguistic database;

where the operation of interpreting all accumulated strokes comprises;

combining input strokes in different ways to produce alternative configurations;

for each alternative configuration, comparing stroke input with the character database to produce multiple character q candidates and a character distance for each candidate, wherein q is a positive integer greater than or equal to two; and

concatenating said character candidates in different combinations to produce complete candidate strings each comprising a potential representation of user-intended strokes from start to end; and

producing a ranked list of the complete candidate strings according to a score of each candidate string, where scores are produced by weighing the distances of the concatenated characters and string likeliness according to said linguistic database such that strings that are more likely to represent the accumulated strokes are ranked higher and strings that are less likely to represent the accumulated strokes or absent from the linguistic database are ranked lower;

and

operating the display to present candidate strings for user selection, the candidate strings being displayed upon one of the following basis;

(1) only after all strokes have disappeared, (2) only after receiving a predefined user submitted delimiter, (3) only after a predetermined delay has elapsed following user entry of a most recent stroke.

6. Circuitry of multiple interconnected electrically conductive elements configured to perform operations to manage a handwriting recognition apparatus that includes an electrically driven display integrated with a touch sensitive screen, a machine-readable storage containing a character database representing predefined character shape data or the character database and a linguistic database representing predefined character strings, and a processor coupled to the display and storage, the operations comprising:

receiving a series of multiple strokes via the touch sensitive screen, each stroke defined by a contact, trace, and lift of a stylus or finger or other member;

presenting each stroke on the display until occurrence of a prescribed event after which the displayed stroke disappears;

accumulating strokes into a buffer and interpreting all accumulated strokes collectively against one of the following to identify multiple candidate strings that could potentially be represented by the accumulated strokes;

the character database alone, the character database and the linguistic database;

where the operation of interpreting all accumulated strokes comprises;

combining input strokes in different ways to produce alternative configurations;

for each alternative configuration, comparing stroke input with the character database to produce multiple character q candidates and a character distance for each candidate, wherein q is a positive integer greater than or equal to two; and

concatenating said character candidates in different combinations to produce complete candidate strings each comprising a potential representation of user-intended strokes from start to end; and

producing a ranked list of the complete candidate strings according to a score of each candidate string, where scores are produced by weighing the distances of the concatenated characters and string likeliness according to said linguistic database such that strings that are more likely to represent the accumulated strokes are ranked higher and strings that are less likely to represent the accumulated strokes or absent from the linguistic database are ranked lower;

and

operating the display to present candidate strings for user selection, the candidate strings being displayed upon one of the following basis;

(1) only after all strokes have disappeared, (2) only after receiving a predefined user submitted delimiter, (3) only after a predetermined delay has elapsed following user entry of a most recent stroke.