Digital ink database searching using handwriting feature synthesis

• US 20050222848A1
• Filed: 10/15/2002
• Published: 10/06/2005
• Est. Priority Date: 10/15/2001
• Status: Active Grant

• Current Assignee: Silverbrook Research Pty Limited
• Original Assignee: Silverbrook Research Pty Limited
• Inventors: Napper, Jonathan Leigh, Lapstun, Paul
• Granted Patent: US 8,219,908 B2
• Time in Patent Office: Days
• US Class Current: 705/1.100
• CPC Class Codes: G06F 16/3332 Query translation
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06Q 10/063 Operations research, analysis or management
• CPC Class Codes: G06V 30/226 of cursive writing

Abstract

A system and method for ink database searching using handwriting feature synthesis is disclosed which allows a digital ink database to be searched using a text-based query. Using a writer-specific handwriting model derived from a handwriting recognition system or suitable training procedure, a text query is converted into feature vectors that are similar to the feature vectors that would have been extracted had the author of the digital ink database written the text query by hand. The feature vectors are then used to search the database. This allows the searching of a digital ink database when the only input mechanism available is text entry, and can allow a person other than the author of the digital ink database to search the digital ink database.

Claims

1. A method of searching a digital ink database using a text query, the method including:

performing a search of a lexicon consisting of a table of letter sequences and associated feature vectors, and determining a sequence of lexicon entries which contain letter sequences that can be combined to produce the text query;

obtaining a set of feature vectors corresponding to the sequence of lexicon entries; and

,

using the set of feature vectors to search the digital ink database.

2. The method as claimed in claim 1, wherein the lexicon is part of a handwriting model obtained from the author of the digital ink database using handwriting recognition results.

3. The method as claimed in either claim 1 or claim 2, wherein the handwriting model stores a mapping of letter sequences to feature vectors.

4. The method as claimed in any one of the claims 1 to 3, wherein the lexicon includes multiple feature vectors for each letter sequence.

5. The method as claimed in any one of the claims 1 to 4, wherein the lexicon includes an end of word character to indicate word endings.

6. The method as claimed in any one of the claims 1 to 5, wherein the handwriting recognition results used to generate the lexicon are derived from a training text.

7. The method as claimed in any one of the claims 1 to 6, wherein if more than one sequence of lexicon entries is determined, then the sequence of lexicon entries having the least number of entries is used.

8. The method as claimed in any one of the claims 1 to 7, wherein a search algorithm is used to search for a letter sequence s.

9. The method as claimed in any one of the claims 1 to 8, wherein the text query is obtained by converting handwritten input into text using a handwriting recognition system.

10. The method as claimed in any one of the claims 1 to 8, wherein the text query is obtained by converting voice input into text using a voice recognition system.

11. The method as claimed in any one of the claims 1 to 10, wherein the handwriting recognition results are obtained by:

sampling the handwriting;

smoothing the handwriting using a filter;

performing slant correction;

using a zone estimation algorithm to perform height normalisation;

using feature extraction to perform segmentation into sub-strokes and generation of feature vectors;

using feature reduction of the set of feature vectors;

performing vector quantization to cluster the feature vectors and generate code word vectors; and

,

searching a dictionary for the most likely word from the generated text letters.

12. Apparatus for searching a digital ink database using a query, the apparatus including:

an input device for a user to input the query;

a processor able to receive the query and communicate with the digital ink database;

means to convert the query to a text query, if the query is not already a text query;

means for performing a search of a lexicon and determining a sequence of lexicon entries which contain letter sequences that can be combined to produce the text query;

means for obtaining a set of feature vectors corresponding to the sequence of lexicon entries;

means for searching the digital ink database using the set of feature vectors; and

an output device to display the results of the search to the user.

13. The apparatus as claimed in claim 12, wherein the lexicon is part of a handwriting model obtained from the author of the digital ink database using handwriting recognition results.

14. The apparatus as claimed in either claim 12 or claim 13, wherein the text query is generated from handwriting using a different handwriting model than the handwriting model used to generate the feature vectors for searching.

15. The apparatus as claimed in any one of the claims 12 to 14, wherein the lexicon is produced using a training text.

16. The apparatus as claimed in any one of the claims 12 to 15, wherein the text query is generated from a voice query by a voice recognition system.

17. Apparatus for searching a digital ink database using a query, the apparatus including:

a store for storing the digital ink database;

a processor, the processor being adapted to;

perform a search of a lexicon and determine a sequence of lexicon entries which contain letter sequences that can be combined to produce the text query;

obtain a set of feature vectors corresponding to the sequence of lexicon entries;

use the set of feature vectors to search the digital ink database;

wherein, the lexicon is part of a handwriting model obtained from the author of the digital ink database using handwriting recognition results.