SEARCHING AN ELECTRONIC FILING SYSTEM USING A HANDWRITTEN SEARCH QUERY AND A TEXT SEARCH QUERY

• US 20100063980A1
• Filed: 11/17/2009
• Published: 03/11/2010
• Est. Priority Date: 01/31/2002
• Status: Active Grant

• Current Assignee: Silverbrook Research Pty Limited
• Original Assignee: Silverbrook Research Pty Limited
• Inventors: Silverbrook, Kia, Napper, Jonathon Leigh, Lapstun, Paul
• Granted Patent: US 7,925,663 B2
• Time in Patent Office: Days
• US Class Current: 707/769
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/53 Querying
• CPC Class Codes: G06F 16/93 Document management systems
• CPC Class Codes: G06V 30/10 Character recognition
• CPC Class Codes: G06V 30/15 Cutting or merging image elements, e.g. region growing, watershed or clustering-based techniques
• CPC Class Codes: G06V 30/268 Lexical context
• CPC Class Codes: Y10S 707/99933 Query processing, i.e. searching

Abstract

A method of providing an electronic filing system which is searchable using a handwritten search query, the method including the steps of: obtaining the handwritten search query using an input device; performing a search of at least one database based on a comparison between the handwritten search query and handwritten annotations made on interactive pages stored in the at least one database; and, providing the results of the search to a user and facilitating access to at least one interactive page identified in the results of the search. An apparatus is also disclosed. Preferably, the interactive page is provided to the user in the form of printed paper, and the handwritten annotations are user handwriting, symbols, drawings, indicia or the like.

Claims

1. A method of searching an electronic filing system, the method including the steps of:

obtaining a handwritten search query in the form of digital ink;

obtaining a text search query including a text search term;

performing a search of at least one database for instances of the text search term;

performing a search of the at least one database for handwritten annotations that are less than a given physical distance from at least one of the instances of the text search term;

comparing the handwritten search query and the handwritten annotations in order to determine matches between the handwritten search query and the handwritten annotations; and

providing results of the comparing step as output.

2. The method as claimed in claim 1 comprising the further step of facilitating access to at least one page upon which the handwritten annotations appear.

3. The method as claimed in claim 1 comprising the further step of displaying at least one page identified in the results on a display device.

4. The method as claimed in claim 1, wherein the digital ink represents one or more of handwriting, symbols, drawings, and indicia.

5. The method as claimed in claim 1, wherein the handwritten search query is obtained from an imaging pen.

6. The method as claimed in claim 1, wherein the handwritten search query is one of a hand-drawn loop, underline, squiggle, cross, circumflex, and asterisk, associated with the text search term.

7. The method as claimed in claim 1, wherein the comparing step comprises the sub-steps of:

segmenting the handwritten annotations in the at least one database into segmented strokes and mapping the segmented strokes into a first sequence of feature representations;

segmenting the handwritten search query into segmented strokes and mapping the segmented strokes into a second sequence of feature representations;

performing an edit distance calculation between the first sequence of feature representations and the second sequence of feature representations; and

,

determining matches between the handwritten annotations in the at least one database and the handwritten search query by locating the smallest values from the edit distance calculation.

8. The method as claimed in claim 7, wherein the edit distance calculation uses sub-stroke insertion, sub-stroke deletion and/or sub-stroke substitution.

9. The method as claimed in claim 7, wherein segmentation at the vertical-axis (Y) extrema coordinates of strokes is utilized.

10. The method as claimed in claim 7, wherein the mapping of the segmented strokes into the feature representations uses a zone-normalized coordinate method.

11. The method as claimed in claim 7, wherein a weighting factor is applied to the vertical-axis (Y) coordinates of the feature representations.

12. The method as claimed in claim 7, wherein the segmented strokes undergo vector quantization and the edit distance calculation is replaced by a look-up table of codeword distances.

13. The method as claimed in claim 7, wherein the method uses full stroke matching of the search query.

14. The method as claimed in claim 1, wherein the at least one database is a digital image database containing one or more of pictures, drawings, images, and figures.