Digital ink-based search

• US 20090003658A1
• Filed: 06/26/2007
• Published: 01/01/2009
• Est. Priority Date: 06/26/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Dongmei, Hou, Xiaohui, Wang, Jian, Qiu, Yingjun
• Granted Patent: US 8,094,939 B2
• Time in Patent Office: Days
• US Class Current: 382/113
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06V 30/32 Digital ink

Abstract

Described is searching directly based on digital ink input to provide a result set of one or more items. Digital ink input (e.g., a handwritten character, sketched shape, gesture, drawing picture) is provided to a search engine and interpreted thereby, with a search result (or results) returned. Different kinds of digital ink can be used as search input without changing modes. The search engine includes a unified digital ink recognizer that recognizes digital ink as a character or another type of digital ink. When the recognition result is a character, the character may be used in a keyword search to find one or more corresponding non-character items, e.g., from a data store. When the recognition result is a non-character item, the non-character item is provided as the result, without keyword searching. The search result may appear as one or more item representations, such as in a user interface result panel.

Claims

1. A computer-readable medium having computer-executable instructions, which when executed perform steps, comprising:

receiving digital ink as input;

providing the digital ink to a unified digital ink recognizer that is capable of recognizing the digital ink as a character or as another type of digital ink without mode selection or recognition parameter input;

receiving a recognition result from the unified digital ink recognizer; and

if the recognition result corresponds to a character, outputting the character or using the recognition result in a keyword search to locate at least one item for output, and if the recognition result corresponds to a non-character item, using the item for output without keyword searching.

2. The computer-readable medium of claim 1 wherein the recognition result corresponds to a character, and wherein using the recognition result in a keyword search comprises returning an item in which the character appears.

3. The computer-readable medium of claim 1 wherein the recognition result corresponds to a character, and wherein using the recognition result in a keyword search comprises building the keyword from the character and at least one other character.

4. The computer-readable medium of claim 1 wherein the recognition result corresponds to a character, and having further computer-executable instructions, comprising, outputting the search results as a visible representation of the at least one item to a user interface result panel.

5. The computer-readable medium of claim 1 wherein the recognition result corresponds to a non-character item, and having further computer-executable instructions, comprising, outputting a visible representation of the item to a user interface result panel.

6. The computer-readable medium of claim 1 wherein the recognition result corresponds to a character, wherein the search results return a single item, and having further computer-executable instructions, comprising, outputting information corresponding to the item to a software program that accepts the information for that type of item.

7. The computer-readable medium of claim 1 wherein the recognition result corresponds to a non-character item, and having further computer-executable instructions, comprising, outputting information corresponding to the item to a software program that accepts the information for that type of item.

8. The computer-readable medium of claim 7 wherein the non-character item comprises a value representing a shape, and wherein outputting the information corresponding to the item comprises mapping the value to an identifier of that shape as used in the software program.

9. In a computing environment, a system comprising:

an input mechanism that provides digital ink to a unified digital ink recognizer that recognizes the digital ink as a character or as another type of digital ink without mode selection or recognition parameter input; and

search logic that processes a recognition result from the unified digital ink recognizer to find a result set corresponding to the recognition result.

10. The system of claim 9 further comprising a non-character item data store, wherein the recognition result corresponds to a character, and wherein the search logic finds a result set by using the recognition result in a keyword search of the item data store to return each item in which the character appears.

11. The system of claim 9 wherein the recognized character is a character of the Chinese character set as defined in Unicode.

12. The system of claim 9 further comprising a non-character item data store, wherein the recognition result corresponds to a character, and wherein the search logic finds a result set by using the recognition result in a keyword search, including by building a keyword from the character and at least one other character, and searching the item data store based upon the keyword.

13. The system of claim 9 further comprising a user interface result panel, wherein the recognition result corresponds to a character, and wherein the search logic causes search results corresponding to the result set to appear as a visible representation of the at least one item to the result panel.

14. The system of claim 9 further comprising a user interface result panel, wherein the recognition result corresponds to a non-character item, and wherein the search logic causes a visible representation of an item corresponding to the recognition result to appear on the result panel.

15. The system of claim 9 wherein the recognition result corresponds to a non-character item representing a shape, a handwritten gesture, or a drawn picture.

16. The system of claim 9 wherein the recognition result corresponds to a shape used in a diagramming application program.

17. In a computing environment, a method comprising, receiving a recognition result comprising a character or another type of digital ink in response to digital ink sent to a unified digital ink recognizer that recognizes the digital ink as a character or as another type of digital ink without mode selection or recognition parameter input, and determining whether the recognition result corresponds to a character, and if so, using the character in a keyword search to locate a non-character item for output, and if not, using the recognition result for output without keyword searching.

18. The method of claim 17 wherein the recognition result corresponds to a shape used in a diagramming program, and further comprising, providing information corresponding to the shape to the diagramming program.

19. The method of claim 18 wherein providing the information comprises displaying a representation of the shape that can be dragged and dropped to an input area of the diagramming program.

20. The method of claim 17 wherein the recognition result corresponds to a character of the Chinese character set as defined in Unicode, and wherein using the character in a keyword search comprises locating each item in which the character appears.