Digital ink-based search

• US 8,094,939 B2
• Filed: 06/26/2007
• Issued: 01/10/2012
• Est. Priority Date: 06/26/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Dongmei, Hou, Xiaohui, Qiu, Yingjun, Wang, Jian
• Primary Examiner(s): Patel, Jayesh A
• Application Number:
  US11/821,837
• Publication Number:
  US 20090003658A1
• Time in Patent Office: 1,659 Days
• Field of Search: 382/305, 382/292, 382/185, 382/187, 382/198, 382/202
• US Class Current: 382/185
• CPC Class Codes: G06F 16/5846 using extracted text
• CPC Class Codes: G06V 30/32 Digital ink

Abstract

Described is searching directly based on digital ink input to provide a result set of one or more items. Digital ink input (e.g., a handwritten character, sketched shape, gesture, drawing picture) is provided to a search engine and interpreted thereby, with a search result (or results) returned. Different kinds of digital ink can be used as search input without changing modes. The search engine includes a unified digital ink recognizer that recognizes digital ink as a character or another type of digital ink. When the recognition result is a character, the character may be used in a keyword search to find one or more corresponding non-character items, e.g., from a data store. When the recognition result is a non-character item, the non-character item is provided as the result, without keyword searching. The search result may appear as one or more item representations, such as in a user interface result panel.

Claims

1. A computer-readable storage device comprising computer-executable instructions, which when executed by a processing unit, cause the processing unit to perform:

receiving digital ink as input;

providing the digital ink to a unified digital ink recognizer that recognizes the digital ink;

receiving a recognition result from the unified digital ink recognizer;

in a first instance when the recognition result corresponds to a character identified by the unified digital ink recognizer;

using the character identified by the unified digital ink recognizer in a keyword search to locate one or more first corresponding non-character items each having a respective keyword that includes the character identified by the unified digital ink recognizer, and

outputting each of the one or more first corresponding non-character items together with the respective keyword that includes the character identified by the unified digital ink recognizer;

in a second instance when the recognition result corresponds to a second non-character item identified by the unified digital ink recognizer, outputting the second non-character item without keyword searching,

wherein the one or more first corresponding non-character items and the second non-character item include shapes, gestures, or drawn pictures.

2. The computer-readable storage device of claim 1, wherein using the character in the keyword search in the first instance comprises building the keyword from the character and at least one other character.

3. The computer-readable storage device of claim 1, wherein the recognition result corresponds to the character and the outputting of the first instance comprises providing a visible representation of the one or more first corresponding non-character items and the keyword to a user interface result panel.

4. The computer-readable storage device of claim 1, wherein the outputting of the second instance comprises providing a visible representation of the second non-character item to a user interface result panel.

5. The computer-readable storage device of claim 1, wherein the outputting of the first instance comprises providing the one or more first corresponding non-character items to a software program that is being searched with the digital ink.

6. The computer-readable storage device of claim 1, wherein the outputting of the second instance comprises providing the second non-character item to a software program that is being searched with the digital ink.

7. The computer-readable storage device of claim 6, wherein the second non-character item comprises a value representing a second non-character shape, and wherein providing the second non-character item to the software program comprises mapping the value to an identifier of the second non-character shape as used in the software program.

8. A method comprising:

receiving digital ink as input;

providing the digital ink to a unified digital ink recognizer that recognizes the digital ink;

receiving a recognition result from the unified digital ink recognizer;

in a first instance when the recognition result corresponds to a character identified by the unified digital ink recognizer;

using the character identified by the unified digital ink recognizer in a keyword search to locate one or more first corresponding non-character items each having a respective keyword that includes the character identified by the unified digital ink recognizer, and

outputting each of the one or more first corresponding non-character items together with the respective keyword that includes the character identified by the unified digital ink recognizer;

in a second instance when the recognition result corresponds to a second non-character item identified by the unified digital ink recognizer, outputting the second non-character item without keyword searching,

wherein the one or more first corresponding non-character items and the second non-character item include shapes, gestures, or drawn pictures.

9. The method of claim 8, wherein using the character in the keyword search in the first instance comprises building the keyword from the character and at least one other character.

10. The method of claim 8, wherein the recognition result corresponds to the character and the outputting of the first instance comprises providing a visible representation of the one or more first corresponding non-character items and the keyword to a user interface result panel.

11. The method of claim 8, wherein the outputting of the second instance comprises providing a visible representation of the second non-character item to a user interface result panel.

12. The method of claim 8, wherein the outputting of the first instance comprises providing the one or more first corresponding non-character items to a software program that is being searched with the digital ink.

13. The method of claim 8, wherein the outputting of the second instance comprises providing the second non-character item to a software program that is being searched with the digital ink.

14. A system comprising:

an input mechanism configured to provide digital ink received as input to a unified digital ink recognizer that recognizes the digital ink;

a search logic configured to;

receive a recognition result from the unified digital ink recognizer;

in a first instance when the recognition result corresponds to a character identified by the unified digital ink recognizer;

use the character identified by the unified digital ink recognizer in a keyword search to locate one or more first corresponding non-character items each having a respective keyword that includes the character identified by the unified digital ink recognizer, and

output each of the one or more first corresponding non-character items together with the respective keyword that includes the character identified by the unified digital ink recognizer;

in a second instance when the recognition result corresponds to a second non-character item identified by the unified digital ink recognizer, output the second non-character item without keyword searching; and

a processor configured to execute the search logic;

wherein the one or more first corresponding non-character items and the second non-character item include shapes, gestures, or drawn pictures.

15. The system of claim 14, wherein the search logic is further configured to, in the first instance, build the keyword from the character and at least one other character.

16. The system of claim 14, wherein the search logic is further configured to, in the first instance, cause a visible representation of the one or more first corresponding non-character items and the keyword to appear on a user interface result panel.

17. The system of claim 14, wherein the search logic is further configured to, in the second instance, cause a visible representation of the second non-character item to appear on a user interface result panel.

18. The system of claim 14, wherein the search logic is further configured to, in the first instance, output the one or more first corresponding non-character items to a software program that is being searched with the digital ink.

19. The system of claim 14, wherein the search logic is further configured to, in the second instance, output the second non-character item to a software program that is being searched with the digital ink.

20. The system of claim 19, wherein the second non-character item comprises a value representing a second non-character shape, and wherein the search logic is configured to output the second non-character item to the software program by mapping the value to an identifier of the second non-character shape as used in the software program.