Multi-mode text input

• US 9,865,262 B2
• Filed: 01/28/2016
• Issued: 01/09/2018
• Est. Priority Date: 05/17/2011
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Varthakavi, Mohan, Nanduri, Jayaram, Kothari, Nikhil
• Primary Examiner(s): Sirjani, Fariba
• Application Number:
  US15/009,375
• Publication Number:
  US 20160148617A1
• Time in Patent Office: 712 Days
• Field of Search: None
• CPC Class Codes: G06F 16/335 Filtering based on additional data, e.g. user or group profiles filtering in web context G06F16/9535, G06F16/9536
• CPC Class Codes: G06F 16/38 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

Concepts and technologies are described herein for multi-mode text input. In accordance with the concepts and technologies disclosed herein, content is received. The content can include one or more input indicators. The input indicators can indicate that user input can be used in conjunction with consumption or use of the content. The application is configured to analyze the content to determine context associated with the content and/or the client device executing the application. The application also is configured to determine, based upon the content and/or the contextual information, which input device to use to obtain input associated with use or consumption of the content. Input captured with the input device can be converted to text and used during use or consumption of the content.

Claims

1. A computer-implemented method for interacting with content, the computer-implemented method comprising performing computer-implemented operations at a computer, the computer implemented operations comprising:

receiving, at a computer, the content from a source;

identifying at least one implicit input indicator in the content, the at least one implicit input indicator including a form element of the content through which textual input is provided to the source in conjunction with consumption of the content;

identifying a value associated with the form element in the content;

in response to identifying the at least one implicit indicator, presenting, by issuing one or more commands to the computer, one or more non-textual input devices associated with the computer based on the identified value;

receiving, by a user interface of the computer, selection of a user interface control indicating a non-textual input device of the one or more non-textual input devices to be used in capturing multi-mode input;

capturing multi-mode input, via the non-textual input device, the multi-mode input including at least one of one or more images from a camera, sound captured by a microphone, and touch input from a touch screen;

converting the multi-mode input to text; and

sending the text to the source as input to the form element.

2. The computer-implemented method of claim 1, wherein the at least one implicit input indicator comprises a keyword.

3. The computer-implemented method of claim 1, wherein identifying the at least one implicit input indicator comprises identifying a non-textual input device capable of receiving the multi-mode input, the multi-mode input comprising one or more of camera, speech, and touch input.

4. The computer-implemented method of claim 1, wherein identifying the at least one implicit input indicator comprises:

identifying at least one input field in the content;

analyzing the at least one input field to identify a type of input; and

determining that a non-textual input device is available for entering the input.

5. The computer-implemented method of claim 1, further comprising:

determining a context associated with the computer; and

determining, by the computer, a type of information to be captured from a plurality of types of information to be captured based on the context, and

wherein activating the one or more non-textual input devices comprises activating the one or more non-textual input devices according to the type of information to be captured based on the context.

6. The computer-implemented method of claim 1, further comprising identifying at least one explicit input indicator, the explicit input indicator comprising a meta tag or flag.

7. An apparatus comprising:

a processor;

a computer-readable storage medium having computer-executable instructions stored thereon which, when executed by a computer, cause the computer to;

receive content from a source,

identify an input field in the content,

identify at least one implicit input indicator with respect to the input field, the at least one implicit input indicator indicating that textual input is provided to the source in conjunction with the input field,

identify a value associated with the form element in the content,

in response to the identifying of the at least one implicit input indicator, analyze the input field to identify a type of non-textual input used in conjunction with the input field and present, by issuing one or more commands to a display, one or more non-textual input devices based on the identified value,

determine that the non-textual input device is available for entering the input,

present, in a user interface, a user interface control for accessing the non-textual input device for capturing multi-mode input, the multi-mode input including at least one of one or more images from a camera, sound captured by a microphone, and touch input from a touch screen,

detect a selection of the non-textual input device via a user interface control,

activate the non-textual input device in response to detecting a selection of the non-textual input device via the user interface control,

capture the multi-mode input, via the activated non-textual input device,

convert the multi-mode input to text, and

send the text to the source as input to the form element.

8. The apparatus of claim 7, wherein the at least one implicit input indicator comprises a keyword.

9. The apparatus of claim 7, wherein identifying the at least one implicit input indicator comprises identifying one or more non-textual input devices capable of receiving the multi-mode input, the multi-mode input comprising one or more of camera, speech, and touch input.

10. The apparatus of claim 7, wherein the computer-readable storage medium has further computer-executable instructions to identify at least one explicit input indicator, the explicit input indicator comprising a meta tag or flag.

11. A computer-readable storage medium comprising one or more of an optical disk, a solid state memory device, or a magnetic storage device, the computer-readable storage medium having computer-executable instructions stored thereon which, when executed by a computer, cause the computer to:

receive content from a source;

identify at least one implicit input indicator in the content, the at least one implicit input indicator including a form element of the content through which textual input is provided to the source in conjunction with consumption of the content; and

identify a value associated with the form element in the content;

in response to the identification of the at least one implicit indicator;

present, by issuing one or more commands to a display, one or more non-textual input devices associated with the computer based on the identified value,

detect a selection of one of the one or more non-textual input devices via a user interface control,

capture multi-mode input, via the selected non-textual input device, the multi-mode input including at least one of one or more images from a camera, sound captured by a microphone, and touch input from a touch screen,

convert the multi-mode input to text, and

submit the text to the source as input to the form element.

12. The computer-readable storage medium of claim 11, wherein the at least one implicit input indicator comprises a keyword.

13. The computer-readable storage medium of claim 11, wherein identifying the at least one implicit input indicator comprises identifying a non-textual input device capable of receiving the multi-mode input, the multi-mode input comprising one or more of camera, speech, and touch input.

14. The computer-readable storage medium of claim 11, wherein identifying the at least one implicit input indicator comprises:

identifying at least one input field in the content;

analyzing the at least one input field to identify a type of input; and

determining that a non-textual input device is available for entering the input.

15. The computer-readable storage medium of claim 11, comprising further computer-executable instructions which, when executed by the computer, cause the computer to:

determine a context associated with the computer;

determine a type of information to be captured from a plurality of types of information to be captured based on the context; and

wherein activating the one or more non-textual input devices comprises activating the one or more non-textual input devices according to the type of information to be captured based on the context.

16. The computer-readable storage medium of claim 11, comprising further computer-executable instructions which, when executed by the computer, cause the computer to:

identify at least one explicit input indicator, the explicit indicator comprising a meta tag or flag.