VOICE RECOGNITION OF COMMANDS EXTRACTED FROM USER INTERFACE SCREEN DEVICES

• US 20150212791A1
• Filed: 01/28/2014
• Published: 07/30/2015
• Est. Priority Date: 01/28/2014
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: KUMAR, Saurabh, KOWDEED, Srinivasa Rao, KUPPUSAMY, Kavin Kumar
• Granted Patent: US 9,858,039 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A method, system, and computer program product for human interface design. Embodiments proceed upon receiving a markup language description of user interface pages (e.g., HTML pages), then, without modifying the user interface page, parsing the markup language description to identify user interface objects configured to perform an operation responsive to a keyboard or mouse or pointing device. One or more mapping techniques serve to relate the parsed-out operation(s) to one or more voice commands. In some embodiments, the parser recognizes interface objects in forms such as a button, a textbox, a checkbox, or an option menu, and the voice commands correspond to an aspect that is displayed when rendering the interface object (e.g., a button label, a menu option, etc.). After receiving a user utterance, the utterance is converted into a text representation which in turn is mapped to voice commands that were parsed from the user interface page.

Claims

1. A method comprising:

using a computing system having at least one processor to perform a process, the process comprising;

identifying a markup language description of a user interface page;

identifying a voice command mapping for the user interface page, wherein the voice command mapping is generated by parsing the markup language description to identify at least one user interface object configured to perform at least one operation responsive to a keyboard or mouse or pointing device, wherein the parsing action does not create a modified version of the page;

receiving an utterance to be converted to a text representation of the utterance;

determining a plurality of matches between the text representation of the utterance and multiple matching voice commands; and

performing a confirmation of a single matching voice command from among the plurality of matches.

2. The method of claim 1, wherein the interface object is at least one of, a button a textbox, and a checkbox.

3. The method of claim 1, wherein the interface object comprises an option menu.

4. The method of claim 1, wherein the markup language description comprises HTML.

5. The method of claim 1, wherein the voice command mapping uses a hash map data structure to store a relationship between the at least one respective word or phrase to the at least one operation.

6. The method of claim 1, wherein the at least one operation comprises two or more operations.

7. The method of claim 6, wherein the confirmation of a single matching voice command from among the plurality of matches comprises disambiguation between the two or more operations.

8. The method of claim 1, wherein the voice command mapping uses a phonetic dictionary.

9. The method of claim 8, wherein the phonetic dictionary comprises words or phrases sorted in decreasing order of frequency of use.

10. A computer program product embodied in a non-transitory computer readable medium, the computer readable medium having stored thereon a sequence of instructions which, when executed by a processor causes the processor to execute a process, the process comprising:

identifying a markup language description of a user interface page;

identifying a voice command mapping for the user interface page, wherein the voice command mapping is generated by parsing the markup language description to identify at least one user interface object configured to perform at least one operation responsive to a keyboard or mouse or pointing device, wherein the parsing action does not create a modified version of the page;

receiving an utterance to be converted to a text representation of the utterance;

determining a plurality of matches between the text representation of the utterance and multiple matching voice commands; and

performing a confirmation of a single matching voice command from among the plurality of matches.

11. The computer program product of claim 10, wherein the interface object is at least one of, a button a textbox, and a checkbox.

12. The computer program product of claim 10, wherein the interface object comprises an option menu.

13. The computer program product of claim 10, wherein the markup language description comprises HTML.

14. The computer program product of claim 10, wherein the voice command mapping uses a hash map data structure to store a relationship between the at least one respective word or phrase to the at least one operation.

15. The computer program product of claim 10, wherein the at least one operation comprises two or more operations.

16. The computer program product of claim 15, wherein the confirmation of a single matching voice command from among the plurality of matches comprises disambiguation between the two or more operations.

17. The computer program product of claim 10, wherein the voice command mapping uses a phonetic dictionary.

18. A computer system comprising:

a parser module to identify a markup language description of a user interface page and to identify a voice command mapping for the user interface page, wherein the voice command mapping is generated by parsing the markup language description to identify at least one user interface object configured to perform at least one operation responsive to a keyboard or mouse or pointing device, wherein the parsing does not create a modified version of the page;

a receiving module to receive an utterance to be converted to a text representation of the utterance wherein the utterance is used to determine a plurality of matches between the text representation of the utterance and multiple matching voice commands; and

a confirmation module to perform a confirmation of a single matching voice command from among the plurality of matches.

19. The computer system of claim 18, wherein the interface object is at least one of, a button a textbox, and a checkbox.

20. The computer system of claim 18, wherein the markup language description comprises HTML.