Method For Finding Elements In A Webpage Suitable For Use In A Voice User Interface (Disambiguation)

• US 20140350941A1
• Filed: 05/21/2013
• Published: 11/27/2014
• Est. Priority Date: 05/21/2013
• Status: Abandoned Application

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kim, Michael H., Benson, Rodger, Sarin, Raman, Ju, Yun-Cheng, Zeigler, Andrew Stephen
• Application Number:
  US13/899,074
• Publication Number:
  US 20140350941A1
• Time in Patent Office: Days
• US Class Current: 704/275
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/08 Speech classification or search
• CPC Class Codes: G10L 15/19 Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 21/10 Transforming into visible information

Abstract

A disambiguation process for a voice interface for web pages or other documents. The process identifies interactive elements such as links, obtains one or more phrases of each interactive element, such as link text, title text and alternative text for images, and adds the phrases to a grammar which is used for speech recognition. A group of interactive elements are identified as potential best matches to a voice command when there is no single, clear best match. The disambiguation process modifies a display of the document to provide unique labels for each interactive element in the group, and the user is prompted to provide a subsequent spoke command to identify one of the unique labels. The selected unique label is identified and a click event is generated for the corresponding interactive element.

Claims

1. A method for providing a voice user interface, comprising:

analyzing a document to identify a plurality of interactive elements in the document, each interactive element of the plurality of interactive elements comprises an associated phrase;

rendering the document to provide a display on a display device, the associated phrases are provided in the display;

comparing a voice command of a user to a plurality of phrases, the plurality of phrases comprise the associated phrases of the plurality of interactive elements;

based on the comparing, determining a matching score for each interactive element indicating a degree of matching of its associated phrase to the voice command;

identifying one of the interactive elements as a closest match to the voice command based on its matching score; and

based on the matching scores, deciding whether to generate a click event for the one of the interactive elements which is the closest match or to initiate a disambiguation process which allows the user to select from among a group of the interactive elements which comprise matching scores which are highest among the plurality of interactive elements.

2. The method of claim 1, wherein:

the click event is generated for the one of the interactive elements which is the closest match if its matching score is sufficiently high in absolute terms and is sufficiently higher than a next lower matching score.

3. The method of claim 1, wherein:

the disambiguation process is initiated if the matching score of the one of the interactive elements which is the closest match is at least one of;

not sufficiently high in absolute terms, or not sufficiently higher than a next lower matching score.

4. The method of claim 1, wherein:

the disambiguation process comprises modifying the display to identify each of the interactive elements in the group.

5. The method of claim 4, wherein:

the modifying the display comprises providing a unique label on the display proximate to each of the interactive elements in the group.

6. The method of claim 5, further comprising:

comparing a subsequent voice command of the user to each unique label;

based on the comparing of the subsequent voice command, identifying one of the unique labels which is a best match to the subsequent voice command; and

generating a click event for one of the interactive elements which is identified by the one of the unique labels.

7. The method of claim 5, further comprising:

displaying a rank on each of the unique labels according to the matching scores of the interactive elements in the group.

8. The method of claim 4, wherein:

the modifying the display comprises changing an appearance on the display of the associated phrases of each of the interactive elements of the group.

9. The method of claim 4, wherein:

the modifying the display comprises removing from the display or visually de-emphasizing on the display an associated phrase of an interactive element of the plurality of interactive elements which is not in the group.

10. The method of claim 1, further comprising:

the voice command comprises a sequence of words; and

the matching scores are based on a number of words in the associated phrases which match the sequence of words.

11. The method of claim 10, wherein:

the matching scores are based on different levels of importance of words in the sequence of words.

12. The method of claim 10, wherein:

the matching scores are based on different levels of importance of different phrases of the plurality of phrases.

13. A computing device, comprising:

a display device;

a storage device which stores code and a document; and

a processor associated with the display device and the storage device, the processor executes the code to;

analyze a document to identify a plurality of interactive elements in the document, each interactive element of the plurality of interactive elements comprises an associated phrase;

render the document to provide a display on a display device, the associated phrases are provided in the display;

compare a voice command of a user to a plurality of phrases, the plurality of phrases comprise the associated phrases of the plurality of interactive elements;

based on the comparing, determine a matching score for each interactive element indicating a degree of matching of its associated phrase to the voice command, the matching scores are based on a number of words in the associated phrases which match the sequence of words;

identify one of the interactive elements as a closest match to the voice command based on its matching score; and

based on the identifying, generating a click event for the one of the interactive elements which is the closest match.

14. The computing device of claim 13, wherein:

the click event is generated without a further command from the user.

15. The computing device of claim 13, further comprising:

the matching scores are based on different levels of importance of words in the sequence of words.

16. The computing device of claim 13, further comprising:

the matching scores are based on an order of words in the sequence of words.

17. The computing device of claim 13, wherein:

the plurality of interactive elements comprises links; and

the associated phrases comprise link text of the links, the link text is provided in the display.

18. A computer-readable storage device having computer-readable software embodied thereon for programming a processor to perform a method for providing a voice user interface, the method comprising:

identify a plurality of links in a document, each link comprises link text, the link text for at least one of the links comprises a sequence of words;

displaying the document including the link text on a display device;

comparing a voice command of a user to a plurality of phrases, the plurality of phrases comprise the link text of the plurality of links, the comparing comprises comparing the sequence of words to the voice command and determining a longest subset of the sequence of words which matches the voice command;

based on the comparing, determining a matching score for each link indicating a degree of matching of its associated link text to the voice command, wherein the matching score for the at least one of the links is based on a number of words in the longest subset of the sequence of words which matches the voice command; and

identifying one of the links as a closest match to the voice command based on its matching score.

19. The computer-readable storage device of claim 18, wherein:

the matching score for the at least one of the links is based on different levels of importance of words in the sequence of words.

20. The computer-readable storage device of claim 18, wherein the method performed comprises:

based on the matching scores, deciding whether to generate a click event for the one of the links which is the closest match or to initiate a disambiguation process which allows the user to select from among a group of the links which have matching scores which are highest among the plurality of links.