Search-Based Dynamic Voice Activation

• US 20140358903A1
• Filed: 08/20/2014
• Published: 12/04/2014
• Est. Priority Date: 12/31/2007
• Status: Active Grant

• Current Assignee: Google Technology Holdings LLC (Alphabet Inc.)
• Original Assignee: Motorola Mobility LLC (Lenovo Group Ltd.)
• Inventors: Cheng, Yan Ming, Ma, Changxue, Mazurkiewicz, Theodore
• Granted Patent: US 10,664,229 B2
• Time in Patent Office: Days
• US Class Current: 707/722
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/16 Sound input; Sound output speech processing G10L
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: G09B 5/04 with audible presentation of the material to be studied sound recording or reproducing G11B
• CPC Class Codes: G10L 19/0018 Speech coding using phonetic or linguistical decoding of the source; Reconstruction using text-to-speech synthesis
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A method, apparatus, and electronic device for voice navigation are disclosed. A voice input mechanism 310 may receive a verbal input from a user to a voice user interface program invisible to the user. A processor 104 may identify in a graphical user interface (GUI) a set of GUI items. The processor 104 may convert the set of GUI items to a set of voice searchable indices 400. The processor 104 may correlate a matching GUI item of the set of GUI items to a phonemic representation of the verbal input.

Claims

1. (canceled)

3. The method of claim 1, wherein identifying a particular term that is associated with a particular control included on the graphical user interface comprises:

identifying a linguistic surface form, surrounding texts, hyperlinked webpage titles, or metadata that are associated with the particular control; and

identifying the particular term from the linguistic surface form, the surrounding texts, the hyperlinked webpage titles, or the metadata.

4. The method of claim 1, wherein the particular control is a hyperlink, an image, or an application icon.

5. The method of claim 1, wherein the particular control links to a file, webpage, or an application.

6. The method of claim 1, wherein using the voice searchable index in determining whether a voice input that is received after the graphical user interface is displayed represents a selection of one or more of the controls included on the graphical user interface comprises:

receiving a phoneme lattice for the voice input;

identifying linguistic search terms from the phoneme lattice; and

identifying a term that matches the linguistic search terms within the voice searchable index.

7. The method of claim 1, wherein using the voice searchable index in determining whether a voice input that is received after the graphical user interface is displayed represents a selection of one of the one or more controls included on the graphical user interface comprises:

determining that the voice input that is received after the graphical user interface is displayed represents a selection of an alternative control of the one or more controls included on the graphical user interface, wherein the alternative control is a control previously selected by a user associated with the voice input;

providing, for display and selection for a temporary period of time, the alternative control.

8. The method of claim 1, wherein using the voice searchable index in determining whether a voice input that is received after the graphical user interface is displayed represents a selection of one of the one or more controls included on the graphical user interface comprises:

determining whether a voice input that is received after the graphical user interface is displayed represents a selection of one of the one or more controls included on the graphical user interface further using previously selected controls of the one or more controls that were selected by a user associated with the voice input.

9. The method of claim 1, wherein the data referencing the particular control comprises:

a term associated with the particular control, one or more alternate terms associated with the particular control, a type of the particular control, or a reference to an object associated with the particular control.

2. A computer-implemented method comprising:

providing, for display, a graphical user interface that includes one or more controls;

after the graphical user interface is displayed, identifying a particular term that is associated with a particular control included on the graphical user interface;

obtaining a phonetic representation of the particular term;

storing, in a voice searchable index that is populated after the graphical user interface is displayed, (i) the phonetic representation of the particular term, and (ii) data referencing the particular control; and

using the voice searchable index in determining whether a voice input that is received after the graphical user interface is displayed represents a selection of one of the one or more controls included on the graphical user interface.

10. A system comprising:

one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising;

providing, for display, a graphical user interface that includes one or more controls;

after the graphical user interface is displayed, identifying a particular term that is associated with a particular control included on the graphical user interface;

obtaining a phonetic representation of the particular term;

storing, in a voice searchable index that is populated after the graphical user interface is displayed, (i) the phonetic representation of the particular term, and (ii) data referencing the particular control; and

using the voice searchable index in determining whether a voice input that is received after the graphical user interface is displayed represents a selection of one of the one or more controls included on the graphical user interface.

11. The system of claim 10, wherein identifying a particular term that is associated with a particular control included on the graphical user interface comprises:

identifying a linguistic surface form, surrounding texts, hyperlinked webpage titles, or metadata that are associated with the particular control; and

identifying the particular term from the linguistic surface form, the surrounding texts, the hyperlinked webpage titles, or the metadata.

12. The system of claim 10, wherein the particular control is a hyperlink, an image, or an application icon.

13. The system of claim 10, wherein the particular control links to a file, webpage, or an application.

14. The system of claim 10, wherein using the voice searchable index in determining whether a voice input that is received after the graphical user interface is displayed represents a selection of one or more of the controls included on the graphical user interface comprises:

receiving a phoneme lattice for the voice input;

identifying linguistic search terms from the phoneme lattice; and

identifying a term that matches the linguistic search terms within the voice searchable index.

15. The system of claim 10, wherein using the voice searchable index in determining whether a voice input that is received after the graphical user interface is displayed represents a selection of one of the one or more controls included on the graphical user interface comprises:

determining that the voice input that is received after the graphical user interface is displayed represents a selection of an alternative control of the one or more controls included on the graphical user interface, wherein the alternative control is a control previously selected by a user associated with the voice input;

providing, for display and selection for a temporary period of time, the alternative control.

16. The system of claim 10, wherein using the voice searchable index in determining whether a voice input that is received after the graphical user interface is displayed represents a selection of one of the one or more controls included on the graphical user interface comprises:

determining whether a voice input that is received after the graphical user interface is displayed represents a selection of one of the one or more controls included on the graphical user interface further using previously selected controls of the one or more controls that were selected by a user associated with the voice input.

17. The system of claim 10, wherein the data referencing the particular control comprises:

a term associated with the particular control, one or more alternate terms associated with the particular control, a type of the particular control, or a reference to an object associated with the particular control.

18. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:

providing, for display, a graphical user interface that includes one or more controls;

after the graphical user interface is displayed, identifying a particular term that is associated with a particular control included on the graphical user interface;

obtaining a phonetic representation of the particular term;

storing, in a voice searchable index that is populated after the graphical user interface is displayed, (i) the phonetic representation of the particular term, and (ii) data referencing the particular control; and

using the voice searchable index in determining whether a voice input that is received after the graphical user interface is displayed represents a selection of one of the one or more controls included on the graphical user interface.

19. The medium of claim 18, wherein identifying a particular term that is associated with a particular control included on the graphical user interface comprises:

identifying a linguistic surface form, surrounding texts, hyperlinked webpage titles, or metadata that are associated with the particular control; and

identifying the particular term from the linguistic surface form, the surrounding texts, the hyperlinked webpage titles, or the metadata.

20. The medium of claim 18, wherein using the voice searchable index in determining whether a voice input that is received after the graphical user interface is displayed represents a selection of one or more of the controls included on the graphical user interface comprises:

receiving a phoneme lattice for the voice input;

identifying linguistic search terms from the phoneme lattice; and

identifying a term that matches the linguistic search terms within the voice searchable index.

21. The medium of claim 18, wherein using the voice searchable index in determining whether a voice input that is received after the graphical user interface is displayed represents a selection of one of the one or more controls included on the graphical user interface comprises:

determining whether a voice input that is received after the graphical user interface is displayed represents a selection of one of the one or more controls included on the graphical user interface further using previously selected controls of the one or more controls that were selected by a user associated with the voice input.