METHOD OF ENABLING VOICE INPUT FOR A VISUALLY BASED INTERFACE

• US 20130246920A1
• Filed: 01/29/2013
• Published: 09/19/2013
• Est. Priority Date: 03/19/2012
• Status: Abandoned Application

• Current Assignee: Blackberry Limited
• Original Assignee: Research In Motion Limited (Blackberry Limited)
• Inventors: FIELDS, Gregory Jason, NIRANJAN, Karthigesu
• Application Number:
  US13/753,263
• Publication Number:
  US 20130246920A1
• Time in Patent Office: Days
• US Class Current: 715/728
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 9/44 Arrangements for executing specific programs
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command

Abstract

A method of enabling voice input for a graphical user interface (GUI) based application on an electronic device. The method includes: obtaining required properties of one or more user interface objects of the GUI-based application, wherein the one or more user interface objects include one or more input objects; receiving a voice input; extracting from the voice input one or more elements; associating the one or more elements with the one or more input objects; identifying, based on said associating, an input object having a required property which is not satisfied; and outputting, based on the required property, audio output for a prompt for a further voice input.

Claims

1. A method of enabling voice input for a graphical user interface (GUI)-based application on an electronic device, the method comprising:

obtaining required properties of one or more user interface objects of the GUI-based application, wherein the one or more user interface objects include one or more input objects;

receiving a voice input;

extracting from the voice input one or more elements;

associating the one or more elements with the one or more input objects;

identifying, based on said associating, an input object having a required property which is not satisfied; and

outputting, based on the required property, audio output for a prompt for a further voice input.

2. The method as claimed in claim 1, further comprising:

receiving the further voice input; and

associating the further voice input with the identified input object.

3. The method as claimed in claim 1, wherein said associating comprises:

generating, based on said one or more elements, an event receivable by the GUI-based application.

4. The method as claimed in claim 1, wherein the one or more input objects are displayed on a display screen of the device, and said associating comprises updating the one or more input objects based on the one or more elements.

5. The method as claimed in claim 4, wherein the one or more input objects include an input field configured for receiving a text-based input, and wherein said updating comprises populating the input field with text based on the one or more elements.

6. The method as claimed in claim 4, wherein the one or more input objects include an input object configured for receiving a pointer input, and wherein said updating comprises one of:

moving a slider on a slider scale;

changing the state of a checkbox;

changing the state of a radio button; and

selecting an item from a drop-down list.

7. The method as claimed in claim 1, wherein the required properties include at least one of:

role information;

state information;

context information; and

contextual relationship information regarding the one or more user interface objects.

8. The method as claimed in claim 1, further comprising:

registering with an application programming interface (API).

9. The method as claimed in claim 8, wherein said obtaining required properties comprises requesting from the API information regarding the GUI-based application.

10. The method as claimed in claim 8, wherein said obtaining required properties comprises receiving the required properties via the API.

11. The method as claimed in claim 8, wherein said obtaining required properties comprises receiving one or more events from the GUI-based application via the API.

12. The method as claimed in claim 8, wherein said associating comprises providing, based on said one or more elements, information to the API.

13. An electronic device comprising:

a processor;

a memory coupled to the processor;

an audio input interface; and

an audio output interface;

wherein the memory includes instructions, which when executed by the processor, cause the processor to;

obtain required properties of one or more user interface objects of a graphical user interface (GUI)-based application, wherein the one or more user interface objects include one or more input objects;

receive a voice input through the audio input interface,

extract from the voice input one or more elements,

associate the one or more elements with the one or more input objects,

identify, based on said associating, an input object having a required property which is not satisfied, and

output, through the audio output interface, an audio output for a prompt for a further voice input, based on the required property.

14. The electronic device as claimed in claim 13, wherein the instructions, when executed by the processor, further cause the processor to:

receive, through the audio input interface, the further voice input; and

associate the further voice input with the identified input object.

15. The electronic device as claimed in claim 13, wherein the instructions when executed by the processor, further cause the processor to:

generate, based on said one or more elements, an event receivable by the GUI-based application.

16. The electronic device as claimed in claim 13, further comprising:

a display screen for displaying the one or more input objects,

wherein said associating comprises updating, on the display screen, the one or more input objects based on the one or more elements.

17. The electronic device as claimed in claim 16, wherein the one or more input objects include an input field configured for receiving a text-based input, and wherein said updating comprises populating the input field with text based on the one or more elements.

18. The electronic device as claimed in claim 16, wherein the one or more input objects include an input object configured for receiving a pointer input, and wherein said updating comprises one of:

moving a slider on a slider scale;

changing the state of a checkbox;

changing the state of a radio button; and

selecting an item from a drop-down list.

19. The electronic device as claimed in claim 13, wherein the required properties include at least one of:

role information;

state information;

context information; and

contextual relationship information regarding the one or more user interface objects.

20. The electronic device as claimed in claim 13, wherein the instructions when executed by the processor, further cause the processor to:

register with an application programming interface (API).

21. The electronic device as claimed in claim 20, wherein said obtaining required properties comprises requesting from the API information regarding the GUI application.

22. The electronic device as claimed in claim 20, wherein said obtaining required properties comprises receiving the required properties via the API.

23. The electronic device as claimed in claim 20, wherein said obtaining required properties comprises receiving one or more events from the GUI application sends events via the API.

24. The electronic device as claimed in claim 20, wherein said associating comprises providing, based on said one or more elements, information to the API.

25. A non-transitory computer readable medium having instructions stored thereon for enabling voice input for a graphical user interface (GUI)-based application on an electronic device, that when executed by a processor causes the electronic device to:

obtain required properties of one or more user interface objects of the GUI-based application, wherein the one or more user interface objects include one or more input objects;

receive a voice input;

extract from the voice input one or more elements;

associate the one or more elements with the one or more input objects;

identify, based on said associating, an input object having a required property which is not satisfied; and

output, based on the required property, audio output for a prompt for a further voice input.