Navigation and data entry for open interaction elements

• US 7,389,236 B2
• Filed: 03/02/2004
• Issued: 06/17/2008
• Est. Priority Date: 09/29/2003
• Status: Active Grant

• Current Assignee: SAP SE
• Original Assignee: SAP AG (SAP SE)
• Inventors: James, Frances
• Primary Examiner(s): Hudspeth; David R.
• Assistant Examiner(s): KOVACEK, DAVID M
• Application Number:
  US10/790,218
• Publication Number:
  US 20050071172A1
• Time in Patent Office: 1,568 Days
• Field of Search: 704/231, 704/246, 704/250, 704270-275, 379/52, 379 671- 72, 379/75, 379 8801- 8804, 379 8822- 8828
• US Class Current: 704/275
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 2015/228 of application context

Abstract

Techniques for using open interaction elements (OIEs) in voice-enabled systems are described. OIEs include multi-select and dictation fields of a user interface, into which a user may enter multiple selections or free-form information. To allow the user to exit from such OIEs, that is, to change from a data entry mode to a navigation mode, multiple exit strategies may be simultaneously enabled for use by the user. For example, an explicit exit command may be available to the user, in addition to an implicit exit command or time-out exit strategy. In this way, users may individually select a preferred or practical way of exiting an OIE.

Claims

1. A method comprising:

displaying one or more open interaction elements in a voice-enabled user interface;

receiving, in a navigation mode, a navigation command;

determining an open interaction element corresponding to the received navigation command;

entering, in a data entry mode, the open interaction element corresponding to the received navigation command;

enabling an exit option for the entered open interaction element, wherein the exit option provides for exiting the open interaction element;

receiving data for the entered open interaction element;

updating the entered open interaction element with the received data;

determining if an exit option has been selected;

exiting the entered open interaction element if it is determined that the exit option has been selected; and

enabling the navigation mode, if it is determined that the exit option has been selected,

wherein determining the open interaction element corresponding to the received navigation command comprises;

determining one or more open interaction elements that match the received navigation command, wherein each open interaction element belongs to a priority group,

determining matching open interaction elements belonging to the priority group with the highest priority,

if only one matching open interaction element belongs to the highest priority group, selecting the matching open interaction element that belongs to the highest priority group,

if more than one matching open interaction element belongs to the highest priority group, marking each of the matching open interaction elements belonging to the highest priority group in the user interface with a unique number,

receiving a navigation command indicating one of the unique numbers, and

selecting the open interaction element corresponding to the indicated unique numbers.

2. The method of claim 1, wherein determining if an exit option has been selected comprises determining if an explicit exit command has been received.

3. The method of claim 1, wherein determining if an exit option has been selected comprises determining if an implicit exit command has been received.

4. The method of claim 1, wherein determining if an exit option has been selected comprises determining if a timeout period has expired.

5. The method of claim 1, wherein determining if an exit option has been selected comprises determining if a tab command has been received.

6. The method of claim 1, wherein the open interaction element is adapted to receive multiple data entries.

7. The method of claim 1, wherein receiving data for the entered open interaction element comprises a voice module translating voice data to text data using a speech recognition engine, and receiving the text data from the voice module.

8. The method of claim 7, wherein the voice module translating voice data to text data using a speech recognition engine further comprises retrieving a grammar associated with the entered open interaction element, and translating voice data to text data using the speech recognition engine and the retrieved grammar.

9. The method of claim 8, wherein the grammar is one of a date grammar and a name grammar.

10. The method of claim 1, wherein marking each of the matching open interaction elements belonging to the highest priority group in the user interface with a unique number comprises displaying semi-transparent overlays of the unique number over the corresponding open interaction elements in the user interface.

11. A method comprising:

receiving a user interface, the user interface including user interface elements;

parsing the received user interface to locate user interface elements;

processing the located user interface elements to generate voice enabled user interface elements;

prioritizing the voice enabled user interface elements into priority groups based on their location in the user interface; and

displaying the received user interface including the voice enabled user interface elements,

wherein processing the located user interface elements to make them voice enabled comprises;

translating each located user interface element to create a speakable identifier, and

associating the speakable identifier with the corresponding user interface element; and

wherein translating each located user interface element to create a speakable identifier, comprises;

extracting text from the user interface element;

using a speech recognition engine to generate a speakable identifier from the extracted text; and

adding the generated speakable identifier to a library of speakable identifiers.

12. The method of claim 11, wherein the user interface elements are XML elements or HTML elements.

13. A device comprising:

a display module adapted to display a user interface, the user interface comprising a plurality of open interaction elements; and

a navigation and data entry module adapted to;

receive, in a navigation mode, a navigation command;

determine an open interaction element corresponding to the received navigation command;

enter, in a data entry mode, the open interaction element corresponding to the received navigation command;

enable an exit option for the entered open interaction element, wherein the exit option provides for exiting the open interaction element;

receive data for the entered open interaction element;

update the entered open interaction element with the received data;

determine if the exit option has been selected;

exit the entered open interaction element if it is determined that the exit option has been selected; and

enable the navigation mode, if it is determined that the exit option has been selected,

wherein, in determining the open interaction element corresponding to the received navigation command, the navigation and data entry module is further adapted to;

determine one or more open interaction elements that match the received navigation command, wherein each open interaction element belongs to a priority group,

determine matching open interaction elements belonging to the priority group with the highest priority,

if only one matching open interaction element belongs to the highest priority group, select the matching open interaction element that belongs to the highest priority group,

if more than one matching open interaction element belongs to the highest priority group, mark each of the matching open interaction elements belonging to the highest priority group in the user interface with a unique number,

receive a navigation command indicating one of the unique numbers, and

select the open interaction element corresponding to the indicated unique numbers.

14. A computer readable medium encoded with a computer program comprising instructions that, when read by a machine, operate to cause a data processing apparatus to:

display one or more open interaction elements in a voice-enabled user interface;

receive, in a navigation mode, a navigation command;

determine an open interaction element corresponding to the received navigation command;

enter, in a data entry mode, the open interaction element corresponding to the received navigation command;

enable an exit option for the entered open interaction element, wherein the exit option provides for exiting the open interaction element;

receive data for the entered open interaction element;

update the entered open interaction element with the received data;

determine if the exit option has been selected;

exit the entered open interaction element if it is determined that the exit option has been selected; and

enable the navigation mode, if it is determined that the exit option has been selected,

wherein, in determining the open interaction element corresponding to the received navigation command, the instructions further operate to cause the data processing apparatus to;

determine one or more open interaction elements that match the received navigation command, wherein each open interaction element belongs to a priority group,

determine matching open interaction elements belonging to the priority group with the highest priority,

if only one matching open interaction element belongs to the highest priority group, select the matching open interaction element that belongs to the highest priority group,

if more than one matching open interaction element belongs to the highest priority group, mark each of the matching open interaction elements belonging to the highest priority group in the user interface with a unique number,

receive a navigation command indicating one of the unique numbers, and

select the open interaction element corresponding to the indicated unique numbers.