Audible presentation and verbal interaction of HTML-like form constructs

• US 9,256,854 B2
• Filed: 06/18/2008
• Issued: 02/09/2016
• Est. Priority Date: 09/22/2000
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Callaghan, Patrick, Feustel, Stephen V., Howland, Michael J., Pritko, Steven M.
• Primary Examiner(s): BLOOMQUIST, KEITH D
• Application Number:
  US12/141,450
• Publication Number:
  US 20080255838A1
• Time in Patent Office: 2,792 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06Q 10/10 Office automation; Time management

Abstract

A method of synchronizing an audio and visual presentation in a multi-modal browser. A form is transmitted over a network having at least one field requiring user supplied information to a multi-modal browser. Blank fields within the form are filled in by user who provides either verbal or tactile interaction, or a combination of verbal and tactile interaction. The browser moves to the next field requiring user provided input. Finally, the form exits after the user has supplied input for all required fields. The method also provides a synchronized verbal and visual presentation by said browser by having the headings for the fields to be filled out and typing in what the user says.

Claims

1. A method for completing a form, said method comprising:

a client workstation storing an audiovisual form in a written markup language, said form defining fields with respective headings and respective blank areas to be updated with text;

a web browser in said client workstation displaying said form with the respective headings and respective blank areas to be updated with text, audibly reading one of said headings and waiting for a user to audibly respond with corresponding text for the blank area associated with said one heading, wherein in response to said web browser progressing to said one heading, while said web browser audibly reads said one heading, said web browser automatically displaying a plurality of valid alternatives for said blank area associated with said one heading, one of said valid alternatives being said corresponding text;

in response to said user speaking a command for the web browser to accept keyboard entry of said corresponding text, said web browser accepting subsequent entry of said corresponding text for said blank area associated with said one heading and displaying an updated state of the form with said one heading and with said corresponding text entered in said blank area associated with said one heading;

in response to all fields in the form having been processed, said web browser generating a pause for a specified period of time; and

in response to the specified period of time of the pause having expired, said web browser looping back to a top of the form.

2. The method of claim 1, said method further comprising:

said web browser updating the written markup language for said form to include said corresponding text for said blank area associated with said one heading; and

said web browser sending said updated written markup language to a server via a network.

3. The method of claim 2, said method further comprising:

after said web browser performs the steps of updating the written markup language for said form and displaying an updated state of the form with said one heading and with said corresponding text typed in said blank area associated with said one heading, said web browser ceases to display said plurality of valid alternatives for said blank area, and subsequently, said web browser audibly reading another of said headings and waiting for a user to speak another text for the blank area associated with said other heading, and in response to said user speaking said other text, said web browser updating the written markup language for said form to include said other text for said blank area associated with said other heading and displaying an updated state of the form with said other heading and with said other text typed in said blank area associated with said other heading.

4. The method of claim 1, wherein in response to said web browser progressing to said one heading, while said web browser audibly reads said one heading, said method further comprising:

said web browser automatically graphically indicating that said blank area associated with said one heading, and not any other blank area associated with any other heading, is currently waiting for said corresponding text from said user, and will include said corresponding text after spoken by said user.

5. The method of claim 1, said method further comprising:

said web browser responding to a spoken command by said user to skip entry of text into said blank area associated with said one heading and advance to a next one of said fields, said user speaking next text for a blank area of said next one of said fields, and in response to said user speaking said next text for said blank area of said next one of said fields, said web browser updating the written markup language for said form to include said next text for said blank area for said next one of said fields and displaying an updated state of the form with said next text for said blank area for said next one of said fields entered in said blank area for said next one of said fields.

6. The method of claim 1, said method further comprising:

after said displaying, said web browser proceeding from a first field with said one heading and said blank area associated with said one heading to a second field with a second heading and a second blank area associated with said second heading;

after said proceeding to the second field and in response to said user speaking a skip command to skip from the second field to a third field with a third heading and a third blank area associated with said third heading, said web browser skipping from the second field to the third field.

7. The method of claim 1, said method further comprising:

in response to said user speaking a review command, said web browser reviewing the form to determine whether all fields in the form contain information.

8. The method of claim 1, said method further comprising:

in response to said user speaking a submit command, said web browser submitting the form to an application program to process the form.

9. The method of claim 1, said method further comprising:

in response to said user speaking a cancel command, said web browser erasing information currently within a field in the form.

10. The method of claim 1, said method further comprising:

after said displaying and in response to said user speaking a reset command, said web browser clearing information previously entered in the form and reprocessing the form, wherein said reprocessing comprises rereading the form to the user.

11. The method of claim 1, said method further comprising:

after said displaying, said web browser proceeding from a first field with said one heading and said blank area associated with said one heading to a second field with a second heading and a second blank area associated with said second heading;

in response to said proceeding to the second field, said web browser enabling a plurality of commands specific to the second field;

after the second field has been processed, said web browser proceeding from the second field to a third field with a third heading and a third blank area associated with said third heading; and

after the second field has been processed and before said proceeding to the third field, said web browser disabling the plurality of commands specific to the second field.