Partially automated technology for converting a graphical interface to a speech-enabled interface

• US 20070233495A1
• Filed: 03/29/2006
• Published: 10/04/2007
• Est. Priority Date: 03/29/2006
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Reich, David, Lewis, James, Gomez, Felipe, Agapi, Ciprian, Mandalia, Baiju, Metz, Brent, Horowitz, Kevin
• Application Number:
  US11/391,825
• Publication Number:
  US 20070233495A1
• Time in Patent Office: Days
• US Class Current: 704/270
• CPC Class Codes: G06F 8/38 for implementing user interfaces
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: H04M 2203/355 Interactive dialogue design tools, features or methods
• CPC Class Codes: H04M 3/4938 comprising a voice browser which renders and interprets, e.g. VoiceXML

Abstract

A method for constructing speech elements within an interface can include a step of identifying a visual interface having multiple visual elements. Visual selectors can be presented proximate each of the visual elements. The visual selectors can permit a user to input a speech control type for the associated visual element. For each presented visual selector, a speech element having a speech control type specified in the visual selector can be automatically generated.

Claims

1. A method for constructing speech elements within an interface comprising:

identifying a visual interface having a plurality of visual elements;

presenting visual selectors proximate each of the visual elements, wherein the visual selectors permit a user to input a speech control type for the associated visual element; and

for each presented visual selector, automatically generating a speech element having a speech control type specified in the visual selector.

2. The method of claim 1, further comprising:

automatically detecting the visual elements, wherein the visual selectors are automatically constructed and presented responsive to the detecting step.

3. The method of claim 2, wherein at least a portion of the visual selectors are associated with a user selectable listing of speech control types, wherein each control type in the listing corresponds to a Reusable Dialog Component.

4. The method of claim 1, further comprising:

automatically determining an initial speech control type for each of the visual selectors, wherein the presenting step initially populates the visual selectors with the determined initial speech control type, which the user is able to change.

5. The method of claim 2, further comprising:

before the generating step, permitting a user to selectively modify a number of visual selectors associated with visual elements, wherein the generating step only generates speech elements associated with a visual selector.

6. The method of claim 1, wherein the visual interface is part of a page written in a markup language, wherein said page is not initially speech-enabled, wherein the generating step automatically creates a speech-enabled page written in a markup language.

7. The method of claim 6, wherein the created speech-enabled page is new Web page having a speech user interface.

8. The method of claim 6, further comprising:

automatically detecting a change to at least one of an original page including the visual interface and the created speech-enabled page; and

triggering a synchronization event designed to synchronize the original page and the created speech-enabled page.

9. The method of claim 8, further comprising:

responsive to the triggering step, automatically conveying a notification to a previously designated user associated with at least one of the original page and the created speech-enabled page, said notification indicating the detected change.

10. The method of claim 6, wherein the created speech-enabled page is a multimodal Web page that includes the plurality of visual elements and the speech elements.

11. The method of claim 1, wherein the identifying, presenting, and generated steps are preformed using a graphical software design tool.

12. The method of claim 11, wherein the visual interface is constructed utilizing the software design tool.

13. The method of claim 12, wherein the software design tool includes a call flow design interface within which the automatically generated speech elements are able to be graphically manipulated.

14. The method of claim 1, wherein said steps of claim 1 are performed by at least one machine in accordance with at least one computer program having a plurality of code sections that are executable by the at least one machine.

15. A software development application comprising:

a visual design window for designing visual elements of a visual interface and for automatically generating programmatic instructions associated with designed visual elements;

a selector enabled window configured to graphically display the visual elements, wherein at least a portion of the displayed elements are associated with displayed visual selectors, wherein each visual selector permits a user of the software development application to input a speech control type; and

a SUI element generation engine for automatically generating SUI elements corresponding to each GUI element that is associated with a visual selector, wherein each generated SUI element has a speech control type as specified by the visual selector.

16. The software development application of claim 15, further comprising:

a call flow design interface configured to graphically present a call flow for the automatically generated SUI elements.

17. The software development application of claim 15, wherein each of the speech control types correspond to a Reusable Dialog Component, wherein the Reusable Dialog Component is used to automatically generate programmatic instructions for each of the SUI elements.

18. The software development application of claim 15, wherein the programmatic instructions are written in a markup language renderable by a visual browser and wherein the SUI elements are associated with programmatic instructions written in a speech-enabled markup language renderable by a speech-enabled browser.

19. A graphical user interface comprising:

a window for rendering markup written in a visual markup language;

a plurality of visual selectors graphically rendered in the window that are not specified in the visual markup language, wherein each visual selector corresponds to a visual element displayed in the window, wherein each visual selector permits a user to designate a speech control type, wherein for each visual selector, a speech-enabled element is automatically generated that has the designated speech control type, and wherein automatically generated markup written in a speech-enabled markup language is created for each of the speech-enabled elements.

20. The graphical user interface of claim 19, wherein the graphical user interface is part of a software development application that facilitates the creation of speech-enabled elements from graphical elements using a partially automated technique that converts visual markup to speech-enabled markup, wherein the converting is based partially upon designer specified, pre-generation parameters, which are specified using the visual selectors.