Visual user interface for creating multimodal applications

• US 20060136870A1
• Filed: 12/22/2004
• Published: 06/22/2006
• Est. Priority Date: 12/22/2004
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Wilson, Leslie Robert, Pietrocarlo, Gary Joseph
• Application Number:
  US11/021,445
• Publication Number:
  US 20060136870A1
• Time in Patent Office: Days
• US Class Current: 717/109
• CPC Class Codes: G06F 8/34 Graphical or visual programming
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence

Abstract

A method to facilitate programming of multimodal access in an integrated development environment (IDE). The method can include receiving at least one user input in a view to create a link between a GUI component and a voice component, and correlating the link to a circumstance under which a voice handler is activated. Multimodal markup code that corresponds to the link can be automatically generated.

Claims

1. A method to facilitate programming of multimodal access in an integrated development environment (IDE), comprising:

receiving at least one user interaction with a view to create a link between at least one graphical user interface (GUI) component and at least a first voice component and correlate said link to at least one circumstance under which a voice handler is activated; and

automatically generating multimodal markup code that corresponds to said link and said at least one circumstance.

2. The method according to claim 1, further comprising displaying in said view at least one multimodal component selected from the group consisting of said GUI component and said first voice component.

3. The method according to claim 2, further comprising displaying in said view a voice handler library comprising a plurality of selectable voice components.

4. The method according to claim 3, further comprising displaying said GUI component and said voice component in a pane in said view, wherein said first voice component is selected from said voice handler library.

5. The method according to claim 1, wherein said step of receiving at least one user interaction comprises receiving at least one identifier that identifies said circumstance.

6. The method according to claim 1, wherein said step of receiving at least one user interaction comprises:

receiving a cursor selection that defines said link between said GUI component and said first voice component; and

receiving at least one identifier that identifies said circumstance.

7. The method according to claim 1, further comprising rendering said GUI component in a pane in said view in accordance with visual markup code.

8. The method according to claim 1, further comprising selectively displaying said view from among a plurality of views in said IDE in response to a said at least one interaction.

9. A machine readable storage, having stored thereon a computer program having a plurality of code sections executable by a machine for causing the machine to perform the steps of:

receiving at least one user interaction with a view to create a link between at least one graphical user interface (GUI) component and at least a first voice component and correlate said link to at least one circumstance under which a voice handler is activated; and

automatically generating multimodal markup code that corresponds to said link and said at least one circumstance.

10. The machine readable storage of claim 9, further causing the machine to perform the step of displaying in said view at least one multimodal component selected from the group consisting of said GUI component and said first voice component.

11. The machine readable storage of claim 10, further causing the machine to perform the step of displaying in said view a voice handler library comprising a plurality of selectable voice components.

12. The machine readable storage of claim 11, further causing the machine to perform the step of displaying said GUI component and said voice component in a pane in said view, wherein said first voice component is selected from said voice handler library.

13. The machine readable storage of claim 9, wherein said step of receiving at least one user interaction comprises receiving at least one identifier that identifies said circumstance.

14. The machine readable storage of claim 9, wherein said step of receiving at least one user interaction comprises:

receiving a cursor selection that defines said link between said GUI component and said first voice component; and

receiving at least one identifier that identifies said circumstance.

15. The machine readable storage of claim 9, further causing the machine to perform the step of rendering said GUI component in a pane in said view in accordance with visual markup code.

16. The machine readable storage of claim 9, further causing the machine to perform the step of selectively displaying said view from among a plurality of views in said IDE in response to said at least one user interaction.

17. An integrated development environment (IDE), comprising:

an IDE that receives at least one user interaction in a view to create a link between at least one GUI component and a first voice component and correlate said link to at least one circumstance under which a voice handler is activated; and

a code module that automatically generates multimodal markup code that corresponds to said link and said at least one circumstance.

18. The IDE of claim 17, wherein at least one multimodal component is displayed in said view, said at least one multimodal component being selected from the group consisting of said GUI component and said first voice component.

19. The IDE of claim 18, wherein a voice handler library comprising a plurality of selectable voice components is displayed in said view.

20. The IDE of claim 17, wherein said at least one user interaction generates at least one identifier that identifies said circumstance.