Software development systems and methods

• US 20020077823A1
• Filed: 03/30/2001
• Published: 06/20/2002
• Est. Priority Date: 10/13/2000
• Status: Abandoned Application

• Current Assignee: Andrew Fox, Bin Liu, Jeffrey M. Hill, Lorin Wilde, Michael Tinglof, Tim F. Rochford, Toffee A. Albina
• Original Assignee: Andrew Fox, Bin Liu, Jeffrey M. Hill, Lorin Wilde, Michael Tinglof, Tim F. Rochford, Toffee A. Albina
• Inventors: Liu, Bin, Wilde, Lorin, Rochford, Tim F., Tinglof, Michael, Fox, Andrew, Hill, Jeffrey M., Albina, Toffee A.
• Application Number:
  US09/822,590
• Publication Number:
  US 20020077823A1
• Time in Patent Office: Days
• US Class Current: 704/260
• CPC Class Codes: G06F 8/34 Graphical or visual programming

Abstract

A software development method and apparatus is provided for the simultaneous creation of software applications that operate on a variety of client devices and include text-to-speech and speech recognition capabilities. A software development system and related method use a graphical user interface that provides a software developer with an intuitive drag and drop technique for building software applications. Program elements, accessible with the drag and drop technique, include corresponding markup code that is adapted to operate on the plurality of different client devices. The software developer can generate a natural language grammar by providing typical or example spoken responses. The grammar is automatically enhanced to increase the number of recognizable words or phrases. The example responses provided by the software developer are further used to automatically build application-specific help. At application runtime, a help interface can be triggered to present these illustrative spoken prompts to guide the end user in responding.

Claims

1. A method of creating a software application, the method comprising the steps of:

accepting a selection of a plurality of target devices;

determining capability parameters for each target device;

rendering a representation of each target device on a graphical user interface;

receiving input from a developer creating the software application;

simulating, in substantially real time and in response to the input, at least a portion of the software application on each target device; and

displaying a result of the simulation on the graphical user interface.

2. The method of claim 1 further comprising the steps of:

defining at least one page of the software application;

associating at least one program element with the at least one page, the at least one program element including a corresponding markup code;

storing the corresponding markup code; and

adapting, in response to the capability parameters, the corresponding markup code to each target device substantially simultaneously.

3. The method of claim 2 wherein the corresponding markup code comprises MTML code.

4. The method of claim 2 further comprising the steps of:

defining content ancillary to the software application; and

associating the ancillary content with the at least one program element.

5. The method of claim 4 wherein the step of defining ancillary content further comprises the steps of:

generating a content source identification file;

generating a request schema;

generating a response schema; and

generating a sample data file.

6. The method of claim 5 further comprising the step of generating a request transform and a response transform.

7. The method of claim 2 wherein the at least one page of the software application comprises at least one of a setup section, a completion section, and a form section.

8. The method of claim 2 further comprising the step of associating Java-based code with the at least one page.

9. The method of claim 2 further comprising the step of associating at least one resource with the at least one program element, wherein the at least one resource comprises at least one of a text prompt, an audio file, a natural language grammar file, and a graphic image.

10. The method of claim 2 wherein the rendering step further comprises displaying a voice conversation template in response to the at least one program element.

11. The method of claim 10 further comprising the step of accepting changes to the voice conversation template.

12. The method of claim 2 further comprising the steps of:

transferring an application definition file to a repository; and

creating, in response to the application definition file, at least one of a Java server page, an XSL style sheet, and an XML file, wherein the Java server page includes software code to (i) identify a client device, (ii) invoke at least a portion of the XSL style sheet, (iii) generate a client-side markup code, and (iv) transmit the client-side markup code to the client device.

13. The method of claim 12 wherein the client-side markup code comprises at least one of WML code, HTML code, and VoiceXML code.

14. The method of claim 12 wherein the application definition file comprises at least one of a source code file, a layout file, and a resource file.

15. The method of claim 12 wherein the step of transferring an application definition file is accomplished using a standard protocol.

16. The method of claim 12 further comprising the step of creating at least one static page in a predetermined format.

17. The method of claim 16 wherein the predetermined format comprises the PQA format.

18. A visual programming apparatus for creating a software application for a plurality of target devices, the visual programming system comprising:

a target device database for storing device-specific profile information;

a graphical user interface that is responsive to input from a developer;

a plurality of program elements for constructing the software application, each program element including corresponding markup code;

a rendering engine in communication with the graphical user interface and the target device database for displaying a representation of the target devices;

a translator in communication with the graphical user interface and the target device database for creating at least one layout element in at least one layout file and linking the corresponding markup code to the at least one layout element; and

at least one simulator in communication with the graphical user interface and the target device database for simulation of at least a portion of the software application and displaying the results of the simulation on the graphical user interface.

19. An article of manufacture comprising a program storage medium having computer readable program code embodied therein for causing the creation of a software application, the computer readable program code in the article of manufacture including:

computer readable code for causing a computer to accept a selection of a plurality of target devices;

computer readable code for causing a computer to determine capability parameters for each target device;

computer readable code for causing a computer to render a representation of each target device on a graphical user interface;

computer readable code for causing a computer to define at least one page of the software application;

computer readable code for causing a computer to associate at least one program element with the at least one page, the at least one program element including a corresponding markup code;

computer readable code for causing a computer to store the corresponding markup code;

computer readable code for causing a computer to adapt, in response to the capability parameters, the corresponding markup code to each target device substantially simultaneously;

computer readable code for causing a computer to simulate, in substantially real time and in response to the capability parameters and the at least one program element, at least a portion of the software application on each target device; and

computer readable code for causing a computer to display a result of the simulation on the graphical user interface, so as to achieve the creation of a software application.

20. A program storage medium readable by a computer, tangibly embodying a program of instructions executable by the computer to perform method steps for creating a software application, the method steps comprising:

accepting a selection of a plurality of target devices;

determining capability parameters for each target device;

rendering a representation of each target device on a graphical user interface;

defining at least one page of the software application;

associating at least one program element with the at least one page, the at least one program element including a corresponding markup code;

storing the corresponding markup code;

adapting, in response to the capability parameters, the corresponding markup code to each target device substantially simultaneously;

simulating, in substantially real time and in response to the capability parameters and the at least one program element, at least a portion of the software application on each target device; and

displaying a result of the simulation on the graphical user interface, so as to achieve the creation of a software application.

21. A method of creating a natural language grammar, the method comprising the steps of:

accepting at least one example user response phrase appropriately responsive to a specific query;

identifying at least one variable in the at least one example user response phrase, the at least one variable having a corresponding value;

specifying a data type for the at least one variable;

associating a subgrammar with the at least one variable;

replacing a portion of the at least one example user response phrase, the portion including the at least one variable, with a reference to the subgrammar; and

defining a computation to be performed by the subgrammar, the computation providing the corresponding value of the at least one variable.

22. The method of claim 21, wherein the step of identifying at least one variable further comprises the steps of:

selecting a segment of the example user response phrase, the segment including the at least one variable; and

copying the segment of the example user response phrase to a grammar template.

23. The method of claim 21, wherein the step of identifying at least one variable further comprises the steps of:

entering the corresponding value of the at least one variable; and

parsing the at least one example user response phrase to locate the at least one variable capable of having the corresponding value.

24. The method of claim 21 further comprising the step of normalizing the at least one example user response phrase.

25. The method of claim 21 further comprising the step of specifying a desired degree of generalization.

26. The method of claim 21 further comprising the steps of:

determining whether the corresponding value is restricted to a set of values and, if so restricted;

generating a subset of phrases associated with the set of values;

removing from the subset of phrases those phrases deemed not sufficiently specific; and

creating at least one flat grammar based at least in part on each remaining phrase in the subset.

27. The method of claim 26 wherein the subgrammar comprises the flat grammar.

28. The method of claim 21 further comprising the step of creating a language model based at least in part on words in the at least one example user response phrase.

29. The method of claim 21 further comprising the step of creating a pronunciation dictionary based at least in part on the at least one example user response phrase, the pronunciation dictionary including at least one pronunciation for each word therein.

30. A natural language grammar generator comprising:

a graphical user interface that is responsive to input from a developer, the input including at least one example user response phrase;

a subgrammar database for storing subgrammars to be associated with the at least one example user response phrase;

a normalizer in communication with the graphical user interface for standardizing orthography in the at least one example user response phrase;

a generalizer in communication with the graphical user interface for operating on the at least one example user response phrase to create at least one additional example user response phrase;

a parser in communication with the graphical user interface for operating on the at least one example user response phrase and identifying at least one variable therein; and

a mapping apparatus in communication with the parser and the subgrammar database for associating the at least one variable with at least one subgrammar.

31. An article of manufacture comprising a program storage medium having computer readable program code embodied therein for causing the creation of a natural language grammar, the computer readable program code in the article of manufacture including:

computer readable code for causing a computer to accept at least one example user response phrase appropriately responsive to a specific query;

computer readable code for causing a computer to identify at least one variable in the at least one example user response phrase, the at least one variable having a corresponding value;

computer readable code for causing a computer to specify a data type for the at least one variable;

computer readable code for causing a computer to associate a subgrammar with the at least one variable;

computer readable code for causing a computer to replace a portion of the at least one example user response phrase, the portion including the at least one variable, with a reference to the subgrammar; and

computer readable code for causing a computer to define a computation to be performed by the subgrammar, the computation providing the corresponding value of the at least one variable, so as to achieve the creation of a natural language grammar.

32. A program storage medium readable by a computer, tangibly embodying a program of instructions executable by the computer to perform method steps for creating a natural language grammar, the method steps comprising:

accepting at least one example user response phrase appropriately responsive to a specific query;

identifying at least one variable in the at least one example user response phrase, the at least one variable having a corresponding value;

specifying a data type for the at least one variable;

associating a subgrammar with the at least one variable;

replacing a portion of the at least one example user response phrase, the portion including the at least one variable, with a reference to the subgrammar; and

defining a computation to be performed by the subgrammar, the computation providing the corresponding value of the at least one variable, so as to achieve the creation of a natural language grammar.

33. A method of providing speech-based assistance during execution of an application, the method comprising the steps of:

receiving a signal from an end user;

processing the signal using a speech recognizer; and

determining, from the processed signal, whether speech-based assistance is appropriate and, if appropriate, (i) accessing at least one of an example user response phrase and a grammar, and (ii) transmitting, to the end user, at least one assistance prompt, wherein the at least one assistance prompt is the example user response phrase, or a phrase generated in response to the grammar.

34. A method of creating a dynamic grammar, the method comprising the steps of:

determining, at application runtime, whether a value corresponding to at least one variable, the at least one variable included in at least one example user response phrase, is restricted to a set of values and, if so restricted;

generating a subset of phrases associated with the set of values;

removing from the subset of phrases those phrases deemed not sufficiently specific;

creating at least one flat grammar based at least in part on each remaining phrase in the subset;

creating at least one language model corresponding to the at least one flat grammar; and

creating at least one pronunciation dictionary corresponding to the at least one flat grammar.

35. A speech-based assistance generator comprising:

a receiver for receiving a signal from an end user;

a speech recognition engine for processing the signal, the speech recognition engine in communication with the receiver;

logic that determines from the processed signal whether speech-based assistance is appropriate;

logic that accesses at least one example user response phrase;

logic that accesses at least one grammar; and

a transmitter for sending to the end user at least one assistance prompt, wherein the at least one assistance prompt is the at least one example user response phrase, or a phrase generated in response to the grammar.

36. An article of manufacture comprising a program storage medium having computer readable program code embodied therein for providing speech-based assistance during execution of an application, the computer readable program code in the article of manufacture including:

computer readable code for causing a computer to receive a signal from an end user;

computer readable code for causing a computer to process the signal using a speech recognizer; and

computer readable code for causing a computer to determine, from the processed signal, whether speech-based assistance is appropriate and, if appropriate, causing a computer to (i) access at least one of an example user response phrase and a grammar, and (ii) transmit, to the end user, at least one assistance prompt, wherein the at least one assistance prompt is the example user response phrase, or a phrase generated in response to the grammar, so as to provide speech-based assistance.

37. A program storage medium readable by a computer, tangibly embodying a program of instructions executable by the computer to perform method steps for providing speech-based assistance, the method steps comprising:

receiving a signal from an end user;

processing the signal using a speech recognizer;

determining, from the processed signal, whether speech-based assistance is appropriate and, if appropriate, (i) accessing at least one of an example user response phrase and a grammar, and (ii) transmitting, to the end user, at least one assistance prompt, wherein the at least one assistance prompt is the example user response phrase, or a phrase generated in response to the grammar, so as to provide speech-based assistance.