Automated voice enablement of applications
First Claim
Patent Images
1. A method, comprising:
- receiving application data of an application that is under development from an application developer computing system;
aggregating static images of a graphical user interface of the application (GUI screens) that are contained within the application data;
analyzing each GUI screen to determine GUI screen navigational flow paths (flow paths);
generating a flow path data structure comprising a list of the flow paths ranked by the number of edge objects comprised within the flow paths and comprising a list of edge objects within the flow paths ranked by the number of dependent edge objects;
identifying a first edge object within the flow path data structure as a select object;
determining a word or phrase associated with the select object from characters that are displayed by the select object with a natural language processing system;
determining synonyms to the word or phrase with the natural language processing system;
generating a select object synonym data structure comprising the word or phrase and the synonyms to the word or phrase associated with the select object;
creating a voice user interface (VUI) select object data structure from the select object synonym data structure;
sending the VUI select object data structure to the application developer computing system to create an application with VW modality; and
wherein a first GUI screen navigational flow path is a first navigational sequence of GUI screens associated with a first set of user interactions with the GUI screens and a second GUI screen navigational flow path is a second navigational sequence of GUI screens associated with a second set of user interactions with the GUI screens.
1 Assignment
0 Petitions
Accused Products
Abstract
Techniques are disclosed for generating a voice user interface (VUI) modality within an application that includes graphical user interface (GUI) screens. A GUI screen parser analyzes the GUI screens to determine the various navigational GUI screen paths that are associated with edge objects within multiple GUI screens. Some edge objects are identified as select objects or prompt objects. A natural language processing system generates a select object synonym data structure and a prompt object data structure that may be utilized by a VUI generator to generate VUI data structures that give the application VUI modality.
-
Citations
16 Claims
-
1. A method, comprising:
-
receiving application data of an application that is under development from an application developer computing system; aggregating static images of a graphical user interface of the application (GUI screens) that are contained within the application data; analyzing each GUI screen to determine GUI screen navigational flow paths (flow paths); generating a flow path data structure comprising a list of the flow paths ranked by the number of edge objects comprised within the flow paths and comprising a list of edge objects within the flow paths ranked by the number of dependent edge objects; identifying a first edge object within the flow path data structure as a select object; determining a word or phrase associated with the select object from characters that are displayed by the select object with a natural language processing system; determining synonyms to the word or phrase with the natural language processing system; generating a select object synonym data structure comprising the word or phrase and the synonyms to the word or phrase associated with the select object; creating a voice user interface (VUI) select object data structure from the select object synonym data structure; sending the VUI select object data structure to the application developer computing system to create an application with VW modality; and wherein a first GUI screen navigational flow path is a first navigational sequence of GUI screens associated with a first set of user interactions with the GUI screens and a second GUI screen navigational flow path is a second navigational sequence of GUI screens associated with a second set of user interactions with the GUI screens. - View Dependent Claims (3, 4, 6, 7, 8)
-
-
2. The method of claim wherein the select object is a GUI object that begins or continues an associated GUI screen flow path upon the selection of the GUI object.
-
5. The method of claim wherein the GUI screens are aggregated from mockup images of the application under development.
-
9. A computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions readable by a service computing system to cause the service computing system to:
-
receive application data of an application that is under development from an application developer computing system; aggregate static images of a graphical user interface of the application (GUI screens) that are contained within the application data; analyze each GUI screen to determine GUI screen navigational flow paths (flow paths); generate a flow path data structure comprising a list of the flow paths ranked by the number of edge objects comprised within the flow paths and comprising a list of edge objects within the flow paths ranked by the number of dependent edge objects; identify a first edge object within the flow path data structure as a select object;
determine a word or phrase associated with the select object from characters that are displayed by the select object with a natural language processing system;determine synonyms to the word or phrase with the natural language processing system; generate a select object synonym data structure comprising the word or phrase and the synonyms to the word or phrase associated with the select object; create a voice user interface (VUI) select object data structure from the select object synonym data structure; send the VUI select object data structure to the application developer computing system to create an application with VUI modality; and wherein a first GUI screen navigational flow path is a first navigational sequence of GUI screens associated with a first set of user interactions with the GUI screens and a second GUI screen navigational flow path is a second navigational sequence of GUI screens associated with a second set of user interactions with the GUI screens. - View Dependent Claims (10, 11, 12, 13, 15, 16)
-
-
14. The computer program product of claim wherein the GUI screens are aggregated from storyboard images of the application under development.
Specification