Automated voice enablement of applications
First Claim
Patent Images
1. A method, comprising:
- identifying application data of an application that is under development within an application developer computing system;
aggregating static images of a graphical user interface of the application (GUI screens) that are contained within the application data;
analyzing each GUI screen to determine GUI screen navigational flow paths (flow paths);
generating a flow path data structure comprising a list of the flow paths ranked by the number of edge objects comprised within the flow paths and comprising a list of edge objects within the flow paths ranked by the number of dependent edge objects;
identifying a first edge object within the flow path data structure as a select object;
determining a word or phrase associated with the select object from characters that are displayed by the select object with a natural language processing system;
determining synonyms to the word or phrase with the natural language processing system;
generating a select object synonym data structure comprising the word or phrase and the synonyms to the word or phrase associated with the select object;
creating a voice user interface (VUI) select object data structure from the select object synonym data structure;
storing the VUI select object data structure within a VUI portion of the application that is under development to create an application with VUI modality; and
wherein a first GUI screen navigational flow path is a first navigational sequence of GUI screens associated with a first set of user interactions with the GUI screens and a second GUI screen navigational flow path is a second navigational sequence of GUI screens associated with a second set of user interactions with the GUI screens.
1 Assignment
0 Petitions
Accused Products
Abstract
Techniques are disclosed for generating a voice user interface (VUI) modality within an application that includes graphical user interface (GUI) screens. A GUI screen parser analyzes the GUI screens to determine the various navigational GUI screen paths that are associated with edge objects within multiple GUI screens. Some edge objects are identified as select objects or prompt objects. A natural language processing system generates a select object synonym data structure and a prompt object data structure that may be utilized by a VUI generator to generate VUI data structures that give the application VUI modality.
21 Citations
16 Claims
-
1. A method, comprising:
-
identifying application data of an application that is under development within an application developer computing system; aggregating static images of a graphical user interface of the application (GUI screens) that are contained within the application data; analyzing each GUI screen to determine GUI screen navigational flow paths (flow paths);
generating a flow path data structure comprising a list of the flow paths ranked by the number of edge objects comprised within the flow paths and comprising a list of edge objects within the flow paths ranked by the number of dependent edge objects;identifying a first edge object within the flow path data structure as a select object;
determining a word or phrase associated with the select object from characters that are displayed by the select object with a natural language processing system;determining synonyms to the word or phrase with the natural language processing system; generating a select object synonym data structure comprising the word or phrase and the synonyms to the word or phrase associated with the select object; creating a voice user interface (VUI) select object data structure from the select object synonym data structure; storing the VUI select object data structure within a VUI portion of the application that is under development to create an application with VUI modality; and wherein a first GUI screen navigational flow path is a first navigational sequence of GUI screens associated with a first set of user interactions with the GUI screens and a second GUI screen navigational flow path is a second navigational sequence of GUI screens associated with a second set of user interactions with the GUI screens. - View Dependent Claims (2, 4, 5, 6, 7, 8)
-
-
3. The method of claim wherein the GUI screens are aggregated screenshots of the application under development that is evoked by the application developer computing system.
-
9. A computer program product comprising, a computer readable storage medium having program instructions embodied therewith, the program instructions readable by an application developer computing system to cause the application developer computing system to:
-
identify application data. of an application that is under development; aggregate static images of a graphical user interface of the application (GUI screens) that are contained within the application data; analyze each GUI screen to determine GUI screen navigational flow paths (flow paths); generate a flow path data structure comprising a list of flow paths ranked by the number of edge objects comprised within each flow path and comprising a list of edge objects within the flow paths ranked by the number of dependent edge objects; identify a first edge object within the flow path data structure as a select object;
determine a word or phrase associated with the select object from characters that are displayed by the select object with a natural language processing system;determine synonyms to the word or phrase with the natural language processing system;
generate a select object synonym data, structure comprising the word or phrase and the synonyms to the word or phrase associated with the select object;create a voice user interface (VUI) select object data structure from the select object synonym data structure; store the VUI select object data structure within a VUI portion of the application that is under development to create an application with modality; and
wherein a first GUI screen navigational flow path is a first navigational sequence of GUI screens associated with a first set of user interactions with the GUI screens and a second GUI screen navigational flow path is a second navigational sequence of GUI screens associated with a second set of user interactions with the GUI screens. - View Dependent Claims (10, 11, 12, 13, 14, 15, 16)
-
Specification