Systems, devices, and methods for automated and programmatic creation and deployment of remediations to non-compliant web pages or user interfaces

• US 10,423,709 B1
• Filed: 08/16/2018
• Issued: 09/24/2019
• Est. Priority Date: 08/16/2018
• Status: Active Grant

• Current Assignee: AudioEye, Inc.
• Original Assignee: AudioEye, Inc.
• Inventors: Bradley, Sean D., Baker, Mark D., Jones, Jeffrey O., Hefner, Kenny P., Finkelstein, Adam, Gilormo, Douglas J., Bodnar, Taylor R., Pinckney, David C., Blevins, Charlie E., Laymon, Helena, Jones, Trevor C., Carrillo, Damien
• Primary Examiner(s): Paula, Cesar B
• Assistant Examiner(s): Nguyen, Luu-Phuong T
• Application Number:
  US15/999,116
• Time in Patent Office: 404 Days
• Field of Search: 715205
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9577 Optimising the visualization of content, e.g. distillation of HTML documents
• CPC Class Codes: G06F 16/986 Document structures and storage, e.g. HTML extensions
• CPC Class Codes: G06F 21/101 by binding digital rights to specific entities
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/103 Formatting, i.e. changing of presentation of documents automatic justification G06F40/189; automatic line break hyphenation G06F40/191
• CPC Class Codes: G06F 40/134 Hyperlinking
• CPC Class Codes: G06F 40/137 Hierarchical processing, e.g. outlines
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G06F 40/16 Automatic learning of transformation rules, e.g. from examples
• CPC Class Codes: G06F 40/174 Form filling; Merging
• CPC Class Codes: G06F 40/216 using statistical methods
• CPC Class Codes: G06F 40/221 Parsing markup language streams streaming G06F40/149
• CPC Class Codes: G06F 40/226 Validation
• CPC Class Codes: G06F 40/279 Recognition of textual entities
• CPC Class Codes: G10L 13/00 Speech synthesis; Text to speech systems
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

Systems and methods are disclosed for manually and programmatically remediating websites to thereby facilitate website navigation by people with diverse abilities. For example, an administrator portal is provided for simplified, form-based creation and deployment of remediation code, and a machine learning system is utilized to create and suggest remediations based on past remediation history. Voice command systems and portable document format (PDF) remediation techniques are also provided for improving the accessibility of such websites.

Claims

1. A system for automated and programmatic creation and deployment of remediations to non-compliant web pages or user interfaces, the system comprising:

one or more remediation code databases configured to store a plurality of remediation code blocks;

a JavaScript code database comprising one or more JavaScript codes for creating client-side interactivity with one or more web pages or user interfaces, the one or more web pages or user interfaces comprising a hypertext markup language (HTML) document object model (DOM), wherein the DOM comprises one or more nodes, the one or more nodes organized into a DOM tree structure;

one or more computer readable storage devices configured to store a plurality of computer executable instructions; and

one or more hardware computer processors in communication with the one or more computer readable storage devices and configured to execute the plurality of computer executable instructions in order to cause the system to;

trigger at least one of the one or more JavaScript codes to be embedded within a browser of a user device;

store, via the at least one of the one or more JavaScript codes embedded within the browser, an inventory comprising a set of one or more web pages or user interfaces previously accessed by the browser;

perform a first compliance assessment on the set of one or more web pages or user interfaces of the inventory, wherein performing the first compliance assessment comprises testing, tracking, and auditing of the set of one or more web pages or user interfaces of the inventory;

dynamically crawl, using a configuration file, a user-requested web page or user interface requested by the user device to generate a list of uniform resource locators (URLs) corresponding to one or more additional web pages or user interfaces, wherein the URLs are embedded in the user-requested web page or user interface, and wherein the configuration file comprises a specified maximum crawl time and level of recursion, wherein the level of recursion defines a depth of web pages or user interfaces to be crawled;

perform a second compliance assessment, by a scanning process commenced before a rendering of the user-requested web page or user interface on the user device, on the user-requested web page or user interface and the one or more additional web pages or user interfaces to identify one or more compliance issues, wherein the second compliance assessment comprises loading each of the user-requested web page or user interface and the one or more additional web pages or user interfaces in a headless browser, applying the at least one of the JavaScript codes, and executing a series of compliance tests;

map the one or more identified compliance issues to one or more pre-stored remediation scripts on the system, wherein each of the one or more pre-stored remediation scripts corresponds to a compliance issue;

access, from the one or more remediation code databases, one or more remediation code blocks comprising the one or more pre-stored remediation scripts mapped to the one or more identified compliance issues;

programmatically apply, before rendering the user-requested web page or user interface on the user device, the one or more remediation code blocks on the user-requested web page or user interface and the one or more additional web pages or user interfaces, wherein application of the one or more remediation code blocks manipulates the DOM, causing a correction or enhancement of the user-requested web page or user interface and the one or more additional web pages or user interfaces upon rendering; and

transmit, to the user device, the user-requested web page or user interface to which the one or more remediation code blocks have been applied.

2. The system of claim 1, wherein the configuration file further comprises a percentage of pages catalogued and configurable threshold.

3. The system of claim 1, wherein the configuration file further comprises one or more configuration options, the configuration options comprising configuration of at least one of:

a user agent, browser dimensions, starting page, inclusion of subdomains, inclusion of cookies, or usage of a defined site map.

4. The system of claim 1, wherein the series of compliance tests assess one or more testable success criteria as supported by Web Content Accessibility Guidelines (WCAG).

5. The system of claim 1, wherein the series of compliance tests are automatically or manually altered based on evolving web accessibility standards.

6. The system of claim 1, wherein the series of compliance tests determine whether or not one or more specific use-cases are accommodated within the hypertext markup language (HTML) document object model (DOM).

7. The system of claim 1, wherein the system is further caused to:

determine a confidence rating for each of the one or more identified compliance issues;

automatically apply the one or more remediation code blocks to the one or more identified compliance issues without administrator approval if the confidence rating is determined to be greater than a predetermined threshold.

8. The system of claim 1, wherein the system is further caused to:

determine a confidence rating and/or severity rating for each of the programmatic applications of the one or more remediation code blocks.

9. The system of claim 1, wherein the programmatic application of the one or more remediation code blocks is combined with one or more manual remediations to completely remove the one or more identified compliance issues.

10. The system of claim 1, wherein the one or more identified compliance issues comprise at least one of the following:

language attribute not set, INPUT missing label, empty anchor, heading missing text, text alternatives missing from non-text content, unnecessary duplication of link description, skip-to link not present, and links do not warn user before opening a new window.

11. The system of claim 1, further comprising a real-time auto-detection and audio enablement (RADAE) engine configured to detect web page content and element types.

12. The system of claim 1, further comprising a machine learning engine configured to:

learn from the programmatic application of the one or more remediation code blocks, allowing it to create and suggest future programmatic applications; and

automatically modify the one or more remediation code blocks during the creation of the future programmatic applications.

13. The system of claim 12, wherein the machine learning engine learns using one or more of supervised, unsupervised, semi-supervised, or reinforcement learning paradigms.

14. The system of claim 1, wherein the one or more remediation code databases comprise one or more of dynamic link libraries (DLL) or JavaScript object notation (JSON) format databases.

15. A system for rendering one or more web pages enhanced by an integrated JavaScript, the system comprising:

one or more remediation code databases configured to store a plurality of remediation code blocks;

a JavaScript code database comprising one or more JavaScript codes for creating client-side interactivity with web pages, the web pages comprising a hypertext markup language (HTML) document object model (DOM), wherein the DOM comprises one or more nodes, the one or more nodes organized into a DOM tree structure;

one or more computer readable storage devices configured to store a plurality of computer executable instructions; and

one or more hardware computer processors in communication with the one or more computer readable storage devices and configured to execute the plurality of computer executable instructions in order to cause the system to;

receive a request to access a web page from a web browser;

retrieve the web page;

dynamically crawl, using a configuration file, the retrieved web page to generate a list of uniform resource locators (URLs) corresponding to one or more additional web pages, wherein the URLs are embedded in the retrieved web page, and wherein the configuration file comprises a specified maximum crawl time and level of recursion, wherein the level of recursion defines a depth of web pages to be crawled;

perform a compliance assessment, by a scanning process commenced before a rendering of the retrieved web page on the web browser, on the retrieved web page and the one or more additional web pages to identify one or more compliance issues, wherein the compliance assessment comprises loading each of the retrieved web page and the one or more additional web pages in a headless browser, applying the at least one of the JavaScript codes, and executing a series of compliance tests;

map the one or more identified compliance issues to one or more pre-stored remediation scripts on the system, wherein each of the one or more pre-stored remediation scripts corresponds to a compliance issue;

receive a request to retrieve one or more remediation code blocks from the one or more remediation code databases JavaScript, the one or more remediation code blocks configured to solve the one or more identified compliance issues of the retrieved web page and the one or more additional web pages;

access, from the one or more remediation code databases, the one or more remediation code blocks comprising the one or more pre-stored remediation scripts mapped to the one or more identified compliance issues of the retrieved web page and the one or more additional web pages;

programmatically apply, before rendering the retrieved web page or user interface on the user device, the one or more remediation code blocks on the retrieved web page and the one or more additional web pages, wherein application of the one or more remediation code blocks manipulates the DOM, causing a correction or enhancement of the retrieved web page and the one or more additional web pages upon rendering;

transmit, to the web browser the retrieved web page to which the one or more remediation code blocks have been applied; and

cause the web browser to render the web page.

16. A system for form-based creation and deployment of remediations to one or more non-compliant web pages or user interfaces requested by a user web browser, the system comprising:

one or more remediation code databases configured to store one or more JavaScript quick-fix remediation codes and one or more accelerated remediation codes, wherein each of the one or more accelerated remediation codes is generated using a combination of at least one of the one or more JavaScript quick-fix remediation codes and one or more inputs provided by a system administrator, and wherein each of the one or more inputs comprises a required remediation type and a scope of a required remediation, the scope defined by a number of remediation instances and a number of remediation pages;

a dynamic user interface comprising a web-based form comprising visual elements of the one or more non-compliant web pages or user interfaces, wherein the web-based form is configured to allow creation and application of the one or more accelerated remediation codes based at least in part on the one or more inputs provided by the system administrator into the web-based form;

a JavaScript code database comprising one or more JavaScript codes for creating client-side interactivity with one or more web pages or user interfaces, the one or more web pages or user interfaces comprising a hypertext markup language (HTML) document object model (DOM), wherein the DOM comprises one or more nodes, the one or more nodes comprising visual elements tagged with alternative text descriptions organized into a DOM tree structure;

one or more computer readable storage devices configured to store a plurality of computer executable instructions; and

one or more hardware computer processors in communication with the one or more computer readable storage devices and configured to execute the plurality of computer executable instructions in order to cause the system to;

cause display, through the dynamic user interface, the web-based form comprising the visual elements of the one or more non-compliant web pages or user interfaces;

evaluate each of the one or more non-compliant web pages or user interfaces to identify one or more compliance issues, wherein the evaluation comprises loading each of the one or more non-compliant web pages or user interfaces in a headless browser, applying at least one of the JavaScript codes, and executing a series of compliance tests;

receive, into the web-based form, an input corresponding to each identified one or more compliance issues, wherein the input is provided by the system administrator;

submit the web-based form to a server infrastructure, wherein the submission of the web-based form causes programmatic generation of an accelerated remediation code for each of the identified one or more compliance issues by combining the input corresponding to each identified one or more compliance issues with at least one of the one or more JavaScript quick-fix remediation codes, and wherein each accelerated remediation code corresponding to an identical required remediation type is stored in a same remediation code database of the one or more remediation code databases;

receive, from at least one of the one or more JavaScript codes, a request for the accelerated remediation code corresponding to each of the identified one or more compliance issues of each of the one or more non-compliant web pages or user interfaces accessed by the user web browser, wherein the at least one of the one or more JavaScript codes is embedded in the user web browser;

access, from the one or more remediation code databases, the accelerated remediation code corresponding to each of the identified one or more compliance issues of each of the one or more non-compliant web pages or user interfaces requested by the user web browser;

transmit, to the user web browser, the one or more accelerated remediation code blocks corresponding to each of the identified one or more compliance issues of each of the one or more non-compliant web pages or user interfaces requested accessed by the user web browser; and

execute the accelerated remediation code corresponding to each of the identified one or more compliance issues of each of the one or more non-compliant web pages or user interfaces requested by the user web browser, blocks, wherein execution comprises modification of the DOM.

17. The system of claim 16, further comprising a machine learning engine, the machine learning engine configured to:

compare the identified one or more compliance issues with one or more previously identified and resolved compliance issues; and

ascertain one or more previously executed remediation code blocks that may be applicable to the identified one or more compliance issues from the one or more remediation code databases; and

automatically modify the one or more previously executed remediation code blocks to address the identified one or more compliance issues.

18. The system of claim 16, wherein the one or more accelerated remediations can be applied to one or more of the following compliance issues:

inadequate ALT-text, link images with missing or inadequate ALT-text, input images with missing or inadequate ALT-text, iFrames missing titles, disordered headers, heading level issues, inputs missing labels, or links missing labels.

19. A computer implemented method for identifying structural patterns across a plurality of web pages and determining a set of remediations that can be applied collectively to the plurality of web pages, the method comprising:

identifying, by a computer system, within a first document object model (DOM) corresponding to a first web page of the plurality of web pages, a first hierarchical hierarchal structure, wherein the first hierarchical structure comprises one or more hierarchies, and wherein each hierarchy of the first hierarchical structure comprises one or more first elements;

identifying, by the computer system, within a first URL corresponding to the first web page, a first URL structure, the first URL structure comprising a specific path structure;

evaluating, by the computer system, the first web page to identify one or more compliance issues, wherein the evaluation comprises loading the first web page in a headless browser, applying at least one JavaScript code embedded in the headless browser, and executing a series of compliance tests;

determining, by the computer system, a first test result pattern, the first test result pattern comprising a first group of the one or more first elements of the first hierarchical structure that pass a first subset of the series of compliance tests and a second group of the one or more first elements of the first hierarchical structure that fail a second subset of the series of compliance tests;

generating, by the computer system, a template based at least in part on the identified first hierarchical structure, the identified first URL structure, and the identified first test result pattern, wherein the generated template comprises one or more remediations to resolve the one or more identified compliance issues of the first web page;

storing, by the computer system, the generated template comprising the one or more remediations in a remediation database;

identifying, by the computer system, within a second document object model (DOM) corresponding to a second web page of the plurality of web pages, a second hierarchical structure, wherein the second hierarchical structure comprises one or more hierarchies, wherein each hierarchy of the second hierarchical structure comprises one or more second elements, and wherein the one or more second elements comprise commonalities to the one or more first elements of the first hierarchical structure in one or more of placement, class, or appearance, and wherein the second hierarchical structure comprises a substantially identical structural pattern to the first hierarchical structure;

identifying, within a second URL corresponding to the second web page, a second URL structure, the second URL structure comprising the specific path structure of the first URL structure;

evaluating, by the computer system, the second web page to identify one or more compliance issues, wherein the evaluation comprises loading the second web page in a headless browser, applying at least one JavaScript code embedded in the headless browser, and executing the series of compliance tests;

determining, by the computer system, a second test result pattern, the second test result pattern comprising a first group of the one or more second elements of the second web page that pass the first subset of compliance tests and a second group of the one or more second elements of the second web page that fail the second subset of compliance tests, wherein the second test result pattern is substantially identical to the first test result pattern; and

applying, by the computer system, the generated template comprising the one or more remediations to the second web page,

wherein the computer system comprises a computer processor and an electronic non-transitory computer storage medium.