Modular Systems and Methods For Selectively Enabling Cloud-Based Assistive Technologies

• US 20170269816A1
• Filed: 03/18/2016
• Published: 09/21/2017
• Est. Priority Date: 03/18/2016
• Status: Active Grant

• Current Assignee: AudioEye, Inc.
• Original Assignee: AudioEye, Inc.
• Inventors: Bradley, Sean D., Baker, Mark D., Jones, Jeffrey O., Hefner, Kenny P., Finkelstein, Adam, Gilormo, Douglas J., Bodnar, Taylor R., Pinckney, David C., Blevins, Charlie E., Jones, Trevor C., Laymon, Helena
• Granted Patent: US 10,444,934 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/117 Tagging; Marking up details of markup languages G06F40/143; Designating a block; Setting of attributes style sheets, e.g. eXtensible Stylesheet Language Transformation [XSLT], G06F40/154
• CPC Class Codes: G06F 40/14 Tree-structured documents parsing G06F40/205; validation G06F40/226
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G10L 13/02 Methods for producing synthetic speech; Speech synthesisers
• CPC Class Codes: G10L 13/027 Concept to speech synthesisers; Generation of natural phrases from machine-based concepts generation of parameters for speech synthesis out of text G10L13/08
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/26 Speech to text systems G10L15/08 takes precedence
• CPC Class Codes: G10L 2015/221 Announcement of recognition results
• CPC Class Codes: H04L 67/02 based on web technology, e.g. hypertext transfer protocol [HTTP]
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

Methods and systems for manual and programmatic remediation of websites. JavaScript code is accessed by a user device and optionally calls TTS, ASR, and RADAE modules from a remote server to thereby facilitate website navigation by people with diverse abilities.

Claims

1. A method of persisting a user preference associated with an assistive technology across multiple web domains, comprising:

installing remediation code on a first server hosting a first website and on a second server hosting a second website;

establishing, using a browser operating on a user device, a first session with the first website;

enabling an assistive technology during the first session;

defining a user preference associated with the assistive technology during the first session;

placing, by the remediation code, a cookie into the browser indicating the first user preference;

establishing, using the browser operating on the user device, a second session with the second website; and

using the cookie to persist the user preference to the second session.

2. The method of claim 1, wherein placing the cookie into the browser comprises passing the cookie through an iframe.

3. The method of claim 2, wherein the iframe comprises an invisible iframe.

4. The method of claim 1, wherein persisting the user preference to the second session is controlled by the remediation code without any additional user action.

5. The method of claim 1, wherein the remediation code comprises JavaScript.

6. The method of claim 1, wherein the first server is remote from the second server.

7. The method of claim 1, wherein the assistive technology comprises one of:

an audio player function;

a reader tool suite;

a voice command feature; and

a page elements menu.

8. The method of claim 7, wherein the reader tool suite comprises font, magnification, and contrast controls.

9. The method of claim 1, wherein the user preference comprises maintaining the assistive technology in the enabled state.

10. The method of claim 1, further comprising embedding indicia of the user preference in the browser cookie.

11. Computer code stored in a non-transient medium and configured to perform, when executed by a computer processor, the steps of:

superimposing a timed toggle of a branding icon over a universal accessibility icon on a web page; and

revealing, in response to a user command, an assistive technology tool bar including controls for independently enabling at least an audio player and a reader tool suite.

12. The computer code of claim 11, wherein the user command comprises one of hovering over and clicking on the branding icon.

13. The computer code of claim 11, wherein independently enabling comprises selecting one of:

a first option to enable the audio player without enabling the reader tool suite;

a second option to enable the reader tool suite without enabling the audio player; and

a third option to simultaneously enable both the audio player and the reader tool suite.

14. The computer code of claim 11, wherein superimposing the timed toggle comprises using one of a cascading style sheet (CSS) and JavaScript to periodically animate the branding icon.

15. The computer code of claim 11, wherein the assistive technology tool bar further includes a control for independently enabling a help desk function.

16. The computer code of claim 11, wherein the computer code comprises JavaScript.

17. A method of ad hoc page navigation through a website using a browser, comprising:

displaying a page element menu comprising a plurality of objects;

in response to the selection of one of the objects by a user, dynamically inserting an anchor into the object;

displaying the page corresponding to the selected object; and

in response to the user focusing away from the selected object, dynamically extracting the anchor from the object.

18. The method of claim 17, wherein the anchor comprises a tabindex element attribute.

19. The method of claim 17, wherein the plurality of objects comprise at least two of:

page heading;

hyperlink;

image;

footer;

and top of page.

20. The method of claim 17, wherein displaying the page comprises setting the browser focus to the selected object.

21. A method of programmatically assigning descriptive alt text to an element on web page, the web page having an associated document object model (DOM), comprising:

inspecting the DOM for an improperly tagged element having an associated hyperlink;

determining the file path associated with the hyperlink; and

assigning an alt text description to the improperly tagged element based on the file path.

22. The method of claim 21, wherein the improperly tagged element comprises a user prompt to “

Like Us on Facebook,” and

the file path comprises a link to a Facebook Like Us landing page.

23. The method of claim 21, wherein the improperly tagged element comprises a user prompt to “

Follow Us on Twitter,” and

the file path comprises a link to a Twitter Follow Us landing page.

24. A method of programmatically assigning descriptive alt text to an image on web page, the web page having an associated document object model (DOM), comprising:

inspecting the DOM for an image lacking an adequate alt text;

sending the image to a remote image recognition server to determine the nature of the image;

assigning an alt text description to the image based on the determined image nature; and

speaking the alt text description to a user using an assistive technology.

25. The method of claim 24, wherein the assistive technology comprises one of a native screen reader and an audio player embedded in JavaScript remediation code.

26. The method of claim 24, wherein the image recognition server comprises an image database and a recognition module for comparing the image to the contents of the image database.

27. The method of claim 24, wherein the nature of the image comprises one of:

a landmark;

a location;

the name of an object; and

the name of a person.

28. A method of programmatically generating an audio description of a video file appearing on web page enhanced with remediation code which includes an embedded audio player, the method comprising:

sending the video file to a remote image recognition server;

sampling respective frames of the video file and identifying corresponding respective images;

using artificial intelligence (AI) techniques to determine the subject matter of the video file based on differences between successive frames;

returning a text-to-speech (TTS) convertible text file which describes the subject matter of the video; and

speaking the text file to a user using the embedded audio player.

29. The method of claim 28, wherein the text file comprises metadata for use in synchronizing the text file with the video file.

30. The method of claim 29, wherein the metadata comprises respective timing data associated with each of a plurality of the respective frames.

31. A method of enhancing a web page hosted by a first server to render the page navigable by users with diverse abilities through voice commands, comprising:

in response to the keyboard selection of a text element by a user, the text element being of the type which prompts the user to enter an answer into a field on the web page, generating a text-to-speech (TTS) file corresponding to the text element;

processing the TTS file to thereby verbalizing the text element to the user;

capturing a recording of the user speaking the answer;

sending the recording to a second server;

generating a speech-to-text (STT) file of the recording at the second server; and

processing the STT file at the first server to thereby enter the answer into the field.

32. The method of claim 31, further comprising establishing a WebSocket connection between the first and second servers.

33. The method of claim 31, further comprising suspending verbalizing the text element to the user while the user is speaking the answer.

34. The method of claim 33, further comprising resuming verbalizing the text element to the user when the user is finished speaking the answer.

35. The method of claim 31, wherein the answer comprises a voice command.

36. The method of claim 31, wherein the web page comprises a form and the field comprises one of a plurality of information fields required to fill out the form.

37. The method of claim 31, further comprising verbalizing the answer to the user as the answer is entered into the field.

38. The method of claim 37, further comprising using an audio sprite to facilitate verbalizing the answer to the user as the answer is entered into the field.

39. The method of claim 38, wherein the audio sprite comprises a CSS to define which characters are verbalized at particular timed intervals.

40. A method of controlling a primary device, hosted by a remote server, using a hand held device to allow individuals with disabilities to interact with the primary device using the hand held device, the method comprising:

pairing the hand held device with the primary device;

displaying a menu of elements on the primary device;

entering a command into the hand held device;

sending the command to the remote server;

returning an instruction based on the command from the remote server to the primary device; and

executing the instruction by the primary device.

41. The method of claim 40 wherein the command comprises a voice command, the method further comprising:

generating an STT file of the voice command at the remote server; and

processing the STT file by the primary device to execute the voice command.