Cloud-based user interface augmentation

• US 9,875,671 B2
• Filed: 12/17/2009
• Issued: 01/23/2018
• Est. Priority Date: 12/17/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google LLC (Alphabet Inc.)
• Inventors: Gharpure, Chaitanya, Chen, Charles L, Shearer, Rachel, Raman, Tiruvilwamalai V.
• Primary Examiner(s): Taylor, Nicholas
• Assistant Examiner(s): Kim, Chong G
• Application Number:
  US12/640,938
• Publication Number:
  US 20110154212A1
• Time in Patent Office: 2,959 Days
• Field of Search: 709200-226, 715760, 715 60, 715771, 715781, 715235, 715716, 707 1, 707506, 707104, 707513
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G09B 21/00 Teaching, or communicating with, the blind, deaf or mute audible presentation of material to be studied G09B5/04; devices or methods for replacing direct visual or auditory perception by another kind of perception A61F9/08, A61F11/04; audible indication of meter readings or of colour G01D7/12; watches for blind persons G04B25/02; speech analysis, speech recognition G10L; sound-recording or reproducing, per se G11B
• CPC Class Codes: G09B 21/009 Teaching or communicating with deaf persons G09B21/04 takes precedence; deaf and dumb persons G09B21/00

Abstract

Method for providing augmented user interface (AUI). AUI is provided in response to a request. AUI configuration information is received, stored in preference file, and sent to remote server. Remote server receives request from computer to login user. Remote server authenticates user, retrieves and sends preference file to computer. Preference file contains information that allows computer to implement and configure AUI by directing output to AUI component. Remote server sends computer web-based API to enable browser to interface with an AUI application. API can include target callable by event handler to respond to HTML document browsing event by invoking AUI application. Remote server can send computer web-based, native, or browser plug-in AUI application. Computer can provide content rich navigation via API, AUI application and content navigation rules that define plural navigation paths through HTML document. Navigation paths providing AUI functionality to particular type of special needs user or user task.

Claims

1. A computer-implemented method comprising:

providing an application programming interface running in a web browser on a computer to receive a call from an event handler to process a detected user navigation event on a web page,

wherein content navigation rules (CNRs) characterize nodes of a document-object-model (DOM) of the web page as a sequence of one or more logical paths through the web page when rendered by the computer, and wherein the CNRs are associated with respective augmented user interface (AUI) functionalities that are tailored to different types of users;

in response to receiving the call, requesting a user preference file from a remote server;

receiving, from the remote server, the user preference file and an AUI component API identified in the user preference file; and

in response to receiving the user preference file;

processing the detected user navigation event by;

identifying a CNR corresponding to the detected navigation event; and

calling, in the web browser, an AUI component application via the received AUI component API to invoke an AUI on the computer, the AUI having the respective AUI functionality associated with the identified CNR.

3. The method of claim 1, wherein the received AUI component API provides the respective AUI functionality on the computer in conjunction with the AUI component application according to the user preference file.

4. The method of claim 3, wherein the AUI component application includes at least one of a cloud-based AUI component application, a native AUI component application or a browser plug-in AUI component application.

5. The method of claim 1, wherein the received AUI component API includes a browser plug-in.

6. The method of claim 1, wherein the received AUI component API includes a target callable by the event handler to call the AUI component application.

7. The method of claim 1, wherein the AUI component application includes one of a native screen reader, a native screen magnifier, a native speech recognizer, a browser plug-in version of a screen reader, a browser plug-in version of a screen magnifier, a browser plug-in version of a speech recognizer, a web based screen reader, a web based screen magnifier and a web based speech recognizer.

2. A non-transitory computer-readable storage medium comprising instructions operable to cause a programmable processor to:

provide an application programming interface running in a web browser on a computer to receive a call from an event handler to process a detected user navigation event on a web page,

wherein content navigation rules (CNRs) characterize nodes of a document-object-model (DOM) of the web page as a sequence of one or more logical paths through the web page when rendered by the computer, and wherein the CNRs are associated with respective augmented user interface (AUI) functionalities that are tailored to different types of users; and

in response to receiving the call, request a user preference file from a remote server;

receive, from the remote server, the user preference file and an AUI component API identified in the user preference file; and

in response to receiving the user preference file;

process the detected user navigation event by;

identifying a CNR corresponding to the detected navigation event; and

calling, in the web browser, an AUI component application via the received AUI component API in the web browser to invoke an AUI on the computer, the AUI having the respective AUI functionality associated with the identified CNR.

8. The non-transitory computer-readable storage medium of claim 2, wherein the received AUI component API provides the respective AUI functionality on the computer in conjunction with the AUI component application according to the user preference file.

9. The non-transitory computer-readable storage medium of claim 8, wherein the AUI component application includes at least one of a cloud-based AUI component application, a native AUI component application or a browser plug-in AUI component application.

10. The non-transitory computer-readable storage medium of claim 2, wherein the received AUI component API includes a browser plug-in.

11. The non-transitory computer-readable storage medium of claim 2, wherein the received AUI component API includes a target callable by the event handler to call the AUI component application.

12. The non-transitory computer-readable storage medium of claim 2, wherein the AUI component application includes one of a native screen reader, a native screen magnifier, a native speech recognizer, a browser plug-in version of a screen reader, a browser plug-in version of a screen magnifier, a browser plug-in version of a speech recognizer, a web based screen reader, a web based screen magnifier and a web based speech recognizer.