CLOUD-BASED USER INTERFACE AUGMENTATION

• US 20110154212A1
• Filed: 12/17/2009
• Published: 06/23/2011
• Est. Priority Date: 12/17/2009
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Raman, Tiruvilwamalai V., Gharpure, Chaitanya, Chen, Charles L., Shearer, Rachel
• Granted Patent: US 9,875,671 B2
• Time in Patent Office: Days
• US Class Current: 715/738
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G09B 21/00 Teaching, or communicating with, the blind, deaf or mute audible presentation of material to be studied G09B5/04; devices or methods for replacing direct visual or auditory perception by another kind of perception A61F9/08, A61F11/04; audible indication of meter readings or of colour G01D7/12; watches for blind persons G04B25/02; speech analysis, speech recognition G10L; sound-recording or reproducing, per se G11B
• CPC Class Codes: G09B 21/009 Teaching or communicating with deaf persons G09B21/04 takes precedence; deaf and dumb persons G09B21/00

Abstract

Method for providing augmented user interface (AUI). AUI is provided in response to a request. AUI configuration information is received, stored in preference file, and sent to remote server. Remote server receives request from computer to login user. Remote server authenticates user, retrieves and sends preference file to computer. Preference file contains information that allows computer to implement and configure AUI by directing output to AUI component. Remote server sends computer web-based API to enable browser to interface with an AUI application. API can include target callable by event handler to respond to HTML document browsing event by invoking AUI application. Remote server can send computer web-based, native, or browser plug-in AUI application. Computer can provide content rich navigation via API, AUI application and content navigation rules that define plural navigation paths through HTML document. Navigation paths providing AUI functionality to particular type of special needs user or user task.

Claims

1. A computer-implemented method for providing a user with an augmented user interface comprising:

receiving at a remote server a request from a requesting computer to login the user;

authenticating the user at the remote server, and upon authenticating the user, retrieving from a database stored on the remote server a user preference file, wherein the user preference file records the user'"'"'s preference for configuring the requesting computer; and

sending to the requesting computer the user'"'"'s preference file, wherein the preference file contains information to allow the requesting computer to implement and configure an augmented user interface (AUI) by directing output on the requesting computer to an augmented user interface component that processes the output to provide the AUI to the user.

2. The method of claim 1, further comprising sending the requesting computer a web-based application programming interface that enables a browser running on the requesting computer to interface with an augmented user interface application.

3. The method of claim 2, wherein the augmented user interface application is an application selected from the group consisting of a web-based application that runs on a network based server, a native application that runs on the requesting computer, and a browser plug-in that runs in a browser on the requesting computer.

4. The method of claim 2, wherein the application programming interface comprises a target callable by an event handler that is configured to respond to an event generated while browsing an HTML document, and to invoke an augmented user interface application in response to the event to provide a user with augmented user interface functionality.

5. The method of claim 1, further comprising sending the requesting computer an augmented user interface application.

6. The method of claim 5, wherein the augmented user interface application is an application selected from the group consisting of a web-based application that runs on a network based server, a native application that runs on the requesting computer, and a browser plug-in that runs in a browser on the requesting computer.

7. A computer-implemented method for providing a user with an augmented user interface over a network comprising:

receiving at a computer a user request for an augmented user interface;

providing the user with the augmented user interface;

receiving configuration information from the user to configure the augmented user interface;

storing the configuration information for the augmented user interface in a preference file on the computer; and

sending the preference file from the computer to a remote server for storage.

8. The method of claim 7, wherein receiving a user request for an augmented user interface comprises receiving the request from an input element selected from the group consisting of a keyboard, a pointing device, and an audio input device.

9. The method of claim 7, wherein providing the user with the augmented user interface comprises providing the user with a web-based application programming interface that enables a browser running on the user'"'"'s computer to interface with an augmented user interface application.

10. The method of claim 9, wherein the augmented user interface application is an application selected from the group consisting of a web-based application that runs on a network based server, a native application that runs on the requesting computer, and a browser plug-in that runs in a browser on the requesting computer.

11. The method of claim 9, wherein the application programming interface comprises a target callable by an event handler configured to respond to an event generated while browsing an HTML document, and to invoke an augmented user interface application in response to the event to provide a user with augmented user interface functionality.

12. A computer-implemented method for providing content rich navigation through a web page via an augmented user interface, comprising:

receiving at an application programming interface running in a web browser a call from an event handler to process a detected user navigation event; and

processing the call from the event handler to invoke an augmented user interface application to respond to the detected user navigation event.

13. The method of claim 12, wherein the detected user navigation event is generated based on user navigation through the web page based on the web page'"'"'s content.

14. The method of claim 12, wherein the detected user navigation event is generated based on user navigation through the web page based on the web page'"'"'s HTML5 document outline.

15. The method of claim 12, wherein the detected user navigation event is generated based on user navigation through a table in the web page.

16. The method of claim 12, wherein the web page includes a content navigation rule, and the detected user navigation event is generated based on user navigation through the web page based on the content navigation rule.

17. The method of claim 16, wherein the content navigation rule provides a navigation path through the web page that is tailored to provide augmented user interface functionality to a particular type of special needs user.

18. The method of claim 12, wherein the web page includes a plurality of content navigation rules that define a plurality of navigation paths through the web page, wherein each navigation path is tailored to provide augmented user interface functionality to a particular type of special needs user.

19. The method of claim 12, wherein the web page includes a plurality of content navigation rules that define a plurality of navigation paths through the web page, wherein each navigation path is tailored to provide augmented user interface functionality to a particular type of user task.

20. A computer program product, tangibly embodied in a computer-readable medium, comprising instructions operable to cause a programmable processor to:

receive at a remote server a request from a requesting computer to login the user;

authenticate the user at the remote server, and upon authenticating the user, retrieve from a database stored on the remote server a user preference file, wherein the user preference file records the user'"'"'s preference for configuring the requesting computer; and

send to the requesting computer the user'"'"'s preference file, wherein the preference file contains information to allow the requesting computer to implement and configure an augmented user interface (AUI) by directing output on the requesting computer to an augmented user interface component that processes the output to provide the AUI to the user.

21. The computer program product of claim 20, further comprising instructions operable to cause the programmable processor to send the requesting computer a web-based application programming interface that enables a browser running on the requesting computer to interface with an augmented user interface application.

22. The computer program product of claim 21, wherein the augmented user interface application is an application selected from the group consisting of a web-based application that runs on a network based server, a native application that runs on the requesting computer, and a browser plug-in that runs in a browser on the requesting computer.

23. The computer program product of claim 21, wherein the application programming interface comprises a target callable by an event handler that is configured to respond to an event generated while browsing an HTML document, and to invoke an augmented user interface application in response to the event to provide a user with augmented user interface functionality.

24. The computer program product of claim 20, further comprising instructions operable to cause the programmable processor to send the requesting computer an augmented user interface application.

25. The computer program product of claim 24, wherein the augmented user interface application is an application selected from the group consisting of a web-based application that runs on a network based server, a native application that runs on the requesting computer, and a browser plug-in that runs in a browser on the requesting computer.

26. A computer program product, tangibly embodied in a computer-readable medium, for providing a user with an augmented user interface over a network, the computer program product comprising instructions operable to cause a programmable processor to:

receive at a computer a user request for an augmented user interface;

provide the user with the augmented user interface;

receive configuration information from the user to configure the augmented user interface;

store the configuration information for the augmented user interface in a preference file on the computer; and

send the preference file from the computer to a remote server for storage.

27. The computer program product of claim 26, wherein the instruction to receive a user request for an augmented user interface comprises an instruction to receive the request from an input element selected from the group consisting of a keyboard, a pointing device, and an audio input device.

28. The computer program product of claim 26, wherein the instruction to provide the user with the augmented user interface comprises an instruction to provide the user with a web-based application programming interface that enables a browser running on the user'"'"'s computer to interface with an augmented user interface application.

29. The computer program product of claim 28, wherein the augmented user interface application is an application selected from the group consisting of a web-based application that runs on a network based server, a native application that runs on the requesting computer, and a browser plug-in that runs in a browser on the requesting computer.

30. The computer program product of claim 28, wherein the application programming interface comprises a target callable by an event handler configured to respond to an event generated while browsing an HTML document, and to invoke an augmented user interface application in response to the event to provide a user with augmented user interface functionality.

31. A computer program product, tangibly embodied in a computer-readable medium, for providing content rich navigation through a web page via an augmented user interface, the computer program product comprising instructions operable to cause a programmable processor to:

receive at an application programming interface running in a web browser a call from an event handler to process a detected user navigation event; and

to

process the call from the event handler to invoke an augmented user interface application to respond to the detected user navigation event.

32. The computer program product of claim 31, wherein the detected user navigation event is generated based on user navigation through the web page based on the web page'"'"'s content.

33. The computer program product of claim 31, wherein the detected user navigation event is generated based on user navigation through the web page based on the web page'"'"'s HTML5 document outline.

34. The computer program product of claim 31, wherein the detected user navigation event is generated based on user navigation through a table in the web page.

35. The computer program product of claim 31, wherein the web page includes a content navigation rule, and the detected user navigation event is generated based on user navigation through the web page based on the content navigation rule.

36. The computer program product of claim 35, wherein the content navigation rule provides a navigation path through the web page that is tailored to provide augmented user interface functionality to a particular type of special needs user.

37. The computer program product of claim 31, wherein the web page includes a plurality of content navigation rules that define a plurality of navigation paths through the web page, wherein each navigation path is tailored to provide augmented user interface functionality to a particular type of special needs user.

38. The computer program product of claim 31, wherein the web page includes a plurality of content navigation rules that define a plurality of navigation paths through the web page, wherein each navigation path is tailored to provide augmented user interface functionality to a particular type of user task.