Techniques for providing accessibility options in remote terminal sessions

• US 7,676,549 B2
• Filed: 05/27/2005
• Issued: 03/09/2010
• Est. Priority Date: 05/27/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Abdo, Nadim Y., McKeon, Brendan, Patwardhan, Makarand V
• Primary Examiner(s): Najjar; Saleh
• Assistant Examiner(s): SHAW, ROBERT A
• Application Number:
  US11/139,427
• Publication Number:
  US 20060271637A1
• Time in Patent Office: 1,747 Days
• Field of Search: 709/217, 707/3, 715/740
• US Class Current: 709/217
• CPC Class Codes: H04L 67/08 specially adapted for terminal emulation, e.g. Telnet

Abstract

Techniques relating to enabling accessibility functionality in remote terminal session scenarios are described. In one instance, a process detects a request from an accessibility functionality regarding a display element in a remote terminal session. The process sends a query relating to the request to a component located on a computing device which generated the display element. The process also receives a response to the query.

Claims

1. A computer-implemented method with computer-executable instructions that are executable by a client computing device, the method comprising:

receiving on the client computing device a representation of a user interface (UI) generated at a server computing device, wherein the UI is implemented as a server desktop on the server computing device and the representation of the UI is implemented as a remote desktop on the client computing device;

displaying the remote desktop as a bitmap on the client computing device, wherein the remote desktop comprises a representation of one or more display elements of the UI that are displayed on the client computing device without a functionality and/or an underlying information which are associated with the one or more display elements of the UI when said one or more display elements are displayed on the server computing device as a part of the server desktop;

presenting to an accessibility aid (AA) client component of an accessibility functionality on the client computing device, by an AA-UI-interface component of the accessibility functionality on the client computing device, a tree of nodes generated at the server computing device, representing which reflects the representation of one or more display elements of the UI,

wherein each node of the tree of nodes corresponds to a display element of the UI, and wherein a structure and/or properties of the each tree node represents a functionality and/or an underlying information of the corresponding display element of the UI

wherein the AA-U I-interface component provides access to the structure and/or properties, for the each node

requesting, by the AA client component, an information regarding the representation of a display element on the remote desktop, including information regarding the server desktop that is utilized to achieve the accessibility functionality on the client computing device during a remote terminal session, wherein the AA client component requests the information from the server computing device via the AA-UI-interface component,

wherein the information request is built as a byte message comprising at least one of;

a reference to the display element, a list of properties to retrieve, or a filter that identifies the elements of interest;

using a processor on the client computing device for detecting a request from the accessibility functionality regarding the display element in the remote terminal session, wherein the request is propagated from the AA client component via the AA-UI-interface component on the client computing device and corresponds to the information requested by the AA client component;

sending the information request to an AA-UI interface-server component located on the server computing device, wherein the information request comprises the byte message; and

,

receiving a response to the information request from the server computing device,

wherein the response comprises accessibility related information from the UI, regarding the representation of the display element on the remote desktop,

wherein the response is presented as nodes of an object tree which reflects the representation of the one or more display elements, and

wherein the AA client component provides an accessibility functionality output at the client computing device based on the response.

2. A method as recited in claim 1, wherein the display element comprises a portion of the remote desktop generated at the server computing device for the client computing device, wherein the remote desktop is displayed as a bitmap representation on the client computing device.

3. A method as recited in claim 1, wherein said sending comprises exposing an interface configured to allow the accessibility functionality to establish a virtual channel with the component.

4. A method as recited in claim 1, further comprising the accessibility functionality establishing communications with an AA-UI-interface component located on the server computing device to obtain information corresponding to the information requested by the AA client component located on the client computing device.

5. A method as recited in claim 4, wherein the establishing communications comprises providing interfaces for the accessibility functionality to communicate with the AA-UI-interface component located on the server computing device.

6. A method as recited in claim 4, wherein the enabling comprises providing interfaces for the accessibility functionality to launch one or more components configured to allow the accessibility functionality to communicate with the AA-UI-interface component located on the server computing device.

7. A method as recited in claim 1, wherein the display element comprises a single user-interface (UI) object.

8. A method as recited in claim 1, wherein receiving the response further comprises processing the response to compensate for at least one factor relating to the remote terminal session.

9. A method as recited in claim 8, wherein the at least one factor comprises adjusting location data obtained from a source of the display element to reflect a location of the display element relative to a client display.

10. A method as recited in claim 1, wherein the sending comprises sending the request over a virtual channel enabled by the accessibility functionality in the remote terminal session.

11. A computer-readable storage media comprising computer-executable instructions that, when executed by a processor, cause the processor to perform acts, comprising:

generating a user-interface (UI) on a server computing device, wherein the UI comprises one or more display elements, and wherein a display element comprises a portion of the UI which is associated with object data and/or information;

generating at the server computing device a representation of the UI, wherein the representation of the UI comprises a tree of nodes reflecting the representation of the one or more display elements of the UI;

sending the representation of the UI to a remote terminal session client, wherein the representation of the UI has a limitation of being associated with a reduced level of object data and/or information, which are associated with the one or more display elements of the UI when said one or more display elements are displayed on the server computing device

wherein the remote terminal session client displays the representation of the UI as a bitmap without a functionality and/or an underlying information associated with the one or more display elements of the UI that the one or more display elements have when displayed on the server computing device;

presenting at the remote terminal session client the tree of nodes, wherein each node of the tree of nodes corresponds to a display element of the UI, and wherein a structure and/or properties of the each tree node represent a functionality and/or an underlying information of the corresponding display element of the UI,

receiving a query at the server computing device relating to the UI from the remote terminal session client, wherein the query comprises at least one of;

a reference to a display element of the UI, a list of properties corresponding to the display element to retrieve, or a filter that identifies the elements of interest corresponding to the display element; and

wherein the query comprises a byte message;

de-serializing the byte message at the server computing device

accessing object data associated with the display element at the server computing device;

acquiring an information responsive to the query from the object data associated with the display element,

wherein the information responsive to the query includes information regarding the UI which is utilized by the remote terminal session client to achieve accessibility functionality locally at the remote terminal session client during a remote terminal session;

serializing the information acquired into another byte message; and

sending the another byte message as a response from the server computing device to the remote terminal session client,

wherein the response comprises accessibility related information from the UI, regarding the representation of the display element on the remote desktop,

wherein the response is presented as nodes of an object tree which reflects the representation of the one or more display elements, and

wherein the remote terminal session client provides an accessibility functionality output at the client computing device based on the response.

12. The computer-readable storage media of claim 11, wherein the accessing is accomplished by an accessibility aid-user-interface (AA-UI) interface component which is started responsive to said receiving.

13. The computer-readable storage media of claim 11, wherein the receiving is accomplished by a component which is started responsive to receiving a start-up request from the remote terminal session client.

14. The computer-readable storage media of claim 11, wherein said UI comprises a sub-tree of UI elements.

15. The computer-readable storage media of claim 11 further comprising sending the UI information to the client over a communication means enabled by a terminal services session located on the terminal services client.

16. A system, comprising:

a memory;

a processor coupled to the memory;

means for receiving a representation at a client computing device of a user interface (UI) generated at a server computing device, wherein the UI is implemented as a server desktop on the server computing device;

means for displaying at the client computing device a remote desktop as a bitmap representation, wherein the remote desktop comprises a representation of one or more display elements of the UI that are displayed without a functionality and/or an underlying information associated with the one or more display elements of the UI when the one or more display elements are displayed on the server computing device as a part of the server desktop;

means for presenting to an accessibility aid (AA) client component of an accessibility functionality, by an AA-UI-interface component of the accessibility functionality, a tree of nodes generated at the server computing device reflecting the representation of one or more display elements of the UI, wherein each node of the tree of nodes represents corresponds to a display element of the UI, wherein a structure and/or properties of the each tree node represents a functionality and/or an underlying information of the corresponding display element of the UI, and wherein the AA-UI-interface component provides access to the structure and/or properties, for the each node;

means for requesting, by the AA client component, an information regarding the representation of a display element on the remote desktop, wherein the AA client component requests the information from the server computing device via the AA-UI-interface component,

wherein the information request is built as a byte message comprising at least one of a reference to the display element, a list of properties to retrieve, or a filter that identifies the elements of interest,

wherein the information request regarding a display element corresponds to accessibility related information from the UI that is provided locally at a client computing device for the AA client component to generate an accessibility functionality output;

means for determining if the display element is remotely generated, wherein the means for determining are executable by the processor;

means for obtaining the information regarding the representation of the display element displayed in the remote terminal session from the server computing device;

means for sending a response to the information request from the server computing device, to the client computing device

wherein the response comprises accessibility related information from the UI, regarding the representation of the display element on the remote desktop,

wherein the response is presented as nodes of an object tree which reflects the representation of the one or more display elements, and

means for providing an accessibility functionality output at the client computing device by the AA client component based on the response wherein the means for providing the accessibility functionality output are executable by the processor.

17. A system as recited in claim 16 further comprising means for communicating with a remote source of the display element through which accessibility related information can be obtained, wherein the means for communicating are executed by the processor.

18. A system as recited in claim 17, wherein the means for communicating provide one or more interfaces through which the means for requesting can access a communication means for communicating with the remote source.

19. A system as recited in claim 18, wherein an interface provided by the means for communicating allows the means for requesting to launch a component within a process boundary of the means for communicating.