Context aware auxiliary display platform and applications

• US 8,188,936 B2
• Filed: 03/31/2009
• Issued: 05/29/2012
• Est. Priority Date: 05/03/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Fuller, Andrew J., Rhoten, Matthew P., Perez, Juan J., Schoppa, Christopher A., Steeb, Curt A., Dongen, Niels van
• Primary Examiner(s): Lefkowitz, Sumati
• Assistant Examiner(s): Carter, III, Robert E
• Application Number:
  US12/415,979
• Publication Number:
  US 20100066642A1
• Time in Patent Office: 1,155 Days
• Field of Search: 345 11- 31, 701/213, 707/9
• US Class Current: 345/2.1
• CPC Class Codes: G06F 2209/544 Remote
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications
• CPC Class Codes: H04M 1/72454 according to context-related or environment-related conditions
• CPC Class Codes: H04M 2250/12 including a sensor for measuring a physical value, e.g. temperature or motion

Abstract

Described is a mechanism by which application programs (or plug-ins) and an auxiliary service adjust the output and/or data available for output on an auxiliary device based on changes detected in the current context of the auxiliary display. Context sensors detect and report changes in context, whereby some action is taken with respect to what information is displayed or can be displayed on the auxiliary display device. Examples of context data that can change include the physical location of the display, detected movement or motion data, presence of a particular user or others, power state, network connectivity status, privacy status and so forth. The auxiliary device can have context as to its position relative to the main display. Information sensitivity levels may be used to specify whether an application program'"'"'s data can be displayed, based on the perceived sensitivity of the information and the user'"'"'s current context.

Claims

1. One or more storage devices having stored thereon computer executable instructions that, when executed by one or more processors of a computer system, implement a method for controlling the display of application data on a main display of the computer system and on an auxiliary remote display device of a separate device connected to the computer system via a remote connection, the method comprising:

an act of the computer system identifying that a remote display is available to the computer system, the remote display belonging to a separate device connected to the computer system via a remote connection;

an act of the computer system identifying one or more display capabilities of the remote display, including at least a display resolution of the remote display;

an act of the computer system determining that a context change has occurred in the computer system;

in response to identifying the context change, an act of the computer system enumerating a list of enabled plug-ins selected from a plurality of available plug-ins, each of the plurality of available plug-ins specifying one or more context properties that specify one or more contexts corresponding to a display of application data from a corresponding application program on the remote display;

based on the list of enabled plug-ins and the identified context change, an act of the computer system permitting a particular application program corresponding to one of the enabled plug-ins to send supplemental application data for the particular application program over the remote connection to the separate device for display at the remote display, the supplemental application data formatted for the one or more display capabilities of the remote display, while the particular application program concurrently displays primary application data for the particular application program at a main display of the computer system;

an act of the computer system receiving user input to the particular application program over the remote connection from the separate device while displaying the primary application data for the particular application program, the user input causing a change in state of the particular application program that changes the primary application data displayed at the main display, the user input generated by a user interacting with the displayed supplemental application data at the remote display using one or more actuators of the separate device; and

based on the received user input and the change in state of the particular application program, an act of the computer system permitting the particular application program to send any changed supplemental application data over the remote connection to the separate device for display at the remote display, while the particular application program concurrently displays the changed primary application data at the main display.

2. The storage device of claim 1, wherein the separate device also independently enables and disables one or more plug-ins based on context.

3. The storage device of claim 1, wherein the context change comprises an estimation of future location of the computer system based on a current location and velocity of one or more of the computer system or the separate device.

4. The storage device of claim 1, wherein at least one plug-in applies different behavior to different application data based on a relative sensitivity of the application data.

5. The storage device of claim 1, wherein the remote display has limited display capabilities compared to the main display.

6. The storage device of claim 1, wherein at least one of the plurality of available plug-ins specifies one or more context properties that makes at least some application data unavailable for display on the remote display based on the presence of other people near one or more of the computing device or the separate device.

7. A method implemented within a computer system that includes one or more processors and memory storing instructions which, when executed by the one or more processors, implement the method for controlling the display of application program data on a main display of the computer system and on a remote display of a separate device connected to the computer system via a remote connection, the method comprising:

an act of the computer system identifying that a remote display is available to the computer system, the remote display belonging to a separate device connected to the computer system via a remote connection;

an act of the computer system identifying one or more display capabilities of the remote display, including at least a display resolution of the remote display;

an act of the computer system identifying that a context change has occurred in one or more of the computer system or the separate device;

in response to identifying the context change, an act of the computer system enumerating a list of enabled plug-ins selected from a plurality of available plug-ins, each of the plurality of available plug-ins specifying one or more contexts associated with a display of application data from a corresponding application program on the remote display;

based on the list of enabled plug-ins and the identified context change, an act of the computer system permitting a particular application program corresponding to one of the enabled plug-ins to send supplemental application data for the particular application program over the remote connection to the separate device for display at the remote display, the supplemental application data formatted for the one or more display capabilities of the remote display, while the particular application program concurrently displays primary application data for the particular application program at a main display of the computer system, the supplemental application data being different from the primary application data;

an act of the computer system receiving user input to the particular application program over the remote connection from the separate device while displaying the primary application data for the particular application program, the user input causing the particular application program to change the primary application data displayed at the main display, the user input generated by a user interacting with the displayed supplemental application data at the remote display using one or more actuators of the separate device; and

based on the received user input, an act of the computer system permitting the particular application program to send different supplemental application data over the remote connection to the separate device for display at the remote display, while the particular application program concurrently displays the changed primary application data at the main display.

8. The method of claim 7, further comprising:

an act of the computer system enabling the one of the enabled plug-ins for displaying application data on the remote display of the separate device based on the list of enabled plug-ins and the identified context change.

9. The method of claim 7, wherein the context change comprises the computer system and the separate device establishing the remote connection.

10. The method of claim 7, wherein the context change comprises the separate device coming into a wireless communications range of the computer system.

11. The method of claim 7, wherein the remote display has limited display capabilities compared to the main display.

12. The method of claim 7, wherein the context change comprises the presence of other people near one or more of the computer system or the separate device, and wherein the computer system disables at least one of the plurality of available plug-ins to prevent a corresponding application program from displaying application data at the remote display, to prevent the display of sensitive information to the other people.

13. The method of claim 7, wherein the remote connection comprises a wireless remote connection.

14. The method of claim 7, wherein the remote connection comprises a wired remote connection.

15. The method of claim 7, wherein the one of the enabled plug-ins is part of the main application code of the particular application program.

16. The method of claim 7, wherein the particular application also sends additional supplemental application data to a different remote display belonging to a different separate device concurrent with sending the supplemental application to the separate device and concurrently displaying the primary application data at the main display.

17. The method of claim 7, wherein the separate device is a standalone computing device.

18. The method of claim 7, wherein:

the particular application program comprises an e-mail application, the primary application data comprises an e-mail application user interface, and the supplemental application data comprises a list of e-mail messages;

the user input generated by the user interacting with the displayed supplemental application data at the remote display using one or more actuators of the separate device comprises user input selecting an e-mail message from the list of e-mail messages displayed at the remote display; and

the different supplemental application data comprises at least a portion of the contents of the selected e-mail message, rather than the list of e-mail messages.

19. A computer system, comprising:

one or more processors;

a primary display device;

a remote communications device; and

one or more computer storage devices storing computer-executable instructions that, when executed, cause the computer system to perform a method for controlling the display of application program data on the primary display device and on a remote display of a separate device connected to the computer system via the remote communications device, comprising;

identifying that a remote display is available to the computer system, the remote display belonging to a separate device connected to the computer system via the remote communications device;

identifying one or more display capabilities of the remote display, including at least a display resolution of the remote display;

identifying that a context change has occurred in one or more of the computer system or the separate device and in response, enumerating a list of enabled plug-ins selected from a plurality of available plug-ins, each of the plurality of available plug-ins specifying one or more contexts associated with a display of application data from a corresponding application program at the remote display;

based on the list of enabled plug-ins and the identified context change, permitting a particular application program corresponding to one of the enabled plug-ins to send supplemental application data for the particular application program over the remote communications device to the separate device for display at the remote display, the supplemental application data formatted for the one or more display capabilities of the remote display, while the particular application program concurrently displays primary application data for the particular application program at the primary display device;

receiving user input for the particular application program over the remote communications device from the separate device, while displaying the primary application data, the user input causing a change in state of the particular application program that changes at least the primary application data displayed at the main display, the user input generated by a user interacting with the displayed supplemental application data at the remote display using one or more actuators of the separate device; and

based on the received user input and the change in state of the particular application program, permitting the particular application program to send different supplemental application data over the remote communications device to the separate device for display at the remote display, while the particular application program concurrently displays the changed primary application data at the primary display device.