Capturing a computing experience

• US 8,949,407 B2
• Filed: 06/17/2009
• Issued: 02/03/2015
• Est. Priority Date: 06/17/2009
• Status: Active Grant

• Current Assignee: Zhigu Holdings Limited
• Original Assignee: Microsoft Corporation
• Inventors: Carpenter, Todd L., Abzarian, David, Panchapagesan, Seshagiri, Kulkarni, Harish S.
• Primary Examiner(s): LIN, WEN TAI
• Application Number:
  US12/486,069
• Publication Number:
  US 20100325258A1
• Time in Patent Office: 2,057 Days
• Field of Search: 709/224, 705/7.11, 705/7.29, 705/7.32, 705/14.1, 705/14.19, 705/14.25, 705/14.39
• US Class Current: 709/224
• CPC Class Codes: G06F 11/3414 Workload generation, e.g. scripts, playback
• CPC Class Codes: G06F 11/3438 monitoring of user actions tracking the activity of the user H04L67/535
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising

Abstract

The described implementations relate to capturing a computing experience. In one case, a user session capture tool can launch a remote user session where a user-interface and user inputs are gathered from a single computing device. Remote user session data produced by the remote user session can be analyzed to determine user activity.

Claims

1. A hardware computer-readable storage media having instructions stored thereon that when executed by a hardware processor cause the hardware processor to perform acts, the acts comprising:

launching a remote user session on a computing device; and

while the remote user session is being conducted on the computing device and without notifying a user of the computing device that the remote user session is being conducted, using the remote user session to;

capture a computing experience on the computing device, wherein the computing experience includes a local user-interface generated by the computing device and associated local user-input entered to the computing device by the user of the computing device;

create remote user session data associated with the computing experience, wherein the remote user session data includes at least one Internet Protocol packet stream comprising images of the local user-interface and the remote user session data is created on the computing device where the remote user session is launched;

redirect the at least one Internet Protocol packet stream including the images of the local user-interface back to the computing device where the remote user session is launched, the redirecting comprising sending the at least one Internet Protocol packet stream including the images of the local user-interface from the computing device to the computing device via a network port on the computing device; and

present the images of the local user-interface included in the at least one Internet Protocol packet stream on the computing device.

2. The hardware computer-readable storage media of claim 1, wherein the remote user session is a Remote Desktop Protocol session.

3. The hardware computer-readable storage media of claim 1, the acts further comprising:

superimposing the images of the local user-interface included in the at least one Internet Protocol packet stream over the local user-interface of the computing device so that the user views the superimposed images instead of the local user-interface.

4. The hardware computer-readable storage media of claim 1, the acts further comprising:

analyzing the images of the local user-interface included in the at least one Internet Protocol packet stream to detect that the user has attempted to access sensitive material using the computing device.

5. The hardware computer-readable storage media of claim 1, wherein the at least one Internet Protocol packet stream that is both created by the computing device and redirected to the computing device includes input session data identifying a position, a movement, or a click of a mouse of the computing device.

6. The hardware computer-readable storage media of claim 5, wherein the at least one Internet Protocol packet stream created by the computing device is continually redirected to the computing device as the user interacts with the computing device and is continually updated during the remote user session to reflect subsequent local user-input entered to the computing device by the user.

7. The hardware computer-readable storage media of claim 1, wherein the at least one Internet Protocol packet stream comprises a Transmission Control Protocol/Internet Protocol (TCP/IP) packet stream that is communicated from the computing device to the computing device via the network port.

8. The hardware computer-readable storage media of claim 1, wherein the at least one Internet Protocol packet stream is redirected to the computing device without sending the at least one Internet Protocol packet stream to another computing device.

9. A computing device, comprising:

a display;

a hardware processor; and

a computer-readable storage media storing instructions which, when executed by the hardware processor, cause the hardware processor to;

launch a remote user session that executes on the hardware processor and captures a computing experience taking place on the computing device with a user, wherein the computing experience includes a local user-interface of the computing device; and

while the remote user session is being conducted and as the user enters input to the computing device, repeatedly;

incorporate the input entered to the computing device into at least one Internet Protocol packet stream;

update the local user-interface of the computing device responsive to the input entered to the computing device;

obtain images of the updated local user-interface of the computing device and incorporate the images of the updated local user-interface into the at least one Internet Protocol packet stream; and

communicate the at least one Internet Protocol packet stream comprising the images of the updated local user-interface of the computing device and the input entered to the computing device from the computing device directly back to the computing device via a network protocol without communicating the at least one Internet Protocol packet stream through an intervening computing device.

10. The computing device of claim 9, wherein the instructions further cause the hardware processor to:

display a video that the user is mandated to watch on the local user-interface of the computing device; and

analyze the at least one Internet Protocol packet stream including the images of the updated local user-interface to determine whether the user has watched the video that the user is mandated to watch.

11. The computing device of claim 9, wherein the instructions further cause the hardware processor to:

include, in the at least one Internet Protocol packet stream, keystroke data indicating keystrokes entered by the user during the remote user session and mouse data indicating mouse position, mouse movement, and mouse clicks entered by the user during the remote user session.

12. The computing device of claim 11, wherein the keystroke data and mouse data are included in an input packet stream of the at least one Internet Protocol packet stream and the images of the local user-interface are included in an output packet stream of the at least one Internet Protocol packet stream.

13. The computing device of claim 12, wherein the instructions further cause the hardware processor to:

send the at least one input packet stream and the at least one output packet stream to a second computing device for analysis.

14. A method comprising:

by a single computing device;

capturing a computing experience on the single computing device using a remote user session that is executing on the single computing device, wherein the computing experience includes user-input entered by a user as the user interacts with the single computing device during the remote user session and a local user-interface generated by the single computing device as the user enters the user-input;

as the user continues interacting with the single computing device during the remote user session;

representing both the user-input and the local user-interface in at least one Internet Protocol packet stream that includes images of the local user-interface as well as input data representing the user-input;

redirecting the at least one Internet Protocol packet stream from the single computing device to the single computing device where the remote user session is executing, the at least one Internet Protocol packet stream being redirected via a network port of the single computing device via a network protocol; and

updating a display of the single computing device using the images of the local user-interface included in the at least one Internet Protocol packet stream, the display being updated based on the user-input entered to the single computing device by the user.

15. The method according to claim 14, further comprising:

analyzing the at least one Internet Protocol packet stream to determine that the user started playing a song or movie on the single computing device; and

detecting that the user did not have digital rights management permission to play the song or movie on the single computing device.

16. The method according to claim 15, further comprising:

stopping the song or the movie from playing on the single computing device responsive to the analyzing the at least one Internet Protocol packet stream revealing that the user did not have the digital rights management permission to play the song or the movie on the single computing device.

17. The method according to claim 14, further comprising:

analyzing the at least one Internet Protocol packet stream to detect that the user accessed inappropriate content using the single computing device; and

responsive to detecting that the user accessed the inappropriate content, causing a source of the inappropriate content to be blocked.

18. The method of claim 14, further comprising:

analyzing the at least one Internet Protocol packet stream to detect that the user accessed inappropriate content using the single computing device; and

responsive to detecting that the user accessed the inappropriate content, adjusting a permission level to lower a level of authorization available to the user on the single computing device.

19. The method of claim 14, further comprising:

analyzing the at least one Internet Protocol packet stream to determine an amount of time that the user viewed the inappropriate content on the single computing device; and

distinguishing between inadvertent user access of the inappropriate content and intentional user access of the inappropriate user content based on the amount of time that the user viewed the inappropriate content on the single computing device.

20. The method of claim 14, further comprising:

causing a local printer of the single computing device to appear as a remote printer instead of as the local printer.