Remote session media data flow and playback

• US 8,127,036 B2
• Filed: 06/30/2006
• Issued: 02/28/2012
• Est. Priority Date: 06/30/2006
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Abdo, Nadim Y., Stoyanov, Vladimir K.
• Primary Examiner(s): Vaughn, Jr., William C.
• Assistant Examiner(s): ZHANG, SHIRLEY X
• Application Number:
  US11/428,238
• Publication Number:
  US 20080005801A1
• Time in Patent Office: 2,069 Days
• Field of Search: 709/231, 725 37- 61, 725 86-134
• US Class Current: 709/231
• CPC Class Codes: G06F 16/40 of multimedia data, e.g. slideshows comprising image and additional audio data retrieval of still image data G06F16/50; retrieval of audio data G06F16/60; retrieval of video data G06F16/70
• CPC Class Codes: H04L 65/61 for supporting one-way streaming services, e.g. Internet radio
• CPC Class Codes: H04L 65/613 for the control of the source by the destination control signals issued by the client directed to the server or network components specially adapted for selective content distribution H04N21/637
• CPC Class Codes: H04L 65/752 adapting media to network capabilities
• CPC Class Codes: H04L 69/04 Protocols for data compression, e.g. ROHC

Abstract

In one embodiment of the present invention, a first computer system is capable of performing a method whereby the first computer system may provide a continuous media data flow to a second computer system. The method involves the first computer system accessing that is to be presented at a second computer system. The first computer system formulates metadata representing the location and boundaries of a media application window. The first computer system formulates windows presentation information representative of the configuration of other interface elements that are to be presented along with the media information at the second computer system. The first computing system transmits the windows presentation information to the second computer system. The first computing system transmits the metadata to the second computer system and separately transmits the media information to the second computer system.

Claims

1. One or more computer storage devices having stored thereon computer-executable instructions that, when executed by one or more processors of a computer system, cause the computer system to perform a method for a first computer system to provide a continuous media data flow to a second computer system, the method comprising:

an act of the first computer system accessing a video file that is to be presented at a second computer system that is connected to a common network with the first computer system while the first and second computer systems are engaged in a terminal server session;

an act of the first computer system rendering the video file within a local media application window at the first computer system, which causes a first portion of a display screen displaying the rendered video file to change more rapidly than one or more other portions of the display screen that remain relatively unchanged;

an act of the first computer system detecting a location and boundaries of the local media application window used to render the video file at the first computer system by detecting the location and boundaries of the first portion of the display screen that is displaying the rendered video file and that is changing more rapidly than the one or more other portions of the display screen that remain relatively unchanged;

an act of the first computer system reformatting the video file to generate corresponding media information to send to the second computer system;

an act of the first computer system formulating metadata representing (i) the location and boundaries of local media application window used to render the video file, which is used by the second computer system to determine where to render corresponding media information at a display at the second computer system when the corresponding media information is received at the second computer system; and

(ii) a format type of the corresponding media information;

an act of the first computer system also formulating windows presentation information, which is different than the location and boundaries represented by the metadata, and which is representative of one or more interface elements of the local media application window, the windows presentation information used by the second computer system to generate a corresponding media application window at the display at the second computer system in which the second computer system renders the corresponding media information when the corresponding media information is received;

an act of the first computer system transmitting the windows presentation information to the second computer system, such that the second computer system can appropriately generate the corresponding media application window at the display at the second computer system;

an act of the first computer system transmitting the metadata to the second computer system to inform the second computer system of (i) the location and boundaries of the rendered video file, and (ii) the corresponding media information format type, such that the second computer system can appropriately determine where to render the corresponding media information at the display at the second computer system; and

an act of the first computer system transmitting the corresponding media information over the network to the second computer system separately from the windows presentation information such that the corresponding media information can be subject to further processing without affecting the windows presentation information.

2. The one or more computer storage devices according to claim 1, wherein rendering the video file within the local media application window at the first computer system comprises using a video mixer renderer (VMR) to render the video file within the local media application window.

3. The one or more computer storage devices according to claim 1, wherein detecting the location and boundaries of the local media application window used to render the video file comprises using driver-level heuristics to determine the location and boundaries of the rendered video file within the local media application window.

4. The one or more computer storage devices according to claim 1, wherein reformatting the video file to generate corresponding media information comprises the first computer system compressing the video file in order to improve transferability over the common network connection.

5. The one or more computer storage devices according to claim 4, wherein the video file is compressed at a variable rate of compression.

6. The one or more computer storage devices according to claim 5, wherein the variable rate of compression is based on network state information.

7. The one or more computer storage devices according to claim 6, wherein the network state information comprises parameters including current level of network traffic, available bandwidth, routing path, and physical distance between nodes.

8. The one or more computer storage devices according to claim 6, wherein the compressed video file is spooled until the network state information indicates that a substantially reliable connection exists such that the compressed video file can be continuously transmitted.

9. The one or more computer storage devices according to claim 4, wherein the video file is compressed using a code.

10. The one or more computer storage devices according to claim 4, wherein the video file is compressed in real-time.

11. The one or more computer storage devices according to claim 4, wherein the video file is compressed using DirectShow.

12. The one or more computer storage devices according to claim 1, wherein reformatting the video file to generate the corresponding media information comprises the first computer system reformatting the video file to a different format type to facilitate playback on the second computer system when the second computer system is incapable of rendering the video file in its original format.

13. The one or more computer storage devices according to claim 12, wherein the format type includes at least one of the following:

Audio Video Interleave (AVI), Windows Media Video (WMV), Windows Media Audio (WMA), Moving Picture Experts Group 1 (MPEG-1), MPEG-2, MPEG-3, MPEG-4, Apple™

Quicktime, Macromedia™

Flash, RealNetworks™

RealMedia, RealVideo, and RealAudio, Ogg Vorbis and Theora, or Google™

Video.

14. The one or more computer storage devices according to claim 1, further comprising:

an act of the first computer system attaching a timestamp to an audio portion of the corresponding media information that is to be transmitted to the second computer system;

an act of the first computer system attaching a timestamp to a video portion corresponding to the time stamp of the audio portion of the corresponding media information;

an act of synchronizing the audio and video portions of the corresponding media information by arranging the audio and video portions according to the timestamp of the audio portion; and

an act of transmitting the synchronized audio and video portions of the corresponding media information over the network to the second computer system separately from the windows presentation information.

17. The one or more computer storage devices according to claim 1, wherein the second computer system uses the terminal server session to gain access to one or more applications provided by the first computer system, including the media application.

18. The one or more computer storage devices according to claim 1, wherein the windows presentation information includes at least one of:

a number of application windows to present, an order in which one or more application windows should be displayed, or whether each application window is maximized or minimized.

19. The one or more computer storage devices according to claim 1, wherein the one or more interface elements of the local media application window include at least one of a window icon or an application icon.

20. The one or more computer storage devices according to claim 1, wherein the second computing system uses the windows presentation information, the metadata, and the corresponding media information to render the corresponding media information within in the corresponding media window at the display at the second computing system and that adheres to the location and boundary information and the windows presentation information, rather than displaying bitmap images received through the terminal server session that represent periodic screen captures of the rendered media information as rendered by the first computing system.

21. The one or more computer storage devices according to claim 1, wherein said accessing the video file that is to be presented at the second computer system includes the second computer system initiating the launching of the local media player application at the first computer system through the terminal server session.

22. The one or more computer storage devices according to claim 1, wherein transmitting the windows presentation information to the second computer system includes transmitting the windows presentation information through the terminal server session.

23. The one or more computer storage devices according to claim 1, wherein transmitting the metadata to the second computer system includes transmitting the metadata through the terminal server session.

15. In a computing environment, a method for a first computer system to provide a continuous media data flow to a second computer system, the method comprising:

an act of the first computer system accessing a video file that is to be presented at a second computer system that is connected to a common network with the first computer system while the first and second computer systems are engaged in a terminal server session;

an act of the first computer system rendering the video file within a local media application window at the first computer system, which causes a first portion of a display screen displaying the rendered video file to change more rapidly than one or more other portions of the display screen that remain relatively unchanged;

an act of the first computer system detecting a location and boundaries of the local media application window used to render the video file at the first computer system by detecting the location and boundaries of the first portion of the display screen that is displaying the rendered video file and that is changing more rapidly than the one or more other portions of the display screen that remain relatively unchanged;

an act of the first computer system formulating metadata representing (i) the location and boundaries of the local media application window used to render the video file, which is used by the second computer system to determine location and boundaries in which to render the video file at a display at the second computer system when the video file is received at the second computer system; and

(ii) a format type of the video file;

an act of the first computer system also formulating windows presentation information that is different than the location and boundaries represented by the metadata and that is representative of one or more interface elements that are to be presented along with the video file as part of a corresponding media application window at the display at the second computer system, the corresponding media application window mirroring the one or more interface elements as displayed in the local media application window;

an act of the first computer system transmitting the windows presentation information to the second computer system, such that the second computer system can appropriately generate the corresponding media application window;

an act of the first computer system transmitting the metadata to the second computer system to inform the second computer system of (i) the location and boundaries of the rendered video file, and (ii) the video file format type, such that the second computer system can appropriately determine where to render the received video file within the corresponding media application window at the display at the second computer system; and

an act of the first computer system transmitting the video file over the network to the second computer system separately from the windows presentation information such that the video file can be subject to further processing without affecting the windows presentation information.

16. The method of claim 15, further comprising an act of the first computer system compressing the video file prior to transmitting the video file over the network in order to improve transferability over the common network connection.

24. A server computer system, comprising:

one or more processors; and

one or more computer storage devices having stored thereon computer executable instructions that, when executed by the one or more processors, implement a method, comprising;

an act of the server computer system accessing media information comprising a video file, that is to be presented at a client computer system that is connected to a common network with the server computer system while the server and client computer systems are engaged in a terminal server session;

an act of the server computer system rendering the video file within a local media application window at the server computer system, which causes a portion of a display screen displaying the rendered video file to change more rapidly than one or more other portions of the display screen that remain relatively unchanged;

an act of the server computer system detecting a location and boundaries of the local media application window used to render the video file at the server computer system by detecting the location and boundaries of the portion of the display screen that is displaying the rendered video file and that is changing more rapidly than the one or more other portions of the display information at the server computer system that remain relatively unchanged;

an act of the server computer system formulating metadata representing (i) the location and boundaries of the local media application window used to render the video file, which is used by the client computer system to determine where to render the video file when at a display at the client computer system when the video file is received configured to present the accessed media information at the client computer system; and

(ii) a format type of the video file;

an act of the server computer system formulating windows presentation information;

which is different than the location and boundaries represented by the metadata and which is representative of one or more interface elements of the local application window, the windows presentation information used by the client computer system to generate a corresponding media application window at the display at the client computer system in which the client computer system renders the video file when the video file is received;

an act of the server computer system transmitting the windows presentation information to the client computer system such that the client computer system can appropriately generate the corresponding media application window at the display at the client computer system;

an act of the server computer system transmitting the metadata to the client computer system to inform the client computer system of (i) the location and boundaries of the rendered video file and (ii) the video file format type, such that the client computer system can appropriately determine where to render the video file at the display at the client computer system; and

an act of the server computer system transmitting the video file over the network to the client computer system separately from the windows presentation information such that the video file can be subject to further processing without affecting the windows presentation information.