Server-side media stream manipulation for emulation of media playback functions

• US 7,890,985 B2
• Filed: 05/22/2006
• Issued: 02/15/2011
• Est. Priority Date: 05/22/2006
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Speicher, Ashley C., Bowra, Todd, Fang, Nicholas J.
• Primary Examiner(s): Koenig; Andrew Y
• Assistant Examiner(s): Le; Rong
• Application Number:
  US11/438,408
• Publication Number:
  US 20070271388A1
• Time in Patent Office: 1,730 Days
• Field of Search: None
• US Class Current: 725/88
• CPC Class Codes: H04L 2012/2849 Audio/video appliances
• CPC Class Codes: H04L 65/762 at the source reformatting of additional data in video distribution servers H04N21/2355

Abstract

A media server in a home entertainment network allows a media rendering device to act as a digital media receiver. The media server enables trick mode functions such as scan forward or reverse, slow motion, pause, and seek on rendering devices with capabilities limited to 1× playback. The media server receives commands from user input at the rendering device transmitted over the network to the media server. The media server manipulates the media stream before transmission over the network by deleting or adding frames to the media stream as necessary to emulate the desired playback functions and replacing frame timestamps to create a media stream that appears to the media rendering device as a sequential 1× stream. The transmitted media stream may not contain all of the media content stored in the corresponding media file on the media server, thus avoiding limitations on the download bandwidth and local client storage.

Claims

1. A method for enabling, over a network, rate-altering media playback functions at a frame rate expected by a rendering device, the method comprising

receiving a command at a media server over the network from the rendering device to initiate a selected rate-altering media playback function;

accessing, by the media server, an original stream of frames having sequential frame timestamps;

generating an intermediate series of frames at the media server for transmission to the rendering device over the network by deleting frames having frame timestamps from the original stream of frames in excess of the frame rate expected by the rendering device when the selected rate-altering media playback function corresponds to an increased play rate;

generating an intermediate series of frames at the media server for transmission to the rendering device over the network by inserting one or more intermediate frames that do not have timestamps into the original stream of frames when the selected rate-altering media playback function corresponds to a decreased play rate and insufficient frames originally exist in the original stream of frames to fulfill the frame rate expected by the rendering device;

detecting discontinuity in frame timestamps between frames in the intermediate series of frames with respect to a timestamp increment expected by the rendering device, the detected discontinuity in the frame timestamps caused by deletion of frames from the original stream of frames when the selected rate-altering media playback function corresponds to the increased play rate and caused by insertion of one or more intermediate frames into the original stream of frames when the selected rate-altering media playback function corresponds to the decreased play rate; and

providing the intermediate series of frames with sequential frame timestamps separated by the timestamp increment expected by the rendering device before transmission of frames in the intermediate series of frames over the network to account for the detected discontinuity with respect to the frame rate expected by the rendering device by;

adjusting frame timestamps of all frames in the intermediate series of frames having frame timestamps according to a frame timestamp of a frame from the original stream of frames immediately preceding the command to initiate the selected rate-altering media playback function and a play rate corresponding to the selected rate-altering media playback function, and

assigning frame timestamps to the intermediate frames that do not have frame timestamps inserted into the original stream of frames when the selected rate-altering media playback function corresponds to the decreased play rate.

2. The method of claim 1 further comprising transmitting the intermediate series of frames over the network to the rendering device.

3. The method of claim 1 further comprising

calculating an inverse of the play rate corresponding to the selected rate-altering media playback function;

subtracting 1 from the inverse to determine a difference; and

inserting a number of intermediate frames equal to the difference resulting from the subtracting operation between each pair of adjacent frames in the original stream of frames.

4. The method of claim 1, wherein the one or more intermediate frames inserted into the original stream of frames are duplicate frames of another frame in the original stream of frames.

5. The method of claim 4, further comprising successively adding a product of an inverse frame rate amount and a respective insertion number of a respective intermediate frame of the one or more intermediate frames to an original frame timestamp of each of the one or more intermediate frames.

6. The method of claim 1, wherein the one or more intermediate frames inserted into the original stream of frames are extrapolated and/or interpolated frames from at least one frame in the original stream of frames.

7. The method of claim 6, further comprising successively adding a product of an inverse frame rate amount and a respective insertion number of a respective intermediate frame of the one or more intermediate frames to an original frame timestamp of each of the one or more intermediate frames.

8. A computer-readable storage medium that does not consist of a signal, the computer-readable storage medium having stored computer-executable instructions causing a computing device to perform a computer process implementing the method of claim 1.

9. A method of for enabling, over a network, rate-altering media playback functions at a frame rate expected by a rendering device, the method comprising

receiving a command over the network from the rendering device to initiate a selected rate-altering media playback function;

deleting frames from a stream of frames in excess of the frame rate expected by the rendering device; and

adjusting frame timestamps of frames remaining in the stream of frames before transmission of the remaining frames over the network at the frame rate expected by the rendering device by;

dividing the frame timestamps of the frames remaining in the stream of frames by a play rate corresponding to the selected rate-altering media playback function, and

substituting a quotient of the dividing operation for the frame timestamps of the frames remaining in the stream of frames, respectively.

10. A computer-readable storage medium that does not consist of a signal, the computer-readable storage medium having stored computer-executable instructions causing a computing device to perform a computer process implementing the method of claim 9.

11. A method of for enabling, over a network, rate-altering media playback functions at a frame rate expected by a rendering device, the method comprising

receiving a command over the network from the rendering device to initiate a selected rate-altering media playback function;

deleting frames from a stream of frames in excess of the frame rate expected by the rendering device; and

adjusting frame timestamps of each of the frames remaining in the stream of frames after the deleting operation before transmission of the remaining frames over the network at the frame rate expected by the rendering device by subtracting a product of the total number of deleted frames to that point and an inverse of the frame rate expected by the rendering device.

12. A computer-readable storage medium that does not consist of a signal, the computer-readable storage medium having stored computer-executable instructions causing a computing device to perform a computer process implementing the method of claim 11.

13. A method of for enabling, over a network, rate-altering media playback functions at a frame rate expected by a rendering device, the method comprising

receiving a command over the network from the rendering device to initiate a selected rate-altering media playback function;

generating a series of frames for transmission to the rendering device over the network by inserting one or more intermediate frames into a stream of frames when insufficient frames originally exist in the stream of frames to fulfill the frame rate expected by the rendering device; and

adjusting frame timestamps of each of the frames in the stream of frames after the inserting operation and before transmission of the stream of frames at the frame rate expected by the rendering device by adding a product of the total number of inserted frames to that point and an inverse of the frame rate expected by the rendering device.

14. A computer-readable storage medium that does not consist of a signal, the computer-readable storage medium having stored computer-executable instructions causing a computing device to perform a computer process implementing the method of claim 13.

15. A method of for enabling, over a network, rate-altering media playback functions at a frame rate expected by a rendering device, the method comprising

receiving a command over the network from the rendering device to initiate a selected rate-altering media playback function;

deleting frames from a stream of frames in excess of the frame rate expected by the rendering device;

subtracting frame timestamps of adjacent frames originally in the stream of frames to determine a difference;

dividing the difference by a play rate corresponding the a selected, rate-altering media playback function to determine a quotient;

adding the quotient to an earlier frame of the adjacent frames to determine a first sum;

substituting the first sum for the frame timestamp of the earlier frame;

multiplying a respective insertion number of a respective of the one or more intermediate frames by an inverse of the frame rate to determine a product;

adding the product to the first sum to determine a second sum; and

assigning the second sum as a generated timestamp to the respective one of the one or more intermediate frames.

16. A computer-readable storage medium that does not consist of a signal, the computer-readable storage medium having stored computer-executable instructions causing a computing device to perform a computer process implementing the method of claim 15.

17. A method for emulating rate-altering media playback functions for playback of a media stream across a network on a constant frame rate rendering device, the method comprising

receiving a command at a media server over the network from the rendering device to initiate a selected rate-altering media playback function;

accessing, by the media server, an original media stream having sequential frame timestamps;

generating an intermediate series of frames at the media server for transmission to the rendering device over the network by deleting frames having frame timestamps from the original media stream in excess of the constant frame rate when the selected rate-altering media playback function corresponds to an increased play rate;

generating an intermediate series of frames at the media server for transmission to the rendering device over the network by inserting one or more intermediate frames that do not have timestamps into the original media stream when the selected rate-altering media playback function corresponds to a decreased play rate and insufficient source frames exist in the original media stream to meet the constant frame rate of the rendering device;

detecting discontinuity in frame timestamps between frames in the intermediate series of frames with respect to a constant timestamp increment of the rendering device, the detected discontinuity in the frame timestamps caused by deletion of frames from the original media stream when the selected rate-altering media playback function corresponds to the increased play rate and caused by insertion of one or more intermediate frames into the original media stream when the selected rate-altering media playback function corresponds to the decreased play rate; and

providing the intermediate series of frames with sequential frame timestamps separated by the constant timestamp increment of the rendering device before transmission of frames in the intermediate series of frames over the network to account for the detected discontinuity with respect to the constant frame rate of the rendering device by;

adjusting frame timestamps of all frames in the intermediate series of frames having frame timestamps according to a frame timestamp of a frame from the original media stream immediately preceding the command to initiate the selected rate-altering media playback function and a play rate corresponding to the selected rate-altering media playback function, and

assigning frame timestamps to the intermediate frames that do not have frame timestamps inserted into the original media stream when the selected rate-altering media playback function corresponds to the decreased play rate.

18. The method of claim 17, wherein the one or more intermediate frames inserted into the original media stream are duplicate frames of another frame in the original media stream.

19. The method of claim 17, wherein the one or more intermediate frames inserted into the original media stream are interpolated frames from at least one frame in the original media stream.

20. A computer-readable storage medium having stored computer-executable instructions causing a computing device to perform a computer process implementing the method of claim 17.

21. A media server comprising a processor;

a memory under control of the processor, wherein the memory stores original media content having sequential frame timestamps;

an audio/video control module, under control of the processor, that receives a command to initiate a rate-altering media playback function over a network from a constant frame rate rendering device;

an audio/video source module, under control of the audio/video control module and the processor, that accesses the original media content stored on the memory;

an audio/video sample manipulator module, under control of the audio/video control module and the processor, that;

receives the original media content from the audio/video source module,

manipulates the original media content to generate an intermediate series of frames for transmission to the rendering device that emulates an increased or decreased play rate corresponding to the rate-altering media playback function,

detects discontinuity in frame timestamps between frames in the intermediate series of frames with respect to a constant timestamp increment of the rendering device, the detected discontinuity in the frame timestamps caused by manipulating the original media content to delete frames having frame timestamps from the original media content when the rate-altering media playback function corresponds to the increased play rate or insert intermediate frames that do not have frame timestamps into the original media content when the rate-altering media playback function corresponds to the decreased play rate,

provides the intermediate series of frames with sequential frame timestamps separated by the constant timestamp increment of the rendering device before transmission of frames in the intermediate series of frames over the network to account for the detected discontinuity with respect to the constant timestamp increment by;

adjusting frame timestamps of all frames in the intermediate series of frames having frame timestamps according to a frame timestamp of a frame from the original media content immediately preceding the command to initiate the rate-altering media playback function and a play rate corresponding to the rate-altering media playback function, and

assigning frame timestamps to the intermediate frames that do not have frame timestamps inserted into the original media content when the rate-altering media playback function corresponds to the decreased play rate, and

outputs the intermediate series of frames over the network to the rendering device at the constant rate of the rendering device.

22. The media server of claim 21, wherein the audio/video sample manipulator module deletes frames from the original media content in excess of the constant rate of the rendering device.

23. The media server of claim 21, wherein the audio/video sample manipulator module inserts one or more intermediate frames into the original media content when insufficient frames originally exist in the original media content to fulfill the constant rate of the rendering device.

24. The media server of claim 21, further comprising a network link for transmission of the intermediate series of frames over the network to the rendering device.