Methods and systems for processing digital data rate and directional playback changes

• US 20040001695A1
• Filed: 04/21/2003
• Published: 01/01/2004
• Est. Priority Date: 06/28/2002
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Evans, Glenn F., Gates, Matthijs A., Chakrabarti, Alok
• Granted Patent: US 8,169,345 B2
• Time in Patent Office: Days
• US Class Current: 386/68
• CPC Class Codes: G11B 20/10527 Audio or video recording; Data buffering arrangements G11B20/12 - G11B20/18 take precedence
• CPC Class Codes: G11B 2020/10537 Audio or video recording
• CPC Class Codes: G11B 2020/10694 output interface, i.e. the way data leave the buffer, e.g. by adjusting the clock rate
• CPC Class Codes: G11B 27/005 Reproducing at a different information rate from the information rate of recording for television signals H04N5/783
• CPC Class Codes: G11B 27/034 on discs G11B27/036, G11B27/038 take precedence
• CPC Class Codes: H04N 5/765 Interface circuits between an apparatus for recording and another apparatus
• CPC Class Codes: H04N 5/775 between a recording apparatus and a television receiver
• CPC Class Codes: H04N 5/783 Adaptations for reproducing at a rate different from the recording rate
• CPC Class Codes: H04N 5/85 on discs or drums
• CPC Class Codes: H04N 9/8042 involving data reduction
• CPC Class Codes: H04N 9/8063 using time division multiplex of the PCM audio and PCM video signals
• CPC Class Codes: H04N 9/8227 the additional signal being at least another television signal

Abstract

Various methods and systems permit digital data, such as video data, audio/video data, audio/video/subpicture data and the like, to be processed in a manner that permits playback at different speeds in both forward and reverse directions. Various embodiments are also directed to handling playback rate changes in a manner that can enhance the user'"'"'s experience.

Claims

1. A method comprising:

providing a filter graph having components for processing video and audio data; and

using a sideband mechanism that is supported by individual components of the filter graph to communicate with the individual components to determine, from one or more of the components, when to schedule a rate change that can be perceived as immediate by a user, and to communicate rate change information to individual components in the filter graph.

2. The method of claim 1, wherein said using of the sideband mechanism comprises using one or more interfaces supported by the individual components, to call the individual components to ascertain when a rate change has been scheduled.

3. The method of claim 1, wherein said using of the sideband mechanism comprises using one or more interfaces supported by the individual components, to call the individual components to ascertain when a rate change has been scheduled, and to call other individual components to schedule the rate change.

4. The method of claim 1, wherein said using of the sideband mechanism comprises initiating a call with a source component, to one or more downstream components, to schedule the rate change at a next available data sample.

5. The method of claim 1, wherein said using of the sideband mechanism comprises initiating a call with a source component, to one or more downstream components, to schedule the rate change at a next available data sample, and further comprising returning the rate change information to the source component responsive to initiating said call, and calling other individual components with the rate change information.

6. The method of claim 1, wherein said using of the sideband mechanism comprises initiating a call with a source component, to one or more downstream components, to schedule the rate change at a next available data sample, and further comprising:

propagating said call down the filter graph to a renderer component;

receiving the call with the renderer component;

checking a data buffer associated with renderer component to ascertain the next available data sample on which a rate change can be implemented;

scheduling a rate change on said data sample; and

propagating rate change information back up the filter graph to the source component.

7. The method of claim 6, wherein said rate change information comprises a timestamp associated with the data sample.

8. One or more computer-readable media embodying computer-readable instructions which, when executed by one or more processors, implement the method of claim 1.

9. A method comprising:

receiving a command from an application that indicates an immediate rate change is to be initiated;

selecting, as a guiding filter graph, a filter graph that is configured to process data samples;

ascertaining, from the guiding filter graph, a next available data sample on which the rate change can be implemented; and

scheduling the rate change for the next available data sample.

10. The method of claim 9, wherein the act of selecting the filter graph comprises selecting a filter graph that represents a least flexible filter graph in terms of the selected filter graph'"'"'s ability to recover from a loss of data.

11. The method of claim 9, wherein the act of selecting the filter graph comprises selecting an audio filter graph that is configured to process audio data samples.

12. The method of claim 9, wherein the act of ascertaining comprises calling a method that requests a rate change at the next available sample.

13. The method of claim 9, wherein the act of ascertaining comprises calling a method that requests a rate change at the next available sample, wherein calling is performed by a source component.

14. The method of claim 9, wherein the act of ascertaining comprises calling a method that requests a rate change at the next available sample, wherein said calling is performed by a filter graph component other than a source component.

15. The method of claim 9, wherein the act of ascertaining comprises calling a method that requests a rate change at the next available sample, wherein said calling is performed, at least in part, by propagating a call along individual Components of the guiding filter graph to a last component that is capable of scheduling a rate change.

16. The method of claim 9, wherein the act of ascertaining comprises calling a method that requests a rate change at the next available sample, wherein said calling is performed, at least in part, by propagating a call along individual components of the guiding filter graph to a renderer component.

17. The method of claim 9, wherein the act of ascertaining comprises calling a method that requests a rate change at the next available sample, wherein said calling is performed, at least in part, by propagating a call along individual components of the guiding filter graph to a renderer component that is capable of scheduling a rate change.

18. The method of claim 9, wherein the act of ascertaining comprises calling a method that requests a rate change at the next available sample, wherein said calling is performed, at least in part, by propagating a call along individual components of the guiding filter graph to a renderer component that is capable of scheduling a rate change, and further comprising examining at least one queue associated with the renderer component of the guiding filter graph to ascertain which of any data samples in the queue is the next available data sample on which a rate change can be scheduled.

19. The method of claim 9, wherein the act of ascertaining comprises calling a method that requests a rate change at the next available sample, wherein the act of ascertaining comprises, responsive to said calling, examining at least one queue associated with at least one component of the guiding filter graph to ascertain which of any data samples in the queue is the next available data sample on which a rate change can be scheduled.

20. The method of claim 9 further comprising after said scheduling, returning, to a source component, rate change information associated with the corresponding sample for which the rate change has been scheduled.

21. The method of claim 9 further comprising after said scheduling, returning, to a source component, rate change information associated with the corresponding sample for which the rate change has been scheduled, and using said rate change information to schedule a rate change on filter graphs other than the guiding filter graph.

22. The method of claim 9 further comprising after said scheduling, returning, to a source component, rate change information associated with the corresponding sample for which the rate change has been scheduled, wherein the rate change information comprises a timestamp associated with a data sample for which the rate change has been scheduled.

23. The method of claim 9, wherein the acts of ascertaining and returning are accomplished using a sideband mechanism that allows communication between individual components of the guiding filter graph independent of data samples processed by the guiding filter graph.

24. The method of claim 9, wherein the acts of ascertaining and returning are accomplished using a sideband mechanism that allows communication between individual components of the guiding filter graph independent of data samples processed by the guiding filter graph, wherein the sideband mechanism comprises one or more interfaces that support one or more callable methods.

25. The method of claim 9, wherein said act of scheduling is performed by associating, with the next available data sample, a data structure that includes data that describes the rate change.

26. The method of claim 9, wherein said act of scheduling is performed by receiving a timestamp associated with the data sample for which the rate change has been scheduled in the guiding filter graph, and issuing a rate change command to other filter graphs for data samples that correspond to the data sample for which the rate change was scheduled in the guiding filter graph.

27. The method of claim 9, wherein said act of scheduling is performed by receiving a timestamp associated with the data sample for which the rate change has been scheduled in the guiding filter graph, and issuing a rate change command to other filter graphs for data samples that correspond to the data sample for which the rate change was scheduled in the guiding filter graph, and further comprising propagating the issued rate change command along a plurality of components of the other filter graphs, individual components using the rate change command to check associated queues and schedule the rate change in an event a corresponding data sample is found.

28. A system comprising:

one or more processors;

one or more computer-readable media;

computer-readable instructions embodied on the one or more computer-readable media which, when executed by the one or more processors, cause the one or more processors to;

receive a command from an application that indicates an immediate rate change is to be initiated;

select, as a guiding filter graph, a filter graph that is configured to process data samples;

ascertain, from the guiding filter graph, a next available data sample on which the rate change can be implemented; and

schedule the rate change for the next available data sample.

29. The system of claim 28, wherein the instructions cause the one or more processors to select a filter graph that represents a least flexible filter graph in terms of the selected filter graph'"'"'s ability to recover from a loss of data.

30. The system of claim 28, wherein the instructions cause the one or more processors to select an audio filter graph that is configured to process audio data samples.

31. The system of claim 28, wherein the instructions cause the one or more processors to call a method that requests a rate change at the next available sample.

32. The system of claim 28, wherein the instructions cause the one or more processors to call a method that requests a rate change at the next available sample, and propagate the call along individual components of the guiding filter graph to a last component that is capable of scheduling a rate change.

33. The system of claim 28, wherein the instructions cause the one or more processors to return, to a source component, rate change information associated with the corresponding sample for which the rate change has been scheduled.

34. The system of claim 28, wherein the instructions cause the one or more processors to return, to a source component, rate change information associated with the corresponding sample for which the rate change has been scheduled, and use the rate change information to schedule a rate change on filter graphs other than the guiding filter graph.

35. The system of claim 28, wherein the instructions cause the one or more processors to schedule the rate change by associating, with the next available data sample, a data structure that includes data that describes the rate change.

36. The system of claim 28, wherein the instructions cause the one or more processors to schedule the rate change by receiving a timestamp associated with the data sample for which the rate change has been scheduled in the guiding filter graph, and issuing a rate change command to other filter graphs for data samples that correspond to the data sample for which the rate change was scheduled in the guiding filter graph.

37. A system comprising:

a source component configured to read multimedia content and receive commands from an application to initiate an immediate rate change on the multimedia content;

a first filter graph associated with the source component and comprising individual components downstream from the source component, the individual components being configured to process data samples associated with the multimedia content for rendering;

at least one other filter graph associated with the source component and comprising individual components downstream from the source component, the individual components of the other filter graph being configured to process data samples associated with the multimedia content for rendering;

a virtual command channel supported by the source component and the filter graph components, the virtual command channel being configured to;

enable the source component to ascertain, from one or more of the filter graph components of one of the filter graphs, a next available data sample on which a rate change can be scheduled; and

communicate a scheduled rate change to components of at least one different filter graph.

38. The system of claim 37, wherein the virtual command channel is configured to enable the source component to request a rate change be scheduled by a renderer component on at least one of the filter graphs.

39. The system of claim 37, wherein at least one filter graph is configured to enable an associated renderer component to schedule a rate change.

40. The system of claim 37, wherein at least one filter graph is configured to enable an associated renderer component to schedule a rate change by having the renderer component:

check an associated queue having data samples therein to identify a next available data sample, and

return a timestamp associated with the next available data sample to the source component.

41. The system of claim 37, wherein at least one filter graph is configured to enable an associated renderer component to schedule a rate change by having the renderer component:

check an associated queue having data samples therein to identify a next available data sample, and

return a timestamp associated with the next available data sample to the source component;

and wherein the virtual command channel is further configured to enable the source component to communicate the returned timestamp to at least one other filter graph.

42. The system of claim 37, wherein each filter graph comprises at least a decoder component and a renderer component, each of which being configured to receive calls initiated at the source component, which calls can cause the decoder and renderer components to check respective associated queues for data samples on which to schedule a rate change.

43. The system of claim 37, wherein the virtual command channel is configured to enable upstream and downstream communication between individual components of the filter graphs.

44. The system of claim 37, wherein the virtual command channel is configured to enable upstream and downstream communication between individual components of the filter graphs, and wherein the virtual command channel is configured to enable rate change commands to be propagated downstream from component to component.

45. The system of claim 37, wherein the virtual command channel is configured to enable upstream and downstream communication between individual components of the filter graphs, and wherein the virtual command channel is configured to enable rate changes to be scheduled by enabling a data structure to be associated with data samples on which rate changes have been scheduled.

46. A computing device embodying the system of claim 37.

47. A system comprising:

one or more computer-readable media;

a first interface embodied on the computer-readable media and which maintains information that indicates whether a rate change has been scheduled for a data sample and, when a rate change is scheduled, an indication of the rate change;

a second interface embodied on the computer-readable media and which is configured to be used by an application to communicate a rate change to a media source; and

a third interface embodied on the computer-readable media and which enables individual components of a filter graph to communicate rate changes with one another.

48. The system of claim 47, wherein the first interface indicates a rate change by using a timestamp associated with a data sample.

49. The system of claim 47, wherein the first interface indicates a rate change by using a timestamp associated with a data sample, and wherein the first interface comprises one or more flags that are associated with data samples and pertain to how a data sample is to be handled by a filter graph.

50. The system of claim 47, wherein the first interface indicates a rate change by using a timestamp associated with a data sample, and wherein the first interface comprises one or more flags that are associated with data samples and pertain to how a data sample is to be handled by a filter graph, wherein one of the flags indicates a first data sample in a reverse playback group of data samples and another flag indicates a last sample the reverse playback group of data samples.

51. The system of claim 47, wherein the first interface indicates a rate change by using a timestamp associated with a data sample, and wherein the first interface comprises one or more flags that are associated with data samples and pertain to how a data sample is to be handled by a filter graph, wherein one of the flags indicates part of a video section in forward or reverse scan mode.

52. The system of claim 47, wherein the first interface indicates a rate change by using a timestamp associated with a data sample, and wherein the first interface comprises one or more flags that are associated with data samples and pertain to how a data sample is to be handled by a filter graph, wherein one of the flags indicates a location that specifies that the data sample should be truncated at this point.

53. The system of claim 47, wherein the first interface indicates a rate change by using a timestamp associated with a data sample, and wherein the first interface comprises one or more flags that are associated with data samples and pertain to how a data sample is to be handled by a filter graph, wherein one of the flags indicates a location before which data samples should be discarded.

54. The system of claim 47, wherein the first interface indicates a rate change by using a timestamp associated with a data sample, and wherein the first interface comprises one or more flags that are associated with data samples and pertain to how a data sample is to be handled by a filter graph, wherein the flags comprise:

a flag that indicates a first data sample in a reverse playback group of data samples;

a flag indicates a last sample the reverse playback group of data samples;

a flag that indicates part of a video section in forward or reverse scan mode;

a flag that indicates a location that specifies that the data sample should be truncated at this point; and

a flag that indicates a location before which data samples should be discarded.

55. The system of claim 47, wherein the second interface comprises a first rate change method and a second rate change method, wherein the first rate change method requests a rate change insertion at a location specified with respect to media content that provides the data samples, the first rate change method providing information that enables a source component to map a media content location into a timestamp that is attached to output data samples, wherein the second rate change method requires the source component to query downstream filter graph components for a current renderer position that is capable of performing the rate change.

56. The system of claim 47, wherein the second interface comprises a first rate change method and a second rate change method, wherein the first rate change method requests a rate change insertion at a location specified with respect to media content that provides the data samples, the first rate change method providing information that enables a source component to map a media content location into a timestamp that is attached to output data samples, wherein the second rate change method requires the source component to query downstream filter graph components for a current renderer position that is capable of performing the rate change, wherein the first and second rate change methods return a timestamp of the rate change.

57. The system of claim 47, wherein the third interface comprises a first method that requests a rate change insertion at a next available location and returns a location on which the rate change is inserted.

58. The system of claim 47, wherein the third interface comprises a first method that requests a rate change insertion at a next available location and returns a location on which the rate change is inserted, wherein the location that is returned comprises the timestamp associated with the data sample on which the rate change was scheduled.

59. The system of claim 47, wherein the third interface comprises a first method that requests a rate change insertion at a next available location and returns a location on which the rate change is inserted, wherein the third interface comprises a second method that requests a rate change insertion at a specified location.

60. The system of claim 47, wherein the third interface comprises a first method that requests a rate change insertion at a next available location and returns a location on which the rate change is inserted, wherein the third interface comprises a second method that requests a rate change insertion at a specified location, wherein the specified location is specified as a timestamp associated with a data sample on which the rate change is to be inserted.

61. The system of claim 47, wherein the third interface comprises a first method that requests a rate change insertion at a next available location and returns a location on which the rate change is inserted, and a second method that requests a data truncation.

62. The system of claim 47, wherein the third interface comprises a first method that requests a rate change insertion at a next available location and returns a location on which the rate change is inserted, and a second method that requests a data skip from a start of a data sample.

63. The system of claim 47, wherein the third interface comprises a first method that requests a rate change insertion at a next available location and returns a location on which the rate change is inserted, and a second method that queries a maximum forward and reverse decode rate.

64. The system of claim 47, wherein the third interface comprises a first method that requests a rate change insertion at a next available location and returns a location on which the rate change is inserted, and a second method that queries a furthest stream time that has not yet been processed.

65. The system of claim 47, wherein the third interface comprises a first method that requests a rate change insertion at a next available location and returns a location on which the rate change is inserted, and a second method that returns a renderer'"'"'s location in source timestamps.

66. A computing device embodying the system of claim 47.

67. A system comprising:

one or more computer-readable media;

a first interface embodied on the computer-readable media and which maintains information that indicates whether a rate change has been scheduled for a data sample and, when a rate change is scheduled, an indication of the rate change, wherein the first interface indicates a rate change by using a timestamp associated with a data sample, and wherein the first interface comprises multiple flags that are associated with data samples and pertain to how a data sample is to be handled by a filter graph, the flags comprising;

a flag that indicates a first data sample in a reverse playback group of data samples;

a flag indicates a last sample the reverse playback group of data samples;

a flag that indicates part of a video section in forward or reverse scan mode;

a flag that indicates a location that specifies that the data sample should be truncated at this point; and

a flag that indicates a location before which data samples should be discarded;

a second interface embodied on the computer-readable media and which is configured to be used by an application to communicate a rate change to a media source, wherein the second interface comprises a first rate change method and a second rate change method, wherein the first rate change method requests a rate change insertion at a location specified with respect to media content that provides the data samples, the first rate change method providing information that enables a source component to map a media content location into a timestamp that is attached to output data samples, and wherein the second rate change method requires the source component to query downstream filter graph components for a current renderer position that is capable of performing the rate change; and

a third interface embodied on the computer-readable media and which enables individual components of a filter graph to communicate rate changes with one another, wherein the third interface comprises;

a first method that requests a rate change insertion at a next available location and returns a location on which the rate change is inserted;

a second method that requests a rate change insertion at a specified location;

a third method that requests a data truncation;

a fourth method that requests a data skip from a start of a data sample;

a fifth method that queries a maximum forward and reverse decode rate;

a sixth method that queries a furthest stream time that has not yet been processed; and

a seventh method that returns a renderer'"'"'s location in source timestamps.

68. A computing device embodying the system of claim 67.

69. A method comprising:

receiving a rate change command for a multimedia stream having unaligned component streams;

selecting a starting point on a primary stream of the multimedia stream;

inserting a skip point and a truncation point relative to one or more of the component streams;

seeking backwards on one or more component streams, other than the primary stream, effective to locate a clean point on the one or more component streams;

processing the one or more component streams by discarding data before the skip point, and truncating data after the truncation point effective to generate a logical block of data across multiple component streams; and

implementing the rate change on the logical block of data.

70. The method of claim 69, wherein the starting point coincides with a clean point on the primary stream.

71. The method of claim 69, wherein the act of inserting a skip point and a truncation point is performed by specifying timestamps associated with data samples relative to which skipping and truncation is to take place.

72. The method of claim 69, wherein the acts of selecting and inserting are performed by a source component that is configured to read multimedia content.

73. A computing device configured to implement the method of claim 69.