Methods and systems for synchronizing visualizations with audio streams
First Claim
1. One or more computer-readable media having computer-readable instructions thereon which, when executed by a computer, implement a system comprising:
- one or more audio sources configured to provide audio samples that are to be rendered by a media player;
an audio sample pre-processor communicatively linked with the one or more audio sources and configured to receive and pre-process audio samples before the samples are rendered, the pre-processing providing characterizing data associated with each sample, wherein the characterizing data is derived from the audio samples;
one or more effects configured to receive the characterizing data and use the characterizing data to render a visualization that is synchronized with an audio sample that is being rendered by the media player; and
multiple data structures configured to hold the characterizing data, each data structure being associated with an audio sample.
1 Assignment
0 Petitions
Accused Products
Abstract
Methods and systems are described that assist media players in rendering visualizations and synchronizing those visualizations with audio samples. In one embodiment, visualizations are synchronized with an audio stream using a technique that builds and maintains various data structures. Each data structure can maintain data that is associated with a particular pre-processed audio sample. The maintained data can include a timestamp that is associated with a time when the audio sample is to be rendered. The maintained data can also include various characteristic data that is associated with the audio stream. When a particular audio sample is being rendered, its timestamp is used to locate a data structure having characteristic data. The characteristic data is then used in a visualization rendering process to render a visualization.
-
Citations
33 Claims
-
1. One or more computer-readable media having computer-readable instructions thereon which, when executed by a computer, implement a system comprising:
-
one or more audio sources configured to provide audio samples that are to be rendered by a media player;
an audio sample pre-processor communicatively linked with the one or more audio sources and configured to receive and pre-process audio samples before the samples are rendered, the pre-processing providing characterizing data associated with each sample, wherein the characterizing data is derived from the audio samples;
one or more effects configured to receive the characterizing data and use the characterizing data to render a visualization that is synchronized with an audio sample that is being rendered by the media player; and
multiple data structures configured to hold the characterizing data, each data structure being associated with an audio sample. - View Dependent Claims (2, 3, 4, 5, 6, 7)
-
-
8. One or more computer-readable media having computer-readable instructions thereon which, when executed by a computer, implement a system comprising:
-
an audio sample pre-processor configured to receive and pre-process audio samples before the samples are rendered by the media player, the pre-processing providing frequency data associated with each sample, wherein the characterizing data is derived from the audio samples;
one or more effects configured to receive the frequency data and use the frequency data to render a visualization that is synchronized with an audio sample that is being rendered by the media player; and
multiple data structures configured to hold the frequency data, each data structure being associated with an audio sample. - View Dependent Claims (9, 10)
-
-
11. One or more computer-readable media having computer-readable instructions thereon which, when executed by a computer, implement a system comprising:
-
an audio sample pre-processor configured to receive and preprocess audio samples before the samples are rendered by a renderer that comprises part of a media player, the audio sample preprocessor preprocessing the samples to provide characterizing data derived from each sample, the characterizing data comprising a timestamp associated with each audio sample, the timestamp being assigned in accordance with when the audio sample is calculated to be rendered by the renderer;
multiple data structures configured to hold the characterizing data, each data structure being associated with an audio sample;
an audio rendering object configured to call the audio sample pre-processor to ascertain the characterizing data associated with an audio sample that is currently being rendered by the renderer;
the audio sample pre-processor being configured to ascertain said characterizing data by querying the renderer for a time associated with the currently-rendered audio sample, and then using said time to identify a data structure having a timestamp that is nearest in value to said time; and
one or more effects configured to receive characterizing data that is associated with the data structure having the timestamp that is nearest in value to said time, and use the characterizing data to render a visualization that is synchronized with the audio sample that is being rendered by the renderer. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18)
-
-
19. A system for processing audio samples comprising:
-
means for providing a timestamp module for assigning timestamps to audio samples that are to be rendered by a media player renderer;
means for providing a spectrum analyzer for processing the audio samples to derive frequency data from the audio samples;
means for providing multiple data structures each of which being associated with an audio sample, the data structures each containing timestamp data and frequency data for its associated audio sample; and
the system being configured to use the timestamp data to ascertain a data structure associated with an audio sample that is currently being rendered by the media player renderer and provide the frequency data associated with that audio sample so that the frequency data can be used to render a visualization associated with that audio sample. - View Dependent Claims (20)
-
-
21. One or more computer-readable media having computer-readable instructions thereon which, when executed by a computer, implement a method comprising:
-
receiving multiple audio samples;
pre-processing the audio samples before they are rendered by a media player renderer, the pre-processing deriving characterizing data from each sample;
determining when an audio sample is being rendered by the media player renderer; and
responsive to said determining, using the characterizing data that is associated with the audio sample that is being rendered to provide a visualization. - View Dependent Claims (22, 23, 24, 25, 26)
-
-
27. A system comprising:
-
means for receiving multiple audio samples;
means for pre-processing the audio samples before they are rendered by a media player renderer, the pre-processing comprising at least (1) using a Fast Fourier Transform to derive frequency data from the samples, and (2) associating a timestamp with each sample;
means for maintaining frequency data and a timestamp for each sample in a data structure;
means for determining when an audio sample is being rendered by a media player renderer by;
ascertaining a time associated with a currently-rendered sample; and
selecting a data structure having a timestamp that is nearest the time; and
means for providing characterizing data associated with the selected data structure to a component configured to provide the visualization.
-
-
28. A system comprising:
-
means for calling an audio sample pre-processor for characterizing data that has been derived from an associated audio sample that is currently being rendered by a media player renderer;
means for calling the media player renderer for a time associated with a currently-rendered audio sample;
means for using the time to select a data structure containing characterizing data associated with the currently-rendered audio sample; and
means for providing the characterizing data to a component for rendering a visualization. - View Dependent Claims (29, 30)
-
-
31. A system comprising:
-
means for defining a frame rate at which visualization frames are to be rendered;
means for setting a threshold associated with an amount of time that is to be spent rendering a visualization frame;
means for monitoring the time associated with rendering individual visualization frames;
means for determining whether a visualization frame rendering time exceeds the threshold; and
means for providing an effective frame rate for rendering visualization frames that is longer than the defined frame rate if the determined visualization frame rendering time exceeds the threshold. - View Dependent Claims (32, 33)
-
Specification