Script-based video rendering
First Claim
1. A method executed by a computer system for rendering video content, the method comprising:
- receiving encoded video data in a pre-determined video format;
receiving a script at the computer system, the script comprising a non-compiled series of instructions for decoding the pre-determined video format that are configured to be interpreted by an application program operating on the computer system;
using the application program to interpret the received script by performing the series of instructions comprising the script to decode the encoded video data into decoded video data comprising one or more frame images without use of a previously compiled program module for decoding video data in the pre-determined video format that is installed prior to the interpreting and without compiling the series of instructions comprising the script prior to the interpreting;
wherein using the application program to interpret the script comprises decoding encoded audio data accessible to the computer system into decoded audio data;
wherein using the application program to interpret the script comprises;
rendering the decoded video data by displaying the one or more frame images on the computer system without use of a previously compiled program module for rendering the decoded video data that is installed prior to the interpreting and without compiling the series of instructions comprising the script prior to the interpreting; and
rendering the decoded audio data by playing back the decoded audio data;
wherein the video data and audio data are associated with one another as parts of a video content item, wherein the one or more frame images comprise a plurality of frame images and wherein using the application program to interpret the script comprises ascertaining an audio playback time reference associated with the playback of the decoded audio data; and
synchronizing the displaying of the frame images with the playback of the decoded audio data based on the audio playback time reference;
wherein synchronizing the displaying of the frame images with the playback of the decoded audio data comprises at least one of;
advancing the display of frame images out of order by skipping the display of one or more frame images based on a determination that the rendering of the video data is behind the audio playback time reference by a first threshold amount of time; and
causing one or more frame images to be displayed for time periods longer than those dictated by a frame rate of the video data based on a determination that the rendering of the video data is ahead of the audio playback time reference by a second threshold amount of time.
1 Assignment
0 Petitions
Accused Products
Abstract
Systems and methods are provided for cross-platform rendering of video content on a user-computing platform that is one type of a plurality of different user-computing platform types. A script is transmitted to the user-computing platform and is interpreted by an application program compiled to operate on any one of the plurality of user-computing platform types. The script is configured to cause the script to be interpreted by the application program operating on the user-computing platform to: decode encoded video data received by the user-computing platform into decoded video data comprising a plurality of frame images; and render the decoded video data by displaying the frame images. Rendering the video data by displaying the frame images comprises alpha-blending at least one pair of frame images together.
-
Citations
56 Claims
-
1. A method executed by a computer system for rendering video content, the method comprising:
-
receiving encoded video data in a pre-determined video format; receiving a script at the computer system, the script comprising a non-compiled series of instructions for decoding the pre-determined video format that are configured to be interpreted by an application program operating on the computer system; using the application program to interpret the received script by performing the series of instructions comprising the script to decode the encoded video data into decoded video data comprising one or more frame images without use of a previously compiled program module for decoding video data in the pre-determined video format that is installed prior to the interpreting and without compiling the series of instructions comprising the script prior to the interpreting; wherein using the application program to interpret the script comprises decoding encoded audio data accessible to the computer system into decoded audio data; wherein using the application program to interpret the script comprises;
rendering the decoded video data by displaying the one or more frame images on the computer system without use of a previously compiled program module for rendering the decoded video data that is installed prior to the interpreting and without compiling the series of instructions comprising the script prior to the interpreting; and
rendering the decoded audio data by playing back the decoded audio data;wherein the video data and audio data are associated with one another as parts of a video content item, wherein the one or more frame images comprise a plurality of frame images and wherein using the application program to interpret the script comprises ascertaining an audio playback time reference associated with the playback of the decoded audio data; and
synchronizing the displaying of the frame images with the playback of the decoded audio data based on the audio playback time reference;wherein synchronizing the displaying of the frame images with the playback of the decoded audio data comprises at least one of;
advancing the display of frame images out of order by skipping the display of one or more frame images based on a determination that the rendering of the video data is behind the audio playback time reference by a first threshold amount of time; and
causing one or more frame images to be displayed for time periods longer than those dictated by a frame rate of the video data based on a determination that the rendering of the video data is ahead of the audio playback time reference by a second threshold amount of time. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27)
-
-
28. A method executed by a computer system for rendering video content embodied by encoded video data received by a user-computing platform that is one type of a plurality of different user-computing platform types, the method comprising:
-
transmitting a script to the user-computing platform, the script configured to be interpreted by an application program operating on any one of the plurality of user-computing platform types and further configured to cause the script to be interpreted by the application program to; decode the encoded video data into decoded video data comprising a plurality of frame images; and render the decoded video data by displaying the frame images; wherein rendering the decoded video data by displaying the frame images comprises blending at least one pair of frame images together; wherein the video content is also embodied by encoded audio data received by the user-computing platform and wherein the script is further configured to cause the script to be interpreted by the application program to decode the encoded audio data into decoded audio data; wherein the script is further configured to cause the script to be interpreted by the application program to render the decoded audio data by playing back the decoded audio data; wherein the video data and the audio data are associated with one another as parts of a video content item, and wherein the script is further configured to cause the script to be interpreted by the application program to; ascertain an audio playback time reference associated with the playback of the decoded audio data; and directly synchronize the displaying of the frame images with the playback of the decoded audio data based on the audio playback time reference; wherein synchronizing the displaying of the frame images with the playback of the decoded audio data comprises determining that the frame image associated with the current frame index is not ready to be displayed and, in dependence on such determination, initiating a drop frame action, the drop frame action comprising one or more of; recording an indication that the frame image associated with the current frame index was not ready for rendering; causing the playback of the decoded audio data to be paused or slowed; causing a rate of decoding of the encoded video data to be increased; causing a switch to video data having a lower frame rate; and causing a switch to effective down-sampling of the video data by decoding a subset of the image frames. - View Dependent Claims (29, 30, 31, 32, 33, 34, 35, 36, 37, 38)
-
-
39. A method executed by a computer system for rendering video content on a user-computing platform that is one type of a plurality of different user-computing platform types, the method comprising:
-
transmitting a script to the user-computing platform, the script comprising a non-compiled series of instructions for decoding encoded video data accessible to the user-computing platform into decoded video data comprising one or more frame images; wherein transmitting the script causes the script to be interpreted by an application program operating on the user-computing platform to interpret the script by performing the series of instructions comprising the script to decode the encoded video data into the decoded video data without use of a program module for decoding the encoded video data that is installed prior to the interpreting and without compiling the series of instructions comprising the script prior to the interpreting; wherein the method further comprises; transmitting the encoded video data to the user-computing platform; and encoding the video content into the encoded video data prior to transmitting the encoded video data to the user-computing platform and wherein encoding the video content comprises; for at least a first one of the frame images provided in a first file format;
analyzing content of the first one of frame images;
selecting one among a plurality of secondary image file formats based on the analysis of the content of the first one of the frame images; and
if the first file format differs from the selected one of the secondary image file formats, converting the image file format of the first one of the frame images into the selected one of the secondary file formats; andfor at least a second one of frame images provided in a second file format;
analyzing content of the second one of frame images;
selecting a different one among the plurality of secondary image file formats based on the analysis of the content of the second one of the frame images; and
if the second file format differs from the selected different one of the secondary image file formats, converting the image file format of the first one of the frame images into the selected different one of the secondary file formats;wherein each of the plurality of secondary image file formats is capable of being rendered by the application program without use of a previously compiled program module for rendering the secondary image file format that is installed prior to the interpreting and without compiling the series of instructions comprising the script prior to the interpreting. - View Dependent Claims (40, 41, 42, 43, 44, 55, 56)
-
-
45. A method executed by a computer system for rendering video content on a user-computing platform that is one type of a plurality of different user-computing platform types, the method comprising:
-
transmitting a script to the user-computing platform, the script comprising a non-compiled series of instructions for decoding encoded video data accessible to the user-computing platform into decoded video data comprising one or more frame images; wherein transmitting the script causes the script to be interpreted by an application program operating on the user-computing platform to interpret the script by performing the series of instructions comprising the script to decode the encoded video data into the decoded video data without use of a program module for decoding the encoded video data that is installed prior to the interpreting and without compiling the series of instructions comprising the script prior to the interpreting; wherein the method further comprises; transmitting the encoded video data to the user-computing platform; and encoding the video content into the encoded video data prior to transmitting the encoded video data to the user-computing platform and wherein encoding the video content comprises;
for each of the plurality of frame images;
encoding the frame image in a content-transfer format; and
padding the content-transfer encoded frame image to a particular size with padding data unrelated to the frame image. - View Dependent Claims (46, 47, 48)
-
-
49. A method executed by a computer system for rendering video content embodied by encoded video data received by a user-computing platform that is one type of a plurality of different user-computing platform types, the method comprising:
-
transmitting a script to the user-computing platform, the script configured to be interpreted by an application program operating on any one of the plurality of user-computing platform types and further configured to cause the script to be interpreted by the application program to; decode the encoded video data into decoded video data comprising a plurality of frame images; and render the decoded video data by displaying the frame images; wherein rendering the decoded video data by displaying the frame images comprises blending at least one pair of frame images together; wherein rendering the decoded video data by displaying the frame images comprises, at each of one or more times between displaying a first one of the frame images and displaying a second one of the frame images, displaying a blended image comprising a blend of the first one of the frame images and the second one of the frame images; wherein, at each of the one or more times between displaying the first one of the frame images and displaying the second one of the frame images, displaying the alpha-blended image comprises blending the second one of the frame images onto a preceding displayed image using a blending coefficient α
provided by;
α
=[β
+1−
mod(f#,β
)]−
1where; β
is a down-sampling factor equal to a number of the one or more times +1;f# is an index referring to a number of a current one of the one or more times; and mod(x,y) is a modulo operator which returns the whole number remainder of x divided by y.
-
-
50. A method executed by a computer system for rendering video content embodied by encoded video data received by a user-computing platform that is one type of a plurality of different user-computing platform types, the method comprising:
-
transmitting a script to the user-computing platform, the script configured to be interpreted by an application program operating on any one of the plurality of user-computing platform types and further configured to cause the script to be interpreted by the application program to; decode the encoded video data into decoded video data comprising a plurality of frame images; and render the decoded video data by displaying the frame images; wherein rendering the decoded video data by displaying the frame images comprises blending at least one pair of frame images together; wherein the video content is also embodied by encoded audio data received by the user-computing platform and wherein the script is further configured to cause the script to be interpreted by the application program to decode the encoded audio data into decoded audio data; wherein the script is further configured to cause the script to be interpreted by the application program to render the decoded audio data by playing back the decoded audio data; wherein the video data and the audio data are associated with one another as parts of a video content item, and wherein the script is further configured to cause the script to be interpreted by the application program to; ascertain an audio playback time reference associated with the playback of the decoded audio data; and directly synchronize the displaying of the frame images with the playback of the decoded audio data based on the audio playback time reference; wherein synchronizing the displaying of the frame images with the playback of the decoded audio data comprises at least one of; advancing the display of frame images out of order by skipping the display of one or more frame images based on a determination that the rendering of the video data is behind the audio playback time reference by a first threshold amount of time; and causing one or more frame images to be displayed for time periods longer than those dictated by a frame rate of the video data based on a determination that the rendering of the video data is ahead of the audio playback time reference by a second threshold amount of time.
-
-
51. A method executed by a computer system for rendering video content embodied by encoded video data received by a user-computing platform that is one type of a plurality of different user-computing platform types, the method comprising:
-
transmitting a script to the user-computing platform, the script configured to be interpreted by an application program operating on any one of the plurality of user-computing platform types and further configured to cause the script to be interpreted by the application program to; decode the encoded video data into decoded video data comprising a plurality of frame images; and render the decoded video data by displaying the frame images; wherein rendering the decoded video data by displaying the frame images comprises blending at least one pair of frame images together; wherein the video content is also embodied by encoded audio data received by the user-computing platform and wherein the script is further configured to cause the script to be interpreted by the application program to decode the encoded audio data into decoded audio data; wherein the script is further configured to cause the script to be interpreted by the application program to render the decoded audio data by playing back the decoded audio data; wherein the video data and the audio data are associated with one another as parts of a video content item, and wherein the script is further configured to cause the script to be interpreted by the application program to; ascertain an audio playback time reference associated with the playback of the decoded audio data; and directly synchronize the displaying of the frame images with the playback of the decoded audio data based on the audio playback time reference; wherein synchronizing the displaying of the frame images with the playback of the decoded audio data comprises; using the audio playback time reference to determine a current frame index; and if the frame image associated with the current frame index is ready to be displayed, causing the frame image associated with the current frame index to be displayed. - View Dependent Claims (52)
-
-
53. A method executed by a computer system for rendering video content embodied by encoded video data received by a user-computing platform that is one type of a plurality of different user-computing platform types, the method comprising:
-
transmitting a script to the user-computing platform, the script configured to be interpreted by an application program operating on any one of the plurality of user-computing platform types and further configured to cause the script to be interpreted by the application program to; decode the encoded video data into decoded video data comprising a plurality of frame images; and render the decoded video data by displaying the frame images; wherein rendering the decoded video data by displaying the frame images comprises blending at least one pair of frame images together; wherein the video content is also embodied by encoded audio data received by the user-computing platform and wherein the script is further configured to cause the script to be interpreted by the application program to decode the encoded audio data into decoded audio data and to render the decoded audio data by playing back the decoded audio data; wherein the script is further configured to cause the script to be interpreted by the application program to simulate a multi-threaded execution environment by; iteratively scheduling and performing a first simulated thread which comprises directly decoding the encoded video data into the decoded video data; and iteratively scheduling and performing a second simulated thread which comprises rendering the video data by displaying the frame images; iteratively scheduling and performing a third simulated thread which comprises directly decoding the encoded audio data into the decoded audio data; and iteratively scheduling and performing a fourth simulated thread which comprises implementing an audio renderer interface which supplies the decoded audio data to an audio renderer; wherein performing the first simulated thread comprises; dividing the first simulated thread into a plurality of sub-components; between the performance of pairs of sub-components, making one or more logical inquiries into one or more corresponding conditions and, in dependence on the one or more corresponding conditions, temporarily interrupting the first simulated thread to select and perform an iteration of one of; the second simulated thread; and another simulated thread which involves implementing an audio renderer interface which provides the decoded audio data to an audio renderer.
-
-
54. A method executed by a computer system for rendering video content embodied by encoded video data received by a user-computing platform that is one type of a plurality of different user-computing platform types, the method comprising:
-
transmitting a script to the user-computing platform, the script configured to be interpreted by an application program operating on any one of the plurality of user-computing platform types and further configured to cause the script to be interpreted by the application program to; decode the encoded video data into decoded video data comprising a plurality of frame images; and render the decoded video data by displaying the frame images; and transmitting the encoded video data to the user-computing platform; wherein rendering the decoded video data by displaying the frame images comprises blending at least one pair of frame images together; wherein the method comprises encoding the video content into the encoded video data prior to transmitting the encoded video data to the user-computing platform and wherein encoding the video content comprises; for at least a first one of the frame images provided in a first file format; analyzing content of the first one of the frame images; selecting one among a plurality of secondary image file formats based on the analysis of the content of the first one of the frame images; and if the first file format differs from the selected one of the secondary image file formats, converting the image file format of the first one of the frame images into the selected one of the secondary file formats; and for at least a second one of the frame images provided in a second file format; analyzing content of the second one of the frame images; selecting a different one among the plurality of secondary image file formats based on the analysis of the content of the second one of the frame images; and if the second file format differs from the selected different one of the secondary image file formats, converting the image file format of the second one of the frame images into the selected different one of the secondary file formats; wherein each of the plurality of secondary image file formats is capable of being natively rendered by the application program.
-
Specification