Buffer objects for web-based configurable pipeline media processing

• US 8,838,680 B1
• Filed: 08/10/2011
• Issued: 09/16/2014
• Est. Priority Date: 02/08/2011
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Jia, Wei
• Primary Examiner(s): YI, DAVID
• Application Number:
  US13/206,986
• Time in Patent Office: 1,133 Days
• Field of Search: 709/203, 709/212, 711/154
• US Class Current: 709/203
• CPC Class Codes: G06F 9/54 Interprogram communication

Abstract

An apparatus and method relating to buffer objects for web-based media processing are disclosed. The disclosed embodiments include a web browser implemented on a computing device. The web browser includes a web application processor for processing a web application that includes instructions to process a media stream using one or more configurable pipelines, each configurable pipeline including a plurality of components connected by data channels using buffer objects, the buffer objects including a data pointer identifying a location in a memory, the data pointer having an associated data size representing an amount of memory available at the memory location, an allocation method configured to allocate memory to the buffer object using the data pointer and the data size, and an accessor method configured to enable a component in the configurable pipeline to access data stored in the memory identified by the data pointer and the data size.

Claims

1. An apparatus, comprising:

a processor configured to execute instructions of a web application available from a web browser, the instructions including;

instructions to process a media stream using one or more configurable pipelines, each configurable pipeline including a plurality of executable software components connected by data channels;

instructions to pass each buffer object through at least one of the data channels to a component of the executable software components to permit processing the media stream and to return the buffer object from the component after processing, at least some of the buffer objects including;

a data pointer identifying a location in a memory, the data pointer having an associated data size representing an amount of memory available at the memory location;

allocation instructions to allocate memory to the buffer object using the data pointer and the data size; and

accessor instructions to enable the component in the configurable pipeline to access data stored in the memory identified by the data pointer and the data size, at least some of the buffer objects further including;

a data type property indicative of a type of data stored in the memory identified by the data pointer and the data size, the data type property usable by the component of the executable software components to restrict buffer objects accepted by the component to those having one or more specified data type properties.

2. The apparatus of claim 1, wherein at least some of the buffer objects are configured to be accessible using JavaScript code that is exposed using a Document Object Model interface in the web browser.

3. The apparatus of claim 1, wherein at least some of the buffer objects further include:

a plurality of data type properties, each property representative of an aspect of the data stored in the memory identified by the data pointer and the data size.

4. The apparatus of claim 3, wherein the plurality of data type properties includes a property selected from the group of an audio sampling rate or a color of a video stream.

5. The apparatus of claim 1, wherein at least some of the buffer objects are used by a circular buffer technique in the configurable pipeline whereby data is stored in a buffer object by a first component, the buffer object is transferred to an input port of a second component via a data channel, the data is processed by the second component, and then the buffer object is returned to the first component to be reused.

6. The apparatus of claim 1, wherein at least some of the buffer objects are used by a pass through technique in the configurable pipeline whereby data is stored in a buffer object by a first component, the buffer object is transferred to an input port of a second component via a data channel, the data is processed by the second component, and then the buffer object is used by the second component to output processed data to a third component.

7. The apparatus of claim 1, wherein the data type property is one of:

raw data, compressed stream, uncompressed stream, or generic.

8. The apparatus of claim 7, wherein the accessor instructions provide access to data at a level of granularity dependent on the data type property of the buffer object.

9. The apparatus of claim 8, wherein the level of granularity includes one of pixel, block, line, or frame.

10. An apparatus comprising:

a processor configured to execute instructions of a web application available from a web browser, the instructions including;

instructions to configure a buffer object for use in a configurable pipeline to pass data within data channels between a plurality of executable software components in the configurable pipeline, the configurable pipeline created within the web browser and executed on a computing device, the buffer object comprising;

a data type property;

a data pointer identifying a location in a memory, the data pointer having an associated data size representing an amount of memory available at the memory location;

an allocation method configured to allocate memory to the buffer object using the data pointer and the data size; and

an accessor method configured to enable a component in the configurable pipeline to access data stored in the memory identified by the data pointer and the data size; and

instructions to pass the buffer object through at least one of the data channels to the component when the data type property of the buffer object matches one or more specified data type properties usable by the component and to return the buffer object from the component after processing, the data type property indicative of a type of data stored in the memory identified by the data pointer and the data size.

11. The apparatus of claim 10, wherein the buffer object is configured to be accessible using JavaScript code that is exposed using a Document Object Model interface in the web browser.

12. The apparatus of claim 10, wherein the buffer object further comprises:

a plurality of data type properties, each property representative of an aspect of the data stored in the memory identified by the data pointer and the data size.

13. The apparatus of claim 12, wherein the plurality of data type properties includes a property selected from the group of an audio sampling rate or a color of a video stream.

14. The apparatus of claim 10, wherein the data type property is one of:

raw data, compressed stream, uncompressed stream, or generic.

15. The apparatus of claim 10, wherein the buffer object is used by a circular buffer technique in the configurable pipeline whereby data is stored in the buffer object by a first component, the buffer object is transferred to an input port of a second component via a data channel, the data is processed by the second component, and then the buffer object is returned to the first component to be reused.

16. The apparatus of claim 10, wherein the buffer object is used by a pass through technique in the configurable pipeline whereby data is stored in the buffer object by a first component, the buffer object is transferred to an input port of a second component via a data channel, the data is processed by the second component, and then the buffer object is used by the second component to output processed data to a third component.

17. A method, comprising:

initializing, using a processor, a configurable pipeline based on instructions included within a web application executed by a web browser, the initialization including creating data channels between computer executable components in the configurable pipeline;

creating a plurality of buffer objects to enable the data channels to transfer data between the components of the configurable pipeline, each buffer object including a data pointer identifying a location in a memory, the data pointer having an associated data size representing an amount of memory available at the memory location, and including a data type property indicative of a type of data stored in the memory identified by the data pointer and the data size;

storing data in a buffer object of the plurality of buffer objects by an output port of a first component;

passing the buffer object from the first component to an input port of a second component by way of at least one of the data channels responsive to the data type property of the buffer object matching one or more specified data type properties usable by the second component;

retrieving the data stored in the buffer object by the second component for processing by the second component; and

returning the buffer object through an output port of the second component via at least one of the data channels after processing.

18. The method of claim 17, wherein passing the buffer object from the first component to an input port of a second component by way of at least one of the data channels comprises:

passing the buffer object through a first data channel to the web application executed by the web browser; and

passing the buffer object through a second data channel to the second component from the web application.

19. The method of claim 17, wherein returning the buffer object comprises:

returning the buffer object to the first component from the second component so that it can be reused by the first component.