TECHNIQUES FOR MORE EFFICIENT GENERATION OF XML EVENTS FROM XML DATA SOURCES

• US 20090125693A1
• Filed: 11/09/2007
• Published: 05/14/2009
• Est. Priority Date: 11/09/2007
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Zhang, Ning, Idicula, Sam, Chandrasekar, Sivasankaran, Agarwal, Nipun
• Granted Patent: US 8,543,898 B2
• Time in Patent Office: Days
• US Class Current: 711/170
• CPC Class Codes: G06F 16/8373 Query execution
• CPC Class Codes: G06F 40/143 Markup, e.g. Standard Generalized Markup Language [SGML] or Document Type Definition [DTD]
• CPC Class Codes: G06F 40/221 Parsing markup language streams streaming G06F40/149

Abstract

One may increase the efficiency of an XML event-generating process by reducing the number of requests to allocate or deallocate system memory. Such reduction may occur as the result of pre-allocating a memory chunk of sufficient size to contain all of the memory buffers required by a particular event-generating process. Instead of allocating new memory chunks for new memory buffers, an application may store any required buffers within the pre-allocated memory chunk. A sufficient memory size may be estimated by performing the event-generating process on a training set of XML documents. Also, an application may re-use buffers during the process or between different iterations of the process, thus avoiding the need to deallocate and reallocate memory that is essentially being used for the same purpose.

Claims

1. A computer-implemented method comprising the steps of:

determining a default memory size for an XML event-generating process;

requesting, from a memory manager, allocation of a memory chunk of the default memory size for use with the XML event-generating process;

processing the XML data source using the XML event-generating process, thereby generating one or more XML events;

wherein processing the XML data source requires the use of a plurality of memory buffers;

wherein the XML event-generating process stores the plurality of memory buffers in the memory chunk; and

requesting that the memory manager deallocate the memory chunk.

2. The method of claim 1, wherein the XML data source is a binary-encoded XML document, wherein processing the XML data source further comprises decoding the binary-encoded XML document.

3. The method of claim 2, wherein processing the XML data source occurs in response to a request from an application for textual form of the binary-encoded XML document.

4. The method of claim 1, wherein no other memory buffer is used during the processing of the XML data source.

5. The method of claim 1, wherein the event-generating process is associated with a state machine representing an XPath expression, wherein the one or more XML events are used as input for the state machine.

6. The method of claim 5, wherein requesting a memory chunk occurs during compilation of the state machine from the XPath expression.

7. The method of claim 1, wherein determining a default memory size comprises the steps of:

executing the event-generating process in a plurality of iterations on a training set of XML data sources;

while executing the event-generating process in a plurality of iterations, monitoring the memory usage of all memory buffers used during each iteration; and

wherein the determination of the default memory size is based at least on the average of the maximum memory usage during each iteration.

8. The method of claim 7, wherein the default memory size is the average of the maximum memory usage during each iteration plus three times the standard deviation in the maximum memory usage over the plurality of iterations.

9. The method of claim 7, wherein the training set comprises XML documents based on an XML schema, wherein the XML data source is also based on the XML schema.

10. The method of claim 1, wherein the memory manager is provided by a database system.

11. The method of claim 1, wherein the memory manager is provided by an operating system.

20. A computer-readable storage medium storing instructions for performing the method recited in claim 1.

21. A computer-readable storage medium storing instructions for performing the method recited in claim 2.

22. A computer-readable storage medium storing instructions for performing the method recited in claim 5.

23. A computer-readable storage medium storing instructions for performing the method recited in claim 7.

12. A computer-implemented method comprising the steps of:

determining a default memory size for an XML event-generating process;

requesting, from a memory manager, allocation of a memory chunk of the default memory size for use with the XML event-generating process;

processing the plurality of XML data sources using the XML event-generating process, thereby generating one or more XML events;

wherein, for each particular XML data source, processing the particular XML data source requires the use of at least one memory buffer;

wherein, for each particular XML data source, the XML event-generating process stores the at least one memory buffer in the memory chunk;

requesting that the memory manager deallocate the memory chunk.

13. The method of claim 12, wherein each of the plurality of XML data sources pertains to the same XML schema.

14. The method of claim 12, wherein each of the plurality of XML data sources matches a particular query.

15. The method of claim 12, wherein processing the plurality of XML data sources further comprises:

processing a first XML data source, wherein processing the first XML data source comprises creating a first memory buffer stored in the memory chunk; and

processing a second XML data source, wherein processing the second XML data source comprises using the first memory buffer.

16. The method of claim 12, wherein processing the plurality of XML data sources further comprises processing a first XML data source, wherein processing the first XML data source comprises determining that the memory chunk has no room to store a particular memory buffer and, in response, requesting from the memory manager allocation of a temporary memory chunk to store the particular memory buffer.

24. A computer-readable storage medium storing instructions for performing the method recited in claim 12.

17. A computer-implemented method comprising the steps of:

(a) processing a first data source in the plurality of XML data sources using the XML event-generating process, thereby generating an XML event,

wherein the step of processing comprises allocating a memory chunk from a memory manager to store one or more memory buffers;

(b) processing at least one subsequent data source in the plurality of XML data sources using the XML event-generating process, thereby generating at least one XML event, wherein for each of the at least one subsequent XML data sources, the step of processing comprises;

storing one or more memory buffers in the memory chunk;

expanding the memory chunk to fit an additional one or more memory buffers;

(c) further processing at least one subsequent data source in the plurality of XML data sources using the XML event-generating process, thereby generating at least one XML event,

wherein for each of the at least one subsequent data sources, the step of further processing comprises storing one or more memory buffers in the memory chunk without expanding the memory chunk; and

(d) deallocating the memory chunk.

18. The method of claim 17, wherein steps (b) and (c) are performed concurrently.

19. The method of claim 17:

wherein the plurality of XML data sources is a training subset of a second plurality of XML data sources, and

further comprising, prior to step (d), the step of processing each remaining data source in the second plurality of XML data sources using the XML event-generating process, thereby generating at least one XML event,

wherein for each remaining data sources, the step of processing comprises storing one or more memory buffers in the memory chunk without expanding the memory chunk.

25. A computer-readable storage medium storing instructions for performing the method recited in claim 17.