Event-level parallel methods and apparatus for XML parsing
First Claim
1. A computer-implemented method for parsing XML data, the method comprising:
- partitioning, by an events partitioning module of a computing device, the XML data into a plurality of XML chunks having a plurality of XML events contained therein, wherein partitioning includes determining a type of event associated with individual XML events of the plurality of XML events and ignoring character data contained within one or more XML events based, at least in part, on the type of event to prevent identification of character data contained within the one or more XML events from being identified as an XML event;
parsing, by a plurality of instances of an events parsing module of the computing device, the plurality of chunks in parallel into sub-event streams, wherein parsing includes creating structure metadata to identify unresolved items in the sub-event streams to avoid a parsing error based on the unresolved items, wherein the unresolved items include one or more of an identity of an unresolved start element, an identity of an unresolved end element, or an identity of an unresolved prefix; and
generating, by a post processing module of the computing device, a result event stream for the XML data from the sub-event streams, wherein generating the result event stream includes resolving an unresolved end element identified in the structure metadata with a preceding unresolved start element identified in the structure metadata or resolving an unresolved prefix identified in the structure metadata with a namespace of a preceding start element to avert the need to reparse the XML chunks that produced the unresolved item.
1 Assignment
0 Petitions
Accused Products
Abstract
Embodiments of techniques and systems for parallel XML parsing are described. An event-level XML parser may include a lightweight events partitioning stage, parallel events parsing stages, and a post-processing stage. The events partition may pick out event boundaries using single-instruction, multiple-data instructions to find occurrences of the “<” character, marking event boundaries. Subsequent checking may be performed to help identify other event boundaries, as well as non-boundary instances of the “<” character. During events parsing, unresolved items, such as namespace resolution or matching of start and end elements, may be recorded in structure metadata. This structure metadata may be used during the subsequent post-processing to perform a check of the XML data. If the XML data is well-formed, individual sub-event streams formed by the events parsing processes may be assembled into a flat result event stream structure. Other embodiments may be described and claimed.
-
Citations
17 Claims
-
1. A computer-implemented method for parsing XML data, the method comprising:
-
partitioning, by an events partitioning module of a computing device, the XML data into a plurality of XML chunks having a plurality of XML events contained therein, wherein partitioning includes determining a type of event associated with individual XML events of the plurality of XML events and ignoring character data contained within one or more XML events based, at least in part, on the type of event to prevent identification of character data contained within the one or more XML events from being identified as an XML event; parsing, by a plurality of instances of an events parsing module of the computing device, the plurality of chunks in parallel into sub-event streams, wherein parsing includes creating structure metadata to identify unresolved items in the sub-event streams to avoid a parsing error based on the unresolved items, wherein the unresolved items include one or more of an identity of an unresolved start element, an identity of an unresolved end element, or an identity of an unresolved prefix; and generating, by a post processing module of the computing device, a result event stream for the XML data from the sub-event streams, wherein generating the result event stream includes resolving an unresolved end element identified in the structure metadata with a preceding unresolved start element identified in the structure metadata or resolving an unresolved prefix identified in the structure metadata with a namespace of a preceding start element to avert the need to reparse the XML chunks that produced the unresolved item. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
-
-
10. A system comprising:
-
storage memory configured to store an XML document; one or more processors having a plurality of processor cores, coupled to the storage memory; an events partitioning module which, upon execution by the processor, causes the processor to partition the XML document into a plurality of XML chunks having a plurality of XML events contained therein, wherein partition includes determination of a type of event associated with individual XML events of the plurality of XML events and exclusion of character data contained within one or more XML events based, at least in part, on the type of event to prevent identification of character data contained within the one or more XML events from being identified as an XML event; a plurality of instances of an events parsing module which, upon execution by one or more of the plurality of processor cores, cause the one or more of the plurality of processor cores to perform, in parallel, events parsing of the plurality of XML chunks to produce respective sub-event streams and structure metadata, the structure metadata identifying unresolved items in the sub-event streams to avoid a parsing error based on the unresolved items, wherein the unresolved items include one or more of an identity of an unresolved start element, an identity of an unresolved end element, or an identity of an unresolved prefix; and a post-processing module which, when executed by at least one processor of the one or more processors, causes the at least one processor to perform post-processing on the sub-event streams to produce a result event stream, wherein to produce the result event stream includes resolution of an unresolved end element identified in the structure metadata with a preceding unresolved start element identified in the structure metadata or resolution of an unresolved prefix identified in the structure metadata with a namespace of a preceding start element to avert the need to reparse the XML chunks that produced the unresolved item. - View Dependent Claims (11, 12, 16)
-
-
13. One or more non-transitory computer-readable storage media containing instructions which, upon execution by a processor having a plurality of cores, cause the processor to:
-
partition an XML document into a plurality of XML chunks containing a plurality of XML events, wherein partition includes determination of a type of event associated with individual XML events of the plurality of XML events and exclusion of character data contained within one or more XML events based, at least in part, on the type of event to prevent identification of character data contained within the one or more XML events from being identified as an XML event; perform, in parallel on respective cores of the processor, events parsing of XML chunks to produce respective sub-event streams and structure metadata to identify unresolved items in the sub-event streams and avoid a parsing error based on the unresolved items, wherein the unresolved items include one or more of an identity of an unresolved start element, an identity of an unresolved end element, or an identity of an unresolved prefix; and perform post-processing on the sub-event streams to produce a result event stream, wherein to produce the result event stream includes resolution of an unresolved end element identified in the structure metadata with a preceding unresolved start element identified in the structure metadata or resolving an unresolved prefix identified in the structure metadata with a namespace of a preceding start element to avert the need to reparse the XML chunks that produced the unresolved item. - View Dependent Claims (14, 15, 17)
-
Specification