Method and system for high performance integration, processing and searching of structured and unstructured data
First Claim
Patent Images
1. A method for low latency stream indexing, the method comprising:
- receiving streaming unstructured data into a member of the group consisting of (1) a reconfigurable logic device, (2) a graphics processor unit (GPU), and (3) a chip multi-processor (CMP), the streaming unstructured data comprising a plurality of data objects, wherein the member has a plurality of parallel processing engines deployed thereon;
the parallel processing engines analyzing the data objects while the data objects stream through the member to perform a classification operation that determines classification metadata about the data objects; and
creating an association that is physically represented in memory between the determined classification metadata and the data objects, wherein the creating step comprises generating a classification index about the streaming unstructured data from the classification metadata and the associations between the classification metadata and the corresponding data objects, the classification index for subsequent querying to locate data items of interest based on the associations between the classification metadata and the data objects.
3 Assignments
0 Petitions
Accused Products
Abstract
Disclosed herein are methods and systems for integrating an enterprise'"'"'s structured and unstructured data to provide users and enterprise applications with efficient and intelligent access to that data. In accordance with exemplary embodiments, the generation of classification information about unstructured data can be hardware-accelerated by processing streaming unstructured data through a reconfigurable logic device, a graphics processor unit (GPU), or chip multi-processor (CMP) to generate the classification metadata about the unstructured data.
484 Citations
42 Claims
-
1. A method for low latency stream indexing, the method comprising:
-
receiving streaming unstructured data into a member of the group consisting of (1) a reconfigurable logic device, (2) a graphics processor unit (GPU), and (3) a chip multi-processor (CMP), the streaming unstructured data comprising a plurality of data objects, wherein the member has a plurality of parallel processing engines deployed thereon; the parallel processing engines analyzing the data objects while the data objects stream through the member to perform a classification operation that determines classification metadata about the data objects; and creating an association that is physically represented in memory between the determined classification metadata and the data objects, wherein the creating step comprises generating a classification index about the streaming unstructured data from the classification metadata and the associations between the classification metadata and the corresponding data objects, the classification index for subsequent querying to locate data items of interest based on the associations between the classification metadata and the data objects. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23)
-
-
24. An apparatus for low latency stream indexing, the apparatus comprising:
-
a member of the group consisting of (1) a reconfigurable logic device, (2) a graphics processor unit (GPU), and (3) a chip multi-processor (CMP), the member configured to receive streaming unstructured data, the streaming unstructured data comprising a plurality of data objects, wherein the member has a plurality of parallel processing engines arranged as a pipeline deployed thereon; wherein the parallel processing engines are configured to (1) analyze the data objects while the data objects stream through the pipeline to perform a classification operation that determines classification metadata about the data objects, and (2) create an association that is physically represented in a memory between the determined classification metadata and the data objects; and wherein the pipeline is configured to create the association through generation of a classification index about the streaming unstructured data from the classification metadata and the associations between the classification metadata and the corresponding data objects, the classification index for subsequent querying to locate data items of interest based on the associations between the classification metadata and the data objects. - View Dependent Claims (25, 26, 27, 28, 29, 41, 42)
-
-
30. An apparatus comprising:
-
a reconfigurable logic device configured to receive streaming unstructured data, the streaming unstructured data comprising a plurality of data objects, wherein the reconfigurable logic device has a plurality of parallel processing engines arranged as a firmware pipeline deployed thereon; wherein the parallel processing engines are configured to (1) analyze the data objects while the data objects stream through the firmware pipeline to perform a classification operation that determines classification metadata about the data objects, and (2) create an association that is physically represented in a memory between the determined classification metadata and the data objects; and wherein the firmware pipeline comprises matching logic, and wherein the matching logic is configured to compare a plurality of the streaming data objects against a plurality of defined terms to determine which of the defined terms are present in the data objects, each defined term having an association with a classification. - View Dependent Claims (31, 32, 33, 34, 35, 36, 37, 38, 39, 40)
-
Specification