Structuring unstructured machine-generated content

• US 10,733,213 B2
• Filed: 12/31/2018
• Issued: 08/04/2020
• Est. Priority Date: 12/31/2018
• Status: Active Grant

• Current Assignee: Iguazio Systems Ltd.
• Original Assignee: Iguazio Systems Ltd.
• Inventors: Modai, Ori, Nissan-Messing, Orit, Haviv, Yaron
• Primary Examiner(s): Dang, Thanh-Ha
• Application Number:
  US16/237,673
• Publication Number:
  US 20200210456A1
• Time in Patent Office: 582 Days
• Field of Search: 707737
• CPC Class Codes: G06F 16/24522 Translation of natural language queries to structured queries
• CPC Class Codes: G06F 16/2455 Query execution
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification

Abstract

Systems and methods are disclosed for structuring unstructured machine-generated content. In one implementation, unstructured content is received and processed to identify a first content segment. Parameter(s) within the first content segment are identified and classified. A content segment skeleton that reflects an arrangement of the first content segment is generated. Based on the classifying of the parameter(s), the parameter(s) are extracted s from the first content segment into a structured content element in a structured content format, with the structured content element including a representation of the content segment skeleton. Based on the structured format, a query adapter is generated. Queries are executed via the query adapter and the structured format.

Claims

1. A system comprising:

a processing device; and

a memory operatively coupled to the processing device and storing instructions that, when executed by the processing device, cause the system to perform operations comprising;

receiving unstructured content;

processing the unstructured content to identify a first content segment;

identifying, within the first content segment, one or more parameters;

classifying the one or more parameters;

within the first content segment, substituting the one or more parameters for one or more type qualifiers determined to correspond to the one or more parameters, thereby generating a content segment skeleton, the content segment skeleton comprising a hashed representation that reflects an arrangement of the first content segment; and

based on the classifying of the one or more parameters, extracting the one or more parameters from the first content segment into a structured content element in a structured content format, wherein the structured content element includes the hashed representation of the content segment skeleton.

2. The system of claim 1, wherein generating a content segment skeleton comprises generating a content segment skeleton that reflects an arrangement of the one or more type qualifiers within the first content segment.

3. The system of claim 1, wherein classifying the one or more parameters comprises classifying a first parameter based on a string pattern identified within the first parameter.

4. The system of claim 1, wherein classifying the one or more parameters comprises classifying a first parameter based on a source of the unstructured content.

5. The system of claim 1, wherein classifying the one or more parameters comprises classifying a first parameter in the first content segment based on a classification of a second parameter in a second content segment.

6. The system of claim 1, wherein classifying the one or more parameters comprises classifying a first parameter in the first content segment based on a classification of a second parameter in the first content segment.

7. The system of claim 1, wherein extracting the one or more parameters comprises indexing the one or more parameters from the first content segment into a database format.

8. The system of claim 1, further comprising:

determining an occurrence of an event based on content associated with one or more structured content elements; and

initiating one or more actions based on the determined occurrence.

9. The system of claim 1, wherein a query adapter is generated in view of the arrangement of the first content segment as reflected in the content segment skeleton.

10. The system of claim 9, wherein executing one or more queries comprises:

executing a first portion of a query;

executing a second portion of a query; and

completing execution of the query via the query adapter based on a result of the execution of the first portion of the query and a result of the execution of a second portion of the query.

11. The system of claim 9, wherein executing one or more queries comprises:

receiving a free text search query comprising a first query element and a second query element;

querying a structured content repository with respect to the first query element to generate a first query result;

querying the structured content repository with respect to the second query element to generate a second query result;

combining the first query result and the second query result to identify a structured content element within the structured content repository that contains the first query element and the second query element;

processing the one or more parameters of the identified structured content element with respect to the content segment skeleton included in the structured content element to reconstruct the content segment from which the identified structured content element was generated; and

querying the reconstructed content segment with respect to the free text search query.

12. The system of claim 1, wherein the unstructured content comprises human generated content.

13. The system of claim 12, wherein processing the unstructured content comprises applying one or more language processing techniques to the human generated content.

14. The system of claim 1, wherein the memory further stores instructions to cause the system to perform operations comprising:

generating, based on the structured format, a query adapter; and

executing one or more queries via the query adapter and the structured format.

15. A method comprising:

receiving unstructured content;

processing the unstructured content to identify a first content segment;

identifying, within the first content segment, one or more parameters;

classifying the one or more parameters;

within the first content segment, substituting the one or more parameters for one or more type qualifiers determined to correspond to the one or more parameters, thereby generating a content segment skeleton, the content segment skeleton comprising a hashed representation that reflects an arrangement of one or more type qualifiers within the first content segment;

based on the classifying, extracting the one or more parameters from the first content segment into a structured content element in a structured content format;

generating, based on the structured format, a query adapter;

receiving, via the query adapter, a search query;

identifying, with respect to the search query, a plurality of query results;

combining one or more of the identified query results to identify a structured content element that contains elements of the search query;

processing one or more parameters of the identified structured content element with respect to a content segment skeleton included in the structured content element to reconstruct the content segment from the unstructured content based on which the identified structured content element was generated; and

querying the reconstructed content segment with respect to the search query.

16. The method of claim 15, wherein generating a content segment skeleton comprises substituting, within the first content segment, the one or more parameters with one or more type qualifiers determined to correspond to the one or more parameters.

17. The method of claim 15, wherein generating a content segment skeleton comprises generating a content segment skeleton that reflects an arrangement of the one or more type qualifiers within the first content segment.

18. A non-transitory computer readable medium having instructions stored thereon that, when executed by a processing device, cause the processing device to perform operations comprising:

receiving unstructured content;

processing the unstructured content to identify a first content segment;

identifying, within the first content segment, one or more parameters;

classifying the one or more parameters;

within the first content segment, substituting the one or more parameters for one or more type qualifiers determined to correspond to the one or more parameters, thereby generating a content segment skeleton, the content segment skeleton comprising a hashed representation of an arrangement of the one or more type qualifiers within the first content segment;

based on the classifying, extracting the one or more parameters from the first content segment into a structured content element in a structured content format, wherein the structured content element includes the hashed representation of the content segment skeleton;

generating, based on the structured format, a query adapter;

receiving, via the query adapter, a free text search query comprising a first query element and a second query element;

querying a structured content repository with respect to the first query element to generate a first query result;

querying the structured content repository with respect to the second query element to generate a second query result;

combining the first query result and the second query result to identify a structured content element within the structured content repository that contains the first query element and the second query element;

processing one or more parameters of the identified structured content element with respect to a content segment skeleton included in the structured content element to reconstruct a content segment from the unstructured content based on which the identified structured content element was generated; and

querying the reconstructed content segment with respect to the free text search query.

19. The non-transitory computer-readable medium of claim 18, wherein extracting the one or more parameters comprises indexing the one or more parameters from the first content segment into a database format.