NORMALIZING AMORPHOUS QUERY RESULT SETS

• US 20150134590A1
• Filed: 11/11/2013
• Published: 05/14/2015
• Est. Priority Date: 11/11/2013
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Boss, Gregory Jensen, Trim, Craig Matthew, ABUELSAAD, TAMER E., Wong, Albert Tien-Yuen
• Application Number:
  US14/076,673
• Publication Number:
  US 20150134590A1
• Time in Patent Office: Days
• US Class Current: 707/602
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses

Abstract

A method, system, and computer program product for normalizing amorphous query result sets are provided in the illustrative embodiments. A property of data in a portion of the result set is identified. the property is usable for normalizing the portion into a structured data. Based on the property, the portion is categorized into a first category as a candidate for normalization using a first structure specification. The portion is transformed, responsive to the first category being selected for normalizing the portion over a second category in an evaluation, into the structured data according to the first structure specification of the first category. The structured data and a metadata of structure specification are added to a normalized result set. The normalized result set is output to a consumer application.

Claims

1. A method for normalizing an amorphous query result set, the method comprising:

identifying a property of data in a portion of the result set, wherein the property is usable for normalizing the portion into a structured data;

categorizing, into a first category, based on the property, the portion as a candidate for normalization using a first structure specification;

transforming, responsive to the first category being selected for normalizing the portion over a second category in an evaluation, the portion into the structured data according to the first structure specification of the first category;

adding the structured data and a metadata of structure specification to a normalized result set; and

outputting the normalized result set to a consumer application.

2. The method of claim 1, further comprising:

receiving a parameter associated with a producer of a data item in the result set;

categorizing, into the second category, based on the parameter, the portion as a candidate for normalization using a second structure specification;

evaluating the first and the second categories to determine a category to use for normalizing the portion; and

transforming, responsive to the second category being selected for normalizing the portion over the first category in the evaluation, the portion into the structured data according to the second structure specification of the second category.

3. The method of claim 2, wherein the parameter indicates a provenance of the producer.

4. The method of claim 1, wherein the property of the data is received from the consumer application as a categorization marker, wherein the marker is received from the consumer application for normalizing the result set in to a specific structured data required by the consumer application.

5. The method of claim 1, further comprising:

assigning a confidence level to the first category;

detecting another property of data in the portion;

categorizing the portion into a second category;

assigning a second confidence level to the second category; and

selecting, from the first and the second categories, a category corresponding to the higher of the first and the second confidence levels.

6. The method of claim 1, further comprising:

assigning a confidence level to the first category, wherein the confidence level is indicative of a probability that the property correctly categorizes the portion for normalization using the structure specification.

7. The method of claim 6, wherein the property is further usable for normalizing the portion into a second structured data according to a second structure specification with a second probability.

8. A computer program product comprising one or more computer-readable tangible storage devices and computer-readable program instructions which are stored on the one or more storage devices and when executed by one or more processors, perform the method of claim 1.

9. A computer system comprising one or more processors, one or more computer-readable memories, one or more computer-readable tangible storage devices and program instructions which are stored on the one or more storage devices for execution by the one or more processors via the one or more memories and when executed by the one or more processors perform the method of claim 1.

10. A computer program product for normalizing an amorphous query result set, the computer program product comprising:

one or more computer-readable tangible storage devices;

program instructions, stored on at least one of the one or more storage devices, to identify a property of data in a portion of the result set, wherein the property is usable for normalizing the portion into a structured data;

program instructions, stored on at least one of the one or more storage devices, to categorize, into a first category, based on the property, the portion as a candidate for normalization using a first structure specification;

program instructions, stored on at least one of the one or more storage devices, to transform, responsive to the first category being selected for normalizing the portion over a second category in an evaluation, the portion into the structured data according to the first structure specification of the first category;

program instructions, stored on at least one of the one or more storage devices, to add the structured data and a metadata of structure specification to a normalized result set; and

program instructions, stored on at least one of the one or more storage devices, to output the normalized result set to a consumer application.

11. The computer program product of claim 10, further comprising:

program instructions, stored on at least one of the one or more storage devices, to receive a parameter associated with a producer of a data item in the result set;

program instructions, stored on at least one of the one or more storage devices, to categorize, into the second category, based on the parameter, the portion as a candidate for normalization using a second structure specification;

program instructions, stored on at least one of the one or more storage devices, to evaluate the first and the second categories to determine a category to use for normalizing the portion; and

program instructions, stored on at least one of the one or more storage devices, to transform, responsive to the second category being selected for normalizing the portion over the first category in the evaluation, the portion into the structured data according to the second structure specification of the second category.

12. The computer program product of claim 11, wherein the parameter indicates a provenance of the producer.

13. The computer program product of claim 10, wherein the property of the data is received from the consumer application as a categorization marker, wherein the marker is received from the consumer application for normalizing the result set in to a specific structured data required by the consumer application.

14. The computer program product of claim 10, further comprising:

program instructions, stored on at least one of the one or more storage devices, to assign a confidence level to the first category;

program instructions, stored on at least one of the one or more storage devices, to detect another property of data in the portion;

program instructions, stored on at least one of the one or more storage devices, to categorize the portion into a second category;

program instructions, stored on at least one of the one or more storage devices, to assign a second confidence level to the second category; and

program instructions, stored on at least one of the one or more storage devices, to select, from the first and the second categories, a category corresponding to the higher of the first and the second confidence levels.

15. The computer program product of claim 10, further comprising:

program instructions, stored on at least one of the one or more storage devices, to assign a confidence level to the first category, wherein the confidence level is indicative of a probability that the property correctly categorizes the portion for normalization using the structure specification.

16. The computer program product of claim 15, wherein the property is further usable for normalizing the portion into a second structured data according to a second structure specification with a second probability.

17. A computer system for normalizing an amorphous query result set, the computer system comprising:

one or more processors, one or more computer-readable memories and one or more computer-readable tangible storage devices;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to identify a property of data in a portion of the result set, wherein the property is usable for normalizing the portion into a structured data;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to categorize, into a first category, based on the property, the portion as a candidate for normalization using a first structure specification;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to transform, responsive to the first category being selected for normalizing the portion over a second category in an evaluation, the portion into the structured data according to the first structure specification of the first category;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to add the structured data and a metadata of structure specification to a normalized result set; and

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to output the normalized result set to a consumer application.

18. The computer system of claim 17, further comprising:

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to receive a parameter associated with a producer of a data item in the result set;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to categorize, into the second category, based on the parameter, the portion as a candidate for normalization using a second structure specification;

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to evaluate the first and the second categories to determine a category to use for normalizing the portion; and

program instructions, stored on at least one of the one or more storage devices for execution by at least one of the one or more processors via at least one of the one or more memories, to transform, responsive to the second category being selected for normalizing the portion over the first category in the evaluation, the portion into the structured data according to the second structure specification of the second category.

19. The computer system of claim 18, wherein the parameter indicates a provenance of the producer.

20. The computer system of claim 17, wherein the property of the data is received from the consumer application as a categorization marker, wherein the marker is received from the consumer application for normalizing the result set in to a specific structured data required by the consumer application.