Data records selection

• US 9,892,026 B2
• Filed: 03/14/2013
• Issued: 02/13/2018
• Est. Priority Date: 02/01/2013
• Status: Active Grant

• Current Assignee: Ab Initio Technology LLC (Ab Initio Software Corporation)
• Original Assignee: Ab Initio Technology LLC (Ab Initio Software Corporation)
• Inventors: Isman, Marshall A., Epstein, Richard A., Haug, Ralf, Roberts, Andrew F., Ralston, John, Richardson, John L., Pniower, Justin
• Primary Examiner(s): Saeed, Usmaan
• Assistant Examiner(s): Perez-Arroyo, Raquel
• Application Number:
  US13/827,558
• Publication Number:
  US 20140222752A1
• Time in Patent Office: 1,797 Days
• Field of Search: 707603
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

A computer-implemented method includes accessing a plurality of data records, each data record having a plurality of data fields. The method further includes analyzing values for one or more of the data fields for at least some of the plurality of data records and generating a profile of the plurality of data records based on the analyzing. The method further includes formulating at least one subsetting rule based on the profile; and selecting a subset of data records from the plurality of data records based on the at least one subsetting rule.

Claims

1. A computer-implemented method for selecting data records to cause execution of a processing rule during testing of a data processing application, the method including:

obtaining a first set of data records;

processing the first set of data records using a data processing application that includes a processing rule, wherein a processing rule operates on at least one input value and generates at least one output value, and wherein whether the processing rule is executed by the data processing application during processing of a particular data record depends directly or indirectly on a value in each of one or more data fields of the particular data record;

receiving execution information indicative of a number of times the processing rule was executed in connection with processing of the first set of data records;

obtaining a second set of data records;

analyzing values in one or more data fields of each of the data records in the second set, the analyzing including generating a profile of each of one or more of the data fields for the second set of data records, the profile of a data field characterizing the values in the data field;

obtaining a subsetting rule based on (i) the generated profile and (ii) the execution information indicative of the number of times the processing rule was executed in connection with processing the first set of data records, the subsetting rule including an identification of a particular one of the data fields of the data records in the second set as a target data field;

selecting a subset of data records from the second set of data records according to the subsetting rule, the selecting of the subset of data records being based on values in the target data field; and

processing the selected subset of data records using the data processing application.

2. The method of claim 1, wherein obtaining the subsetting rule includes formulating the subsetting rule, including identifying the one of the data fields as the target data field based on a cardinality of the identified one of the data fields.

3. The method of claim 2, wherein the target data field has a set of distinct values in the plurality of data records, and wherein selecting a subset of data records includes selecting data records such that there is at least one data record in the selected subset that has each of the distinct values for the target data field.

4. The method of claim 1, wherein generating a profile includes classifying values for a first data field of the data records in the second set of data records;

and

wherein obtaining the subsetting rule includes formulating the subsetting rule, including identifying the first data field as the target data field based on the classifying.

5. The method of claim 4, wherein the target data field has a set of distinct values of the data records in the second set of data records, and wherein selecting a subset of data records includes selecting data records such that there is at least one data record in the selected subset that has each of the distinct values for the target data field.

6. The method of claim 1, wherein the subsetting rule identifies a first data field as a first target data field and a second data field as a second target data field.

7. The method of claim 6, wherein selecting a subset of data records includes selecting the subset of data records based on combinations of a first set of distinct values for the first target data field and a second set of distinct values for the second target data field.

8. The method of claim 1, wherein generating a profile includes identifying a relationship between data records of the second set of data records related via values of a first data field;

and

wherein the at least one subsetting rule includes an identification of the relationship.

9. The method of claim 8, wherein selecting a subset of data records includes:

selecting a first data record; and

selecting one or more second data records related to the first data record via the relationship identified in the subsetting rule.

10. The method of claim 8, wherein the relationship between data records includes a relationship between data records in the second set of data records and data records in a third set of data records.

11. The method of claim 1, wherein generating a profile includes:

generating a pseudofield for at least some of the data records in the second set of data records; and

populating the pseudofield for each corresponding data record with an accumulated value, wherein the accumulated value for a first data record is determined based on the first data record and at least one other data record related to the first data record,

wherein the first data record and the at least one other data record are related via values of a first data field.

12. The method of claim 11, including determining the accumulated value based on a sum of a value for a second data field of the first data record and values for the second data field for each other related data record.

13. The method of claim 1, wherein obtaining the subsetting rule includes receiving the subsetting rule.

14. The method of claim 1, including providing the selected subset of data records to the data processing application.

15. The method of claim 1, including:

formulating a second subsetting rule based on results of processing the selected subset of data records by the data processing application; and

selecting a second subset of data records based on the second subsetting rule.

16. A non-transitory computer-readable medium storing instructions for causing a computing system to select data records to cause execution of a processing rule during testing of a data processing application, the instructions causing the computing system to:

obtain a first set of data records;

process the first set of data records using a data processing application that includes a processing rule, wherein a processing rule operates on at least one input value and generates at least one output value, and wherein whether the processing rule is executed by the data processing application during processing of a particular data record depends directly or indirectly on a value in each of one or more data fields of the particular data record;

receive execution information indicative of a number of times the processing rule was executed in connection with processing of the first set of data records;

obtain a second set of data records;

analyze values in one or more data fields of each of the data records in the second set, the analyzing including generating a profile of each of one or more of the data fields for the second set of data records, the profile of a data field characterizing the values in the data field;

obtain a subsetting rule based on (i) the generated profiles and (ii) the execution information indicative of the number of times the processing rule was executed in connection with processing the first set of data records, the subsetting rule including an identification of a particular one of the data fields of the data records in the second set as a target data field;

select a subset of data records from the second set of data records according to the subsetting rule, the selecting of the subset of data records being based on values in the target data field; and

process the selected subset of data records using the data processing application.

17. The non-transitory computer-readable medium of claim 16, wherein obtaining the subsetting rule includes formulating the subsetting rule, including identifying the one of the data fields as the target data field based on a cardinality of the identified one of the data fields.

18. The non-transitory computer-readable medium of claim 17, wherein the target data field has a set of distinct values in the plurality of data records, and wherein selecting a subset of data records includes selecting data records such that there is at least one data record in the selected subset that has each of the distinct values for the target data field.

19. The non-transitory computer-readable medium of claim 16, wherein generating a profile includes classifying values for a first data field of the data records in the second set of data records;

and

wherein obtaining the subsetting rule includes formulating the subsetting rule, including identifying the first data field as the target data field based on the classifying.

20. The non-transitory computer-readable medium of claim 19, wherein the target data field has a set of distinct values of the data records in the second set of data records, and wherein selecting a subset of data records includes selecting data records such that there is at least one data record in the selected subset that has each of the distinct values for the target data field.

21. The non-transitory computer-readable medium of claim 16, wherein the subsetting rule identifies a first data field as a first target data field and a second data field as a second target data field.

22. The non-transitory computer-readable medium of claim 21, wherein selecting a subset of data records includes selecting the subset of data records based on combinations of a first set of distinct values for the first target data field and a second set of distinct values for the second target data field.

23. The non-transitory computer-readable medium of claim 16, wherein generating a profile includes identifying a relationship between data records of the second set of data records related via values of a first data field;

and

wherein the at least one subsetting rule includes an identification of the relationship.

24. The non-transitory computer-readable medium of claim 23, wherein selecting a subset of data records includes:

selecting a first data record; and

selecting one or more second data records related to the first data record via the relationship identified in the subsetting rule.

25. The non-transitory computer-readable medium of claim 16, wherein generating a profile includes:

generating a pseudofield for at least some of the data records in the second set of data records; and

populating the pseudofield for each corresponding data record with an accumulated value, wherein the accumulated value for a first data record is determined based on the first data record and at least one other data record related to the first data record,

wherein the first data record and the at least one other data record are related via values of a first data field.

26. The non-transitory computer-readable medium of claim 25, including determining the accumulated value based on a sum of a value for a second data field of the first data record and values for the second data field for each other related data record.

27. The non-transitory computer-readable medium of claim 16, including providing the selected subset of data records to the data processing application.

28. The non-transitory computer-readable medium of claim 16, including:

formulating a second subsetting rule based on results of processing the selected subset of data records by the data processing application; and

selecting a second subset of data records based on the second subsetting rule.

29. A computing system for selecting data records to cause execution of a processing rule during testing of a data processing application, the system including:

at least one processor configured to;

obtain a first set of data records;

process the first set of data records using a data processing application that includes a processing rule, wherein a processing rule operates on at least one input value and generates at least one output value, and wherein whether the processing rule is executed by the data processing application during processing of a particular data record depends directly or indirectly on a value in each of one or more data fields of the particular data record;

receive execution information indicative of a number of times the processing rule was executed in connection with processing of the first set of data records;

obtain a second set of data records;

analyze values in one or more data fields of each of the data records in the second set, the analyzing including generating a profile of each of one or more of the data fields for the second set of data records, the profile of a data field characterizing the values in the data field;

obtain a subsetting rule based on (i) the generated profiles and (ii) the execution information indicative of the number of times the processing rule was executed in connection with processing the first set of data records, the subsetting rule including an identification of a particular one of the data fields of the data records in the second set as a target data field;

select a subset of data records from the second set of data records according to the subsetting rule, the selecting of the subset of data records being based on values in the target data field; and

processing the selected subset of data records using the data processing application.

30. The computing system of claim 29, wherein obtaining the subsetting rule includes formulating the subsetting rule, including identifying the one of the data fields as the target data field based on a cardinality of the identified one of the data fields.

31. The computing system of claim 30, wherein the target data field has a set of distinct values in the plurality of data records, and wherein selecting a subset of data records includes selecting data records such that there is at least one data record in the selected subset that has each of the distinct values for the target data field.

32. The computing system of claim 29, wherein generating a profile includes classifying values for a first data field of the data records in the second set of data records;

and

wherein obtaining the subsetting rule includes formulating the subsetting rule, including identifying the first data field as the target data field based on the classifying.

33. The computing system of claim 32, wherein the target data field has a set of distinct values of the data records in the second set of data records, and wherein selecting a subset of data records includes selecting data records such that there is at least one data record in the selected subset that has each of the distinct values for the target data field.

34. The computing system of claim 29, wherein the subsetting rule identifies a first data field as a first target data field and a second data field as a second target data field.

35. The computing system of claim 34, wherein selecting a subset of data records includes selecting the subset of data records based on combinations of a first set of distinct values for the first target data field and a second set of distinct values for the second target data field.

36. The computing system of claim 29, wherein generating a profile includes identifying a relationship between data records of the second set of data records related via values of a first data field;

and

wherein the at least one subsetting rule includes an identification of the relationship.

37. The computing system of claim 36, wherein selecting a subset of data records includes:

selecting a first data record; and

selecting one or more second data records related to the first data record via the relationship identified in the subsetting rule.

38. The computing system of claim 29, wherein generating a profile includes:

generating a pseudofield for at least some of the data records in the second set of data records; and

populating the pseudofield for each corresponding data record with an accumulated value, wherein the accumulated value for a first data record is determined based on the first data record and at least one other data record related to the first data record,

wherein the first data record and the at least one other data record are related via values of a first data field.

39. The computing system of claim 38, including determining the accumulated value based on a sum of a value for a second data field of the first data record and values for the second data field for each other related data record.

40. The computing system of claim 29, including providing the selected subset of data records to the data processing application.

41. The computing system of claim 29, including:

formulating a second subsetting rule based on results of processing the selected subset of data records by the data processing application; and

selecting a second subset of data records based on the second subsetting rule.

42. A computing system for selecting data records to cause execution of a processing rule during testing of a data processing application, the computing system including:

means for obtaining a first set of data records;

means for processing the first set of data records using a data processing application that includes a processing rule, wherein a processing rule operates on at least one input value and generates at least one output value, and wherein whether the processing rule is executed by the data processing application during processing of a particular data record depends directly or indirectly on a value in each of one or more data fields of the particular data record;

means for receiving execution information indicative of a number of times the processing rule was executed in connection with processing of the first set of data records;

means for obtaining a second set of data records;

means for analyzing values in one or more data fields of each of the data records in the second set, the analyzing including generating a profile of each of one or more of the data fields for the second set of data records, the profile of a data field characterizing the values in the data field;

means for obtaining a subsetting rule based on (i) the generated profiles and (ii) the execution information indicative of the number of times the processing rule was executed in connection with processing the first set of data records, the subsetting rule including an identification of a particular one of the data fields of the data records in the second set as a target data field;

means for selecting a subset of data records from the second set of data records according to the subsetting rule, the selecting of the subset of data records being based on values in the target data field; and

means for processing the selected subset of data records using the data processing application.