Data records selection

• US 10,241,900 B2
• Filed: 02/08/2018
• Issued: 03/26/2019
• Est. Priority Date: 02/01/2013
• Status: Active Grant

• Current Assignee: Ab Initio Technology LLC (Ab Initio Software Corporation)
• Original Assignee: Ab Initio Technology LLC (Ab Initio Software Corporation)
• Inventors: Isman, Marshall A., Epstein, Richard A., Haug, Ralf, Roberts, Andrew F., Ralston, John, Richardson, John L., Pniower, Justin
• Primary Examiner(s): Saeed, Usmaan
• Assistant Examiner(s): Perez-Arroyo, Raquel
• Application Number:
  US15/891,794
• Publication Number:
  US 20180165181A1
• Time in Patent Office: 411 Days
• Field of Search: 707603
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases
• CPC Class Codes: G06F 16/217 Database tuning G06F16/2282 takes precedence; database performance monitoring G06F11/3409
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

A computer-implemented method includes accessing a plurality of data records, each data record having a plurality of data fields. The method further includes analyzing values for one or more of the data fields for at least some of the plurality of data records and generating a profile of the plurality of data records based on the analyzing. The method further includes formulating at least one subsetting rule based on the profile; and selecting a subset of data records from the plurality of data records based on the at least one subsetting rule.

Claims

1. A computer-implemented method for selecting data records to cause execution of a processing rule during testing of a data processing application, the method including:

obtaining a first set of data records;

processing in a data processing system the first set of data records using a data processing application that includes a processing rule that operates on at least one input value and generates at least one output value, and wherein whether the processing rule is executed by the data processing application during processing of a particular data record depends directly or indirectly on a value in each of one or more data fields of the particular data record;

receiving execution information indicative of a number of times the processing rule was executed in connection with processing of the first set of data records;

obtaining a second set of data records;

characterizing values in one or more data fields of each of the data records in the second set;

determining an identification criterion that identifies a particular one of the data fields of the data records in the second set as a target data field, the identification criterion being based on (i) the characterization of the values in the one or more data fields of each of the data records in the second set and (ii) the execution information indicative of the number of times the processing rule was executed in connection with processing the first set of data records;

applying the identification criterion to select one or more data records from the second set of data records based on values in the target data field; and

processing the selected one or more data records using the data processing application.

2. The method of claim 1, including identifying a particular one of the data fields of the data records in the second set as a target data field based on a cardinality of each of one or more of the data fields of the data records in the second set.

3. The method of claim 2, wherein there are multiple distinct values in the target data field across the data records in the second set, and wherein selecting one or more data records from the second set of data records includes selecting at least one data record having each of the distinct values in the target data field.

4. The method of claim 1, wherein the target data field is a first target data field, and including identifying a second particular one of the data fields of the data records in the second set as a second target data field.

5. The method of claim 4, wherein selecting one or more data records from the second set of data records includes selecting the data records based on a combination of values in the first target data field and values in the second target data field.

6. The method of claim 5, wherein there are multiple distinct values in the first target data field across the data records in the second set and multiple distinct values in the second target data field across the data records in the second set, and wherein selecting one or more data records from the second set of data records includes selecting the data records based on combinations of the distinct values in the first target data field and the distinct values in the second target data field.

7. The method of claim 1, wherein characterizing values in one or more data fields includes identifying a first data field having a value that indicates a relationship among two or more data records in the second set of data records.

8. The method of claim 7, wherein selecting one or more data records from the second set of data records includes:

selecting a first data record;

identifying one or more second data records related to the first data record based on the value in the first data field of the first data record; and

selecting the one or more second data records.

9. The method of claim 7, including determining an accumulated value for the first data record based on a value in a second data field of the first data record and a value in the second data field for each of the two or more data records that are related to the first data record;

generating a new field for the first data record; and

populating the new field with the accumulated value.

10. The method of claim 9, wherein determining the accumulated value includes determining the accumulated value based on a sum of the value in the second data field of the first data record and the value in the second data field for each of the two or more data records that are related to the first data record.

11. The method of claim 9, wherein selecting one or more data records from the second set of data records includes selecting the one or more data records based on the accumulated value in the new field for each of one or more data records in the second set of data records.

12. A system for selecting data records to cause execution of a processing rule during testing of a data processing application, the system including:

at least one processor configured to;

obtain a first set of data records;

process in a data processing system the first set of data records using a data processing application that includes a processing rule that operates on at least one input value and generates at least one output value, and wherein whether the processing rule is executed by the data processing application during processing of a particular data record depends directly or indirectly on a value in each of one or more data fields of the particular data record;

receive execution information indicative of a number of times the processing rule was executed in connection with processing of the first set of data records;

obtain a second set of data records;

characterize values in one or more data fields of each of the data records in the second set;

determine an identification criterion that identifies a particular one of the data fields of the data records in the second set as a target data field, the identification criterion being based on (i) the characterization of the values in the one or more data fields of each of the data records in the second set and (ii) the execution information indicative of the number of times the processing rule was executed in connection with processing the first set of data records;

applying the identification criterion to select one or more data records from the second set of data records based on values in the target data field; and

process the selected one or more data records using the data processing application.

13. The system of claim 12, wherein the at least one processor is configured to identify a particular one of the data fields of the data records in the second set as a target data field based on a cardinality of each of one or more of the data fields of the data records in the second set.

14. The system of claim 13, wherein there are multiple distinct values in the target data field across the data records in the second set, and wherein selecting one or more data records from the second set of data records includes selecting at least one data record having each of the distinct values in the target data field.

15. The system of claim 12, wherein the target data field is a first target data field, and including identifying a second particular one of the data fields of the data records in the second set as a second target data field.

16. The system of claim 15, wherein selecting one or more data records from the second set of data records includes selecting the data records based on a combination of values in the first target data field and values in the second target data field.

17. The system of claim 16, wherein there are multiple distinct values in the first target data field across the data records in the second set and multiple distinct values in the second target data field across the data records in the second set, and wherein selecting one or more data records from the second set of data records includes selecting the data records based on combinations of the distinct values in the first target data field and the distinct values in the second target data field.

18. The system of claim 12, wherein characterizing values in one or more data fields includes identifying a first data field having a value that indicates a relationship among two or more data records in the second set of data records.

19. The system of claim 18, wherein selecting one or more data records from the second set of data records includes:

selecting a first data record;

identifying one or more second data records related to the first data record based on the value in the first data field of the first data record; and

selecting the one or more second data records.

20. The system of claim 18, wherein the at least one processor is configured to determine an accumulated value for the first data record based on a value in a second data field of the first data record and a value in the second data field for each of the two or more data records that are related to the first data record;

generating a new field for the first data record;

populating the new field with the accumulated value.

21. The system of claim 20, wherein selecting one or more data records from the second set of data records includes selecting the one or more data records based on the accumulated value in the new field for each of one or more data records in the second set of data records.

22. A non-transitory computer readable medium storing instructions for causing a computer system to select data records to cause execution of a processing rule during testing of a data processing application, the instructions causing the computer system to:

obtain a first set of data records;

process in a data processing system the first set of data records using a data processing application that includes a processing rule that operates on at least one input value and generates at least one output value, and wherein whether the processing rule is executed by the data processing application during processing of a particular data record depends directly or indirectly on a value in each of one or more data fields of the particular data record;

receive execution information indicative of a number of times the processing rule was executed in connection with processing of the first set of data records;

obtain a second set of data records;

characterize values in one or more data fields of each of the data records in the second set;

determine an identification criterion that identifies a particular one of the data fields of the data records in the second set as a target data field, the identification criterion being based on (i) the characterization of the values in the one or more data fields of each of the data records in the second set and (ii) the execution information indicative of the number of times the processing rule was executed in connection with processing the first set of data records;

applying the identification criterion to select one or more data records from the second set of data records based on values in the target data field; and

process the selected one or more data records using the data processing application.

23. The non-transitory computer readable medium of claim 22, the instructions causing the computer system to identify a particular one of the data fields of the data records in the second set as a target data field based on a cardinality of each of one or more of the data fields of the data records in the second set.

24. The non-transitory computer readable medium of claim 23, wherein there are multiple distinct values in the target data field across the data records in the second set, and wherein selecting one or more data records from the second set of data records includes selecting at least one data record having each of the distinct values in the target data field.

25. The non-transitory computer readable medium of claim 22, wherein the target data field is a first target data field, and including identifying a second particular one of the data fields of the data records in the second set as a second target data field.

26. The non-transitory computer readable medium of claim 25, wherein selecting one or more data records from the second set of data records includes selecting the data records based on a combination of values in the first target data field and values in the second target data field.

27. The non-transitory computer readable medium of claim 26, wherein there are multiple distinct values in the first target data field across the data records in the second set and multiple distinct values in the second target data field across the data records in the second set, and wherein selecting one or more data records from the second set of data records includes selecting the data records based on combinations of the distinct values in the first target data field and the distinct values in the second target data field.

28. The non-transitory computer readable medium of claim 22, wherein characterizing values in one or more data fields includes identifying a first data field having a value that indicates a relationship among two or more data records in the second set of data records.

29. The non-transitory computer readable medium of claim 28, wherein selecting one or more data records from the second set of data records includes:

selecting a first data record;

identifying one or more second data records related to the first data record based on the value in the first data field of the first data record; and

selecting the one or more second data records.

30. The non-transitory computer readable medium of claim 28, the instructions causing the computer system to determine an accumulated value for the first data record based on a value in a second data field of the first data record and a value in the second data field for each of the two or more data records that are related to the first data record;

generating a new field for the first data record;

populating the new field with the accumulated value.

31. The non-transitory computer readable medium of claim 30, wherein selecting one or more data records from the second set of data records includes selecting the one or more data records based on the accumulated value in the new field for each of one or more data records in the second set of data records.

32. A system for selecting data records to cause execution of a processing rule during testing of a data processing application, the system including:

means for obtaining a first set of data records;

means for processing in a data processing system the first set of data records using a data processing application that includes a processing rule that operates on at least one input value and generates at least one output value, and wherein whether the processing rule is executed by the data processing application during processing of a particular data record depends directly or indirectly on a value in each of one or more data fields of the particular data record;

means for receiving execution information indicative of a number of times the processing rule was executed in connection with processing of the first set of data records;

means for obtaining a second set of data records;

means for characterizing values in one or more data fields of each of the data records in the second set;

means for determining an identification criterion that identifies a particular one of the data fields of the data records in the second set as a target data field, the identification criterion being based on (i) the characterization of the values in the one or more data fields of each of the data records in the second set and (ii) the execution information indicative of the number of times the processing rule was executed in connection with processing the first set of data records;

means for applying the identification criterion to select one or more data records from the second set of data records based on values in the target data field; and

means for processing the selected one or more data records using the data processing application.