Adapting data quality rules based upon user application requirements

• US 10,331,635 B2
• Filed: 03/09/2016
• Issued: 06/25/2019
• Est. Priority Date: 06/30/2011
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Dani, Mohan N., Gupta, Anuj, Mohania, Mukesh K., Venkatachaliah, Girish
• Primary Examiner(s): Syed, Farhan M
• Application Number:
  US15/064,780
• Publication Number:
  US 20160188640A1
• Time in Patent Office: 1,203 Days
• CPC Class Codes: G06F 16/215 Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/252 between a Database Management System and a front-end application
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence

Abstract

During application of data quality rules to a data set obtained from a data source, data is retrieved from the data source along with a common set of rules configured to format the retrieved data in a manner in accordance with one or more predefined data quality rules of the common set of rules. At least one predefined data quality rule is adjusted utilizing at least one editable widget to form a modified set of data quality rules adapted for use with a specified application. The modified set of data quality rules is applied to the retrieved data.

Claims

1. A system for applying data quality rules to a data set obtained from at least one data source, the system comprising:

a client including at least one processor configured to;

generate an editable widget, the generating comprising the at least one processor being further configured to;

select attributes from data tables of the at least one data source,

discover patterns corresponding to the selected attributes,

determine which rules of a common set of rules to apply to the widget based on the discovered patterns, and

group the determined rules of the common set of rules with corresponding selected attributes for selection and modification by editing the widget;

receive, in response to a query, data retrieved from the at least one data source and the common set of rules retrieved from a rules database, the common set of rules including one or more predefined data quality rules that are fixed and applicable to extracting and formatting data in a same manner for each client requesting data from any of the at least one data source;

modify the common set of rules to form a modified set of data quality rules for the editable widget in accordance with requirements of a specified application, residing on the client, during operation of the specified application, the modified set of data quality rules being different from the common set of rules, the common set of rules being modified by editing the editable widget residing on the client, the editable widget including a user interface having adjustable control objects that facilitate making adjustments to the received data to further adapt the received data for use with the specified application based upon the requirements; and

apply, by the client, the modified set of data quality rules to the retrieved data to output the retrieved data in accordance with the modified set of data quality rules for the specified application executing on the client.

2. The system of claim 1, wherein the editable widget is associated with an entity that defines a category of data records, and the at least one processor is further configured to apply the modified set of data quality rules to records or columns of the retrieved data.

3. The system of claim 1, wherein the editable widget facilitates modification of a predefined data quality rule applied to one attribute associated with the editable widget separately from another attribute associated with the editable widget.

4. The system of claim 1, wherein the editable widget comprises a first graphical user interface listing attributes associated with the editable widget and at least one additional graphical user interface comprising at least one editable field that facilitates modification of a predefined data quality rule.

5. The system of claim 1, further comprising:

a source system providing the data retrieved from the at least one data source and the common set of rules to a plurality of clients.

10. The system of claim 1, wherein the user interface of the editable widget includes at least one graphical user interface menu having the adjustable control objects including at least one of toggle on/off buttons, check boxes, selectable menus, icons and fillable fields.

12. The system of claim 1, wherein the at least one processor being configured to generate an editable widget further comprises the at least one processor being configured to:

parameterize each of the determined rules based upon rule artifacts.

6. A computer program product for applying data quality rules to a data set obtained from at least one data source, the computer program product comprising:

a computer readable storage medium having computer readable program code embodied therewith such that when a client executes the computer readable program code, the client is configured to;

generate an editable widget, the generating comprising the client being further configured to;

select attributes from data tables of the at least one data source,

discover patterns corresponding to the selected attributes by a context based pattern discovery method,

determine which rules of a common set of rules to apply to the widget based on the discovered patterns, and

group the determined rules of the common set of rules with corresponding selected attributes for selection and modification by editing the widget;

receive, in response to a query, data retrieved from the at least one data source along with the common set of rules retrieved from a rules database, the common set of rules including one or more predefined data quality rules that are fixed and applicable to extracting and formatting data in a same manner for each client requesting data from any of the at least one data source;

modify the common set of rules to form a modified set of data quality rules for the editable widget in accordance with requirements of a specified application, residing on the client, during operation of the specified application, the modified set of data quality rules being different from the common set of rules, the common set of rules being modified by editing the editable widget residing on the client, the editable widget including a user interface having adjustable control objects that facilitate making adjustments to the received data to further adapt the received data for use with the specified application based upon the requirements; and

apply, by the client, the modified set of data quality rules to the retrieved data to output the retrieved data in accordance with the modified set of data quality rules for the specified application executing on the client.

7. The computer program product of claim 6, wherein the editable widget is associated with an entity that defines a category of data records, and the modified set of data quality rules is applied by the computer readable program code to records or columns of the retrieved data.

8. The computer program product of claim 6, wherein each editable widget facilitates modification of a predefined data quality rule applied to one attribute associated with the editable widget separately from another attribute associated with the editable widget.

9. The computer program product of claim 6, wherein the editable widget comprises a first graphical user interface listing attributes associated with the editable widget and at least one additional graphical user interface comprising at least one editable field that facilitates modification of a predefined data quality rule.

11. The computer program product of claim 6, wherein the editable widget includes at least one graphical user interface menu having the adjustable control objects including at least one of toggle on/off buttons, check boxes, selectable menus, icons and fillable fields.

13. The computer program product of claim 6, wherein the client being configured to generate an editable widget further comprises the client being configured to:

parameterize each of the determined rules based upon rule artifacts.