Data decay management

• US 8,452,733 B2
• Filed: 10/17/2008
• Issued: 05/28/2013
• Est. Priority Date: 10/17/2008
• Status: Active Grant

• Current Assignee: Oracle International Corporation (Oracle Corporation)
• Original Assignee: Oracle International Corporation (Oracle Corporation)
• Inventors: Chan, Cynthia Mee Yee, Jain, Chetana, Li, Wenxin, Song, Haidong, Tahiliani, Manoj
• Primary Examiner(s): SYED, FARHAN M
• Application Number:
  US12/253,788
• Publication Number:
  US 20100114836A1
• Time in Patent Office: 1,684 Days
• US Class Current: 707/662
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06F 16/2322 using timestamps

Abstract

Embodiments presented herein provide systems and methods for managing data decay. A system is provided for maintaining metadata about data attributes or relationships between data. A data decay engine can read the metadata and perform a decay calculation. The type of decay calculation can be associated with the type of data or be determined from user inputs. The decay engine can provide a score as to the staleness of the data. An update engine can determine specific data attributes that may require updating. The update engine may be able to update the data from external data sources.

Claims

1. A computer-implemented method for determining decay for an item of data in a database, the method comprising:

retrieving, by a computer system, a date/time stamp from the database for each of a plurality of items of data previously stored in the database, wherein the date/time stamp indicates a last update for a record storing the item of data in the database;

calculating, by the computer system, a time from the last update of each of the plurality of items of data previously stored in the database based on the date/time stamp;

retrieving, by the computer system, one or more decay rules;

calculating, by the computer system, an individual decay score for each of the plurality of items of data previously stored in the database based on the time from the last update of each of the plurality of items and according to the one or more decay rules for a data type of the items of data, the decay score indicating a relative staleness of each of the plurality of items based on a data type of each of the plurality of items;

reporting, by the computer system, the individual decay score for each of the plurality of items of data to the user interface;

determining, by the computer system, whether the individual decay score for each of the plurality of items of data crosses a threshold;

setting, by the computer system, an out-of-date flag for an item of data when the decay score for that item of data crossed the threshold;

storing, by the computer system, the individual decay score for each of the plurality of items of data as a decay metric;

retrieving, by the computer system, the decay metric for each of the plurality of items of data;

retrieving, by the computer system, a threshold;

determining, by the computer system, whether any of the decay metrics cross the threshold;

when the decay metric for an item of data crosses the threshold, retrieving, by the computer system, an update for the item of data;

storing, by the computer system, the update for the item of data in the database; and

updating, by the computer system, the date/time stamp for the item of data in the database to indicate a date/time of the storing of the update.

2. The method as defined in claim 1, further comprising:

determining, by the computer system, an overall decay score for all of the database based on the decay scores of each of the plurality of items of data in the database; and

reporting, by the computer system, the decay score for all of the database.

3. The method as defined in claim 1, wherein the one or more decay rules includes a decay calculation.

4. The method as defined in claim 3, wherein the decay calculation is one of a linear function, logarithmic function, or exponential function.

5. The method as defined in claim 2, wherein the individual decay score for each of the plurality of items of data and the overall decay score for all of the database are calculated periodically.

6. The method as defined in claim 1, further comprising calculating, by the computer system, a decay score for one or more relationships between two or more items of data.

7. A computer system for managing data decay in a database, comprising:

a processor; and

a memory device in communication with the processor and encoded with a database program that causes the processor to execute;

the database;

a decay engine in communication with the database, the decay engine operable to cause the processor to determine decay in the database by;

retrieving a date/time stamp from the database for each of a plurality of items of data previously stored in the database, wherein the date/time stamp indicates a last update of a record storing the item of data in the database;

calculating a time from the last update of each of the plurality of items of data previously stored in the database based on the date/time stamp;

retrieving one or more decay rules;

calculating an individual decay score for each of the plurality of items of data previously stored in the database based on the time from the last update of each of the plurality of items and according to the one or more decay rules for a data type of the items of data, the decay score indicating a relative staleness of each of the plurality of items based on a data type of each of the plurality of items;

storing the individual decay score for each of the plurality of items of data as a decay metric for the item of data;

reporting the individual decay score for each of the plurality of items of data;

determining an overall decay score for all of the database based on the individual decay scores of each of the plurality of items of data previously stored in the database; and

storing the overall decay score for all of the database; and

an update engine in communication with the database, the update engine operable to cause the processor to update decayed data stored in the database by;

retrieving the decay metric for each of the plurality of items of data;

retrieving a threshold;

determining whether any of the decay metrics cross the threshold;

when the decay metric for an item of data crosses the threshold, setting an out-of-date flag for an item of data and retrieving an update for the item of data previously stored in the database;

storing the update for the item of data in the database; and

updating the date/time stamp for the item of data in the database to indicate a date/time of the storing of the update.

8. The computer system as defined in claim 7, further comprising:

a decay rules database in communication with the decay engine, the decay rules database operable to store the one or more decay rules.

9. The computer system as defined in claim 8, further comprising a database engine, the database engine in communication with the decay engine and the update engine, the database engine receiving one or more requests from the decay engine and the update engine, the database engine providing one or more data files to the decay engine and the update engine.

10. The computer system as defined in claim 7, wherein the update engine is in communication with one or more external databases, the update engine operable to receive the update for the item of data from the one or more external databases.

11. The computer system as defined in claim 7, wherein a database administrator creates the one or more decay rules.

12. The computer system as defined in claim 7, wherein the database comprises one or more items of metadata, the metadata comprising:

a date stamp field, the date stamp field storing a date for the date/time stamp, wherein the date is a date of a last update;

a time stamp field, the time stamp field storing a time for the date/time stamp, wherein the time is a time of a last update; and

a decay metric field, the decay metric field storing the decay score as a decay metric.

13. The computer system as defined in claim 12, the metadata further comprising:

an out-of-date flag field, the out-of-date flag field storing an out-of-date flag that represents to the update engine that the item of data needs to be updated; and

an update flag field, the update flag field storing an update flag that represents to the update engine that a user wants the item of data to be updated when the out-of-date flag is set.

14. The computer system as defined in claim 13, wherein one or more of the metadata is associated with one or more items of data and one or more relationships between two or more items of data.

15. A non-transitory computer-readable memory having stored thereon a sequence of instructions which, when executed by a processor, cause the processor to update decayed data in a database by:

retrieving a date/time stamp from the database for each of a plurality of items of data previously stored in the database, wherein the date/time stamp indicates a last update for a record storing the item of data in the database;

calculating a time from the last update of each of the plurality of items of data previously stored in the database based on the date/time stamp;

retrieving one or more decay rules;

calculating an individual decay score for each of the plurality of items of data previously stored in the database based on the time from the last update of each of the plurality of items and according to the one or more decay rules for a data type of the items of data, the decay score indicating a relative staleness of each of the plurality of items based on a data type of each of the plurality of items;

storing the decay score for each of the plurality of items of data as a decay metric for the item of data;

reporting the individual decay score for each of the plurality of items of data;

determining an overall decay score for all of the database based on the individual decay scores of each of the plurality of items of data previously stored in the database;

storing the decay score for all of the database;

retrieving the decay metric for a first item of data previously stored in the database;

retrieving a threshold associated with the item of data;

determining whether the decay metric crosses the threshold;

when the decay metric crosses the threshold, setting an out-of-date flag for an item of data and retrieving an update for the item of data;

when the decay metric does not cross the threshold, retrieving a decay metric for a second item of data;

storing the update for the first item of data in the database; and

updating the date/time stamp for the item of data in the database to indicate a date/time of the storing of the update.

16. The computer-readable memory as defined in claim 15, further comprising setting an out-of-date flag when the decay metric crosses the threshold.

17. The computer-readable memory as defined in claim 16, further comprising:

determining whether an update flag is set for the first item of data; and

determining whether an update is requested on the first item of data, wherein the update is requested when the update flag is set.

18. The computer-readable memory as defined in claim 15, further comprising retrieving the update from one of one or more external databases.