Asset repository hub

• US 7,747,480 B1
• Filed: 03/28/2007
• Issued: 06/29/2010
• Est. Priority Date: 03/31/2006
• Status: Active Grant

• Current Assignee: SAS Institute Incorporated
• Original Assignee: SAS Institute Incorporated
• Inventors: Gidley, Scott Aron, Agresta, Ronald Jeffrey, Rineer, Brian Carl, Novozhilov, Alexander A.
• Primary Examiner(s): Fischer; Andrew J.
• Assistant Examiner(s): Dega; Murali K
• Application Number:
  US11/692,433
• Time in Patent Office: 1,189 Days
• Field of Search: 705 1- 79, 707/100
• US Class Current: 705/28
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders

Abstract

Systems and methods for managing data assets in an enterprise computing environment are provided. Data assets associated with a plurality of source systems operating within the enterprise computing environment may be registered with an asset repository hub. The asset repository hub receives a registration request from a source system for a particular data asset and determines whether the data asset is unique within the enterprise computing environment. If the data asset is unique, then the data asset is stored at the asset repository hub and a unique enterprise identifier is associated with the data asset. In determining whether the asset is unique, the asset repository hub may cleanse the data asset against a set of business rules; generate a plurality of match codes that describe the content of the data asset; and cluster the generated match codes against clusters of previously generated match codes. Also provided herein is a mechanism for searching and locating data assets stored within the enterprise computing environment by submitting queries to the asset repository hub.

Claims

1. A method of managing data assets in an enterprise computing environment having a plurality of source systems storing a plurality of data assets, comprising:

transmitting, using one or more computer processors, a registration request from a source system to an asset repository hub including a master reference schema, the registration request including a reference to a data asset within the enterprise computing environment;

receiving the registration request at the asset repository hub;

using the received registration request to obtain the referenced data asset;

cleansing the referenced data asset against a set of business rules;

applying one or more matching algorithms to the cleansed referenced data asset to generate one or more match codes, wherein each matching algorithm includes a set of parameters, and wherein each generated match code includes string data configured for imprecise matching within the master reference schema, such that a plurality of match codes can be matched to a single data asset; and

clustering the generated one or more match codes against clusters of previously generated match codes to determine whether the cleansed referenced data asset is unique to the enterprise computing environment, wherein when the cleansed referenced data asset is unique, the cleansed referenced data asset is assigned a unique enterprise identifier and stored at the asset repository hub.

2. The method of claim 1, wherein when the data asset is not unique to the enterprise computing environment, checking the contents of the data asset against an enterprise version of the data asset stored in the asset repository hub to determine if the data asset includes additional data to the enterprise version;

and

when the data asset includes additional data, determining whether to add the additional data to the enterprise version stored at the asset repository.

3. The method of claim 1, further comprising:

validating the data asset against a set of validation rules; and

when the data asset does not meet the set of validation rules, denying the registration request.

4. The method of claim 1, wherein transmitting comprises:

transmitting a web service call from the source system to the asset repository hub, the asset repository hub providing a web service interface for receiving registration requests over a network connection.

5. The method of claim 1, wherein cleansing further comprises:

comparing the form and content of data attributes of the data asset against the set of business rules; and

transforming the data attributes to be compatible with the set of business rules.

6. The method of claim 1, wherein generating the one or more match codes further comprises:

associating one or more match definitions with string data from the data asset; and

for each match definition, parsing the string data into a plurality of tokens.

7. The method of claim 6, further comprising:

associating a token value for each of the plurality of tokens.

8. The method of claim 7, further comprising:

determining which of the tokens are significant; and

selecting the most significant tokens for further processing.

9. The method of claim 8, further comprising:

generating one or more match values based upon the most significant token values; and

encoding the one or more match values to form the one or more match codes that describe the content of the data asset.

10. The method of claim 9, wherein generating one or more match values further comprises:

normalizing the significant tokens by transforming the significant token values against a set of normalization rules;

identifying and eliminating any statistically insignificant token values from the normalized significant tokens;

standardizing the significant tokens by transforming the significant token values against one or more standardization definitions; and

phonetically reducing the significant token values.

11. The method of claim 9, further comprising:

concatenating one or more of the one or more match values prior to forming the one or more match codes.

12. The method of claim 9, wherein the one or more match codes are generated using portions of the one or more match values, and wherein a portion of a match value can include the entire match value.

13. The method of claim 12, wherein the amount of the portions of the match values used to generate the one or more match codes is responsive to a match code sensitivity setting.

14. The method of claim 13, wherein when the match code sensitivity setting is set to a relatively high value, greater portions of each of the match values are used to generate the one or more match codes.

15. The method of claim 1, further comprising:

storing location information along with the referenced data asset, the location information indicating the source system where the referenced data asset is stored.

16. The method of claim 1, wherein the clustering step further comprises:

determining whether the generated one or more match codes are distinct from the clusters of previously generated match codes; and

when the generated one or more match codes are distinct, assigning a unique cluster ID to the generated one or more match codes, the unique cluster ID indicating that the data asset is unique to the enterprise computing environment.

17. The method of claim 16, wherein determining further comprises:

loading a cluster state file into memory comprising the clusters of previously generated match codes and one or more previously generated cluster IDs associated with the previously generated match codes; and

hashing the generated one or more match codes into the cluster state file.

18. The method of claim 17, further comprising:

determining if the generated one or more match codes are already stored in the cluster state file, and when the generated one or more match codes are already stored in the cluster state file, associating the cluster ID of the previously generated match codes with the data asset that generated the one or more match codes.

19. The method of claim 18, further comprising:

analyzing the cluster state file to determine whether a generated match code is stored in more than one location in the cluster state file, and when the generated match code is stored in more than one location, associating a lowest cluster ID with the generated match code.

20. The method of claim 19, further comprising:

determining whether the hashing results in overlapping cluster IDs within the cluster state file, and when the hashing results in overlapping cluster IDs within the cluster state file, minimizing the number of unique cluster IDs maintained in the file by collapsing the cluster state file.

21. The method of claim 1, further comprising:

transmitting a search request, including an asset description search criteria, from another source system to the asset repository hub, the search request seeking to locate a data asset within the enterprise computing environment;

receiving the search request at the asset repository hub;

cleansing the asset description search criteria provided along with the search request;

generating one or more match codes based upon the asset description search criteria that describe the data asset being searched for; and

clustering the generated one or more match codes against clusters of previously generated match codes to determine whether the data asset being searched for is registered within the enterprise computing environment; and

when the data asset is registered within the enterprise computing environment, transmitting the data asset information back to the source system generating the search request.

22. A method of managing data assets in an enterprise computing environment having a plurality of source systems storing a plurality of data assets, comprising:

receiving, using one or more computer processors, a registration request at an asset repository hub including a master reference schema, the registration request including a reference to a data asset within the enterprise computing environment;

using the received registration request to obtain the referenced data asset;

cleansing the referenced data asset against a set of business rules;

applying one or more matching algorithms to the cleansed referenced data asset to generate one or more match codes, wherein each matching algorithm includes a set of parameters, and wherein each generated match code includes string data configured for imprecise matching within the master reference schema, such that a plurality of match codes can be matched to a single data asset; and

clustering the generated one or more match codes against clusters of previously generated match codes to determine whether the cleansed referenced data asset is unique to the enterprise computing environment;

determining that the cleansed referenced data asset is unique; and

assigning the cleansed referenced data asset a unique enterprise identifier and storing the cleansed referenced data asset at the asset repository hub.

23. A system, comprising:

one or more processors;

one or more computer-readable storage mediums containing software instructions executable on the one or more processors to cause the one or more processors to perform operations including;

receiving, using one or more computer processors, a registration request at an asset repository hub including a master reference schema, the registration request including a reference to a data asset within the enterprise computing environment;

using the received registration request to obtain the referenced data asset;

cleansing the referenced data asset against a set of business rules;

applying one or more matching algorithms to the cleansed referenced data asset to generate one or more match codes, wherein each matching algorithm includes a set of parameters, and wherein each generated match code includes string data configured for imprecise matching within the master reference schema, such that a plurality of match codes can be matched to a single data asset; and

clustering the generated one or more match codes against clusters of previously generated match codes to determine whether the cleansed referenced data asset is unique to the enterprise computing environment;

determining that the cleansed referenced data asset is unique; and

assigning the cleansed referenced data asset a unique enterprise identifier and storing the cleansed referenced data asset at the asset repository hub.