Method and system for model-based replication of data

• US 7,487,191 B2
• Filed: 06/10/2005
• Issued: 02/03/2009
• Est. Priority Date: 06/10/2005
• Status: Expired due to Fees

• Current Assignee: ServiceNow Incorporated
• Original Assignee: International Business Machines Corporation
• Inventors: Yeh, Danny L., Konuru, Ravindranath, Purakayastha, Apratim, Castro, Paul C., Giraud, Frederique A.
• Primary Examiner(s): Rones; Charles
• Assistant Examiner(s): CAO, PHUONG THAO
• Application Number:
  US11/150,072
• Publication Number:
  US 20060282482A1
• Time in Patent Office: 1,334 Days
• Field of Search: 707/204, 707/100, 707/101, 707/102, 707/10, 709/248, 709/205, 700/29
• US Class Current: 1/1
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/273 Asynchronous replication or reconciliation
• CPC Class Codes: Y10S 707/99942 Manipulating data structure, e.g. compression, compaction, compilation
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface
• CPC Class Codes: Y10S 707/99955 Archiving or backup

Abstract

Techniques for replicating data in a system having a data tier with a plurality of data sources and an application tier with a plurality of applications are provided. Such techniques can include creating an instance of an information model of the data that is specified by a schema. The model can be created in a form that is suitable for manipulation by at least one of the applications and that forms at least a portion of a model tier that is intermediate the data tier and the application tier. A replica of an instance of the information model can be created in a form suitable for remote transmission, and a consistency policy can be obtained that specifies a level of consistency required between the instance of the information model and the replica.

Claims

1. A computer implemented method for replicating data in a system having a data tier with a plurality of data sources and an application tier with a plurality of applications, said method comprising the steps of:

creating an instance of an information model of the data specified by a schema, said model instance being created in a form suitable for manipulation by at least one of the applications of the application tier and forming at least a portion of a model tier intermediate the data tier and the application tier, said model tier comprising a virtual store interface through which said at least one of the applications accesses said instances;

creating a replica of said instance of said information model in a form suitable for remote transmission; and

obtaining a consistency policy that specifies a level of consistency required between said instance of said information model and said replica;

wherein said data tier comprises;

a store layer comprising said plurality of data sources; and

an aggregation layer intermediate said store layer and said model tier, said aggregation layer comprising a plurality of unified store model interfaces corresponding to respective ones of said plurality of data sources;

wherein said store layer comprises a plurality of store-specific information models;

wherein said aggregation layer comprises an elementary information model; and

wherein said model tier comprises a self-describing high-level information model.

2. The method of claim 1, further comprising the additional step of obtaining said schema, said schema specifying said information model via application-level requirements, said schema describing at least property-level content and structure to be conveyed by said model.

3. The method of claim 2, wherein said schema further describes constraints on said content to be conveyed by said model.

4. The method of claim 2, wherein said instance of said information model is created on a fast computer, said method further comprising the additional steps of:

copying said replica to at least a second computer; and

identifying said model instance as a model instance that is to be copied;

and wherein;

said step of creating said replica comprises marshalling said model instance into said form suitable for said remote transmission; and

said step of copying said replica comprises un-marshalling said model instance, at said second computer, from said form suitable for said remote transmission.

5. The method of claim 2, further comprising the additional steps of:

repeating said step of creating an instance in order to obtain a plurality of model instances; and

providing a virtual store view abstraction of said plurality of said model instances wherein said model instances are logically grouped together in sets, said virtual store view abstraction being configured to allow at least some of the applications to perform create, retrieve, update and delete operations thereon.

6. The method of claim 5, wherein said virtual store view abstraction is configured to permit access via model instance retrieval, operation on said model instance, and model instance storage commitment.

7. The method of claim 5, wherein said virtual store view abstraction is configured to permit access via model and operation specification and store acknowledgement.

8. The method of claim 5, wherein said step of providing said virtual store view abstraction in turn comprises:

organizing said model instances into first level categories based at least in part on said property-level content and structure;

organizing said first level categories into multiple higher level categories based on requirements of at least some of the applications; and

organizing said multiple higher level categories into a single administrative domain for management purposes.

9. The method of claim 5, further comprising the additional steps of:

repeating said step of creating a replica in order to obtain a plurality of replicas, at least some of said replicas originating from different ones of the data sources; and

managing at least some of said replicas on a first computer.

10. The method of claim 9, wherein said managing step in turn comprises:

managing local resources associated with said first computer to perform at least one of temporarily and persistently storing at least some of said replicas; and

applying said consistency policy to maintain desired consistency between said replicas and corresponding ones of said plurality of model instances;

said method further comprising the additional step of specifying said consistency policy as an application-independent consistency policy, said step of specifying said consistency policy in turn comprising;

determining whether one of strong consistency requirements and weak consistency requirements apply for a given one of said model instances, to obtain relevant consistency requirements;

mapping said relevant consistency requirements to consistency primitives; and

encoding said relevant consistency requirements into a policy file;

wherein said step of applying said consistency policy in turn comprises;

tracking changes made on those of said replicas corresponding to a given one of said model instances; and

interfacing with at least one computer remote from said first computer to perform at least one of;

(i) information exchange pertaining to at least one of said model instances, (ii) resolution of conflicting updates pertaining to at least one of said model instances, and (iii) making a given one of said replicas consistent with a corresponding one of said plurality of model instances.

11. The method of claim 2, further comprising the additional steps of:

creating said schema by encoding symbols representative of said schema into a machine-readable format;

generating model-tier compatible model code from said machine-readable format of said schema, said model code being generated in a form to facilitate creation of said instance of said information model; and

accessing a plurality of additional schemas, in heterogeneous machine-readable formats, for generation of model-tier compatible code for creation of instances of additional information models.

12. The method of claim 11, further comprising the additional step of repeating said step of generating said model code for said additional information models;

wherein said step of generating said code, for a given one of said information models, comprises;

obtaining a user-defined specification specifying generation of said code; and

generating said code according to said specification, said code being generated in a form to ensure compliance of instances of said given one of said information models with a corresponding one of said schemas.

13. The method of claim 2, further comprising the additional step of assigning to said model instance globally unique identifier'"'"'s specified by at least one of the applications, said identifiers pertaining to at least one of said property-level content and structure of said model described by said schema.

14. The method of claim 13, wherein said assigning step in turn comprises:

creating a data namespace based at least in part on;

(i) said schema and (ii) location of said data for which said schema specifies said information model thereof;

specifying portions of said model instance to be uniquely identified;

generating said globally unique identifiers, based on said data namespace, for said portions; and

encoding said globally unique identifiers into said model instance in a manner to be accessible to at least one of (i) system-level processes, and (ii) at least one of the applications.

15. The method of claim 13, wherein said step of creating said replica in turn comprises:

creating an empty instance of a model having a same schema type as said instance of said information model;

copying at least said property-level content and applicable ones of said globally unique identifiers from said instance of said information model to said empty instance to form said replica of said instance.

16. The method of claim 2, further comprising the additional steps of

providing a high-level interface in a form for at least some of the applications to specify storage requirements to persistent storage;

encoding said instance of said model in a format suitable for storage; and

dissembling said instance of said model into data suitable for storage across at least two of the data sources.