Storing and processing data organized as flexible records

• US 9,767,126 B2
• Filed: 08/17/2016
• Issued: 09/19/2017
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Excalibur Acquisitionco LLC
• Original Assignee: Tactile, Inc.
• Inventors: Prasanna, Dhananjay, Kent, Matthew, Shank, Kyle Glendon, Seebaldt, Kurtis L.
• Primary Examiner(s): Corrielus, Jean M
• Application Number:
  US15/239,620
• Publication Number:
  US 20170004150A1
• Time in Patent Office: 398 Days
• Field of Search: 707610, 707741
• CPC Class Codes: G06F 16/211 Schema design and management
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/2358 Change logging, detection, and notification replication G06F16/27
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor

Abstract

A system and method are disclosed for persisting data received from disparate data sources having different internal schemas. In operation, a data processing engine aggregates related data received from the different data sources and organizes the aggregated data into flexible records. A flexible record is a composite of associated fields aggregated from a set of records received from one or more data sources. Each field associated with a flexible record includes data received from a particular data source and specifies the particular data source as the source of the data. Flexible records are stored in a storage repository, and each flexible record is associated with at least one user who accesses data via a client device.

Claims

1. A method for persisting data received from different data sources, the method comprising:

receiving, at a data aggregation server, data from a plurality of data sources having a plurality of schemas, the data including a plurality of data fields, and at least two of the data fields being structured according to different schema of the plurality of schema;

automatically, in response to receiving the data from the plurality of data sources;

aggregating a subset of the plurality of data fields that are relevant to an object stored by the data aggregation server;

generating a record identifier associated with the object;

generating a different insert event associated with each data field of the aggregated subset of data fields, each insert event including a value corresponding to the associated data field; and

storing each insert event in a different row of an operation log stored by the data aggregation server, each row including the record identifier, a field name corresponding to the data field associated with the insert event stored in the row, the value corresponding to the data field, and a timestamp of when the insert event occurred;

receiving, from a client device, a first data request for accessing a value corresponding to a data field of the aggregated subset of data fields;

in response to receiving the first data request, automatically transmitting, to the client device, the value included in the identified row in response to the first data request;

generating a modify event in a new row of the operation log, the new row including the record identifier, the field name corresponding to the data field associated with the modified value, a modified value, and a timestamp of when the modify event occurred;

identifying a row in the operation log that stores a most recent event associated with the data field corresponding to the modified value; and

responsive to identifying the row associated with the modify event as the most recent event, transmitting, to the client device, the modified value.

2. The method of claim 1, further comprising transmitting a modification update to one or more of the plurality of data sources, the modification update comprising the modified value.

3. The method of claim 1, wherein each of the plurality of data sources is one of a customer relationship management system or a human resource management system.

4. The method of claim 1, wherein each of the plurality of data sources is associated with a different schema of the plurality of schema.

5. The method of claim 1, wherein the object is one of a contact, a document, or a calendar item.

6. The method of claim 1, wherein each row in the operation log comprises a user identifier associated with a user associated with the data.

7. The method of claim 1, wherein identifying a row in the operation log that stores a most recent event comprises identifying a modify event of a plurality of modify events as a most recent event, each of the plurality of modify events comprising a different modified value for the data field.

8. The method of claim 7, wherein the identified modify event is identified based on timestamps of the plurality of modify events.

9. The method of claim 1, wherein each row in the operation log comprises an event type.

10. The method of claim 1, wherein each row in the operation log comprises a source identifier associated with one of the plurality of data sources.

11. A non-transitory computer readable medium comprising instructions that, when executed by a processor, cause the processor to:

receive, at a data aggregation server, data from a plurality of data sources having a plurality of schemas, the data including a plurality of data fields, and at least two of the data fields being structured according to different schema of the plurality of schema;

automatically, in response to receiving the data from the plurality of data sources;

aggregate a subset of the plurality of data fields that are relevant to an object stored by the data aggregation server;

generate a record identifier associated with the object;

generate a different insert event associated with each data field of the aggregated subset of data fields, each insert event including a value corresponding to the associated data field; and

store each insert event in a different row of an operation log stored by the data aggregation server, each row including the record identifier, a field name corresponding to the data field associated with the insert event stored in the row, the value corresponding to the data field, and a timestamp of when the insert event occurred;

receive, from a client device, a first data request for accessing a value corresponding to a data field of the aggregated subset of data fields;

in response to receiving the first data request, automatically transmit, to the client device, the value included in the identified row in response to the first data request;

generate a modify event in a new row of the operation log, the new row including the record identifier, the field name corresponding to the data field associated with the modified value, a modified value, and a timestamp of when the modify event occurred;

identify a row in the operation log that stores a most recent event associated with the data field corresponding to the modified value; and

responsive to identifying the row associated with the modify event as the most recent event, transmit, to the client device, the modified value.

12. The computer readable medium of claim 11, wherein the instructions further cause the processor to transmit a modification update to one or more of the plurality of data sources, the modification update comprising the modified value.

13. The computer readable medium of claim 11, wherein each of the plurality of data sources is one of a customer relationship management system or a human resource management system.

14. The computer readable medium of claim 11, wherein each of the plurality of data sources is associated with a different schema of the plurality of schema.

15. The computer readable medium of claim 11, wherein the object is one of a contact, a document, or a calendar item.

16. The computer readable medium of claim 11, wherein each row in the operation log comprises a user identifier associated with a user associated with the data.

17. The computer readable medium of claim 11, wherein the instructions for identifying a row in the operation log that stores a most recent event further comprise instructions that cause the processor to identify a modify event of a plurality of modify events as a most recent event, each of the plurality of modify events comprising a different modified value for the data field.

18. The computer readable medium of claim 17, wherein the identified modify event is identified based on timestamps of the plurality of modify events.

19. The computer readable medium of claim 11, wherein each row in the operation log comprises an event type.

20. The computer readable medium of claim 11, wherein each row in the operation log comprises a source identifier associated with one of the plurality of data sources.