Targeted multi-dimension data extraction for real-time analysis

• US 9,208,453 B2
• Filed: 02/15/2010
• Issued: 12/08/2015
• Est. Priority Date: 02/13/2009
• Status: Active Grant

• Current Assignee: PayPal, Inc. (PayPal Holdings, Inc.)
• Original Assignee: PayPal, Inc. (PayPal Holdings, Inc.)
• Inventors: Sinha, Ayon, Chakraborty, Rabi
• Primary Examiner(s): Shmatov, Alexey
• Application Number:
  US12/705,940
• Publication Number:
  US 20100223244A1
• Time in Patent Office: 2,122 Days
• Field of Search: 707/602, 707/604, 707/607, 707/705, 707/736, 707/770, 707/776, 707/802, 707/826, 707/E17.002, 707/E17.014, 707/E17.032, 707/E17.044, 707/E17.094, 707/999.01, 707/999.101
• US Class Current: 1/1
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/282 Hierarchical databases, e.g. IMS, LDAP data stores or Lotus Notes
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: G06Q 10/00 Administration; Management
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising

Abstract

Methods and systems for extracting targeted data for real-time reporting are discussed. In an example, a system can include a data store, a server, and a denormalized database. The data store can maintain data created by an application. The server can be communicatively coupled to the data store. The server can include a data extraction module and a data compression module. The data extraction module can extract a subset of the data stored in the data store according to an extraction scheme. The data compression module can compress the extracted subset of the data into a set of aggregated key value pairs. The denormalized database can store the aggregated key value pairs.

Claims

1. A system for extracting targeted transactional data for real-time reporting, the system comprising:

a data store to maintain data created by an application;

a server communicatively coupled to the data store, the server including one or more processors to execute the following modules;

a data extraction module to extract a subset of the data stored in the data store according to an extraction scheme, the extraction scheme including,

a computation block defining parameters for how data attributed to an individual user of the application will be extracted from the data maintained in the data store;

a list of tracked events, each tracked event identifying a specific operation performed by a user while interacting with the application and stored within the data store;

a success metric defining an operation performed by a user within the application and stored within the data store; and

a target dimension defining a programmable characteristic of the application;

wherein the data extraction module is configured to;

filter the data according to the computation block to obtain data attributed to the individual user;

extract events from the data attributed to the individual user based on the list of tracked events; and

identify within the extracted events from the data attributed to the individual user operations defined by the success metric;

attribute each of the identified operations to at least one of the extracted events from the data attributed to the individual user;

a data compression module to compress the attributed extracted subset of the data into a set of aggregated key value pairs, each of the aggregated key value pairs including a key defined by the target dimension and an associated value from success metric attribution, the target dimension representing a unique configuration of the application; and

a denormalized database to store the aggregated key value pairs.

2. The system of claim 1, including a report generation module to generate reports from the aggregated key value pairs stored in the denormalized database.

3. The system of claim 1, wherein the computation block includes a user session with a start defined by the user logging into the online application and an end defined by one of the following events:

the user logging out of the online application;

the user navigating away from the online application;

or

the user becoming inactive for a pre-determined amount of time.

4. The system of claim 1, wherein the list of tracked events can include one or more of the following tracked events:

viewing a page;

entering a search request;

uploading content;

downloading content;

entering a comment;

or

entering a tag.

5. The system of claim 4, wherein the success metric is selected from the list of tracked events.

6. The system of claim 1, wherein the success metric comprises a user action including one or more of:

completing a purchase transaction;

uploading content;

downloading content;

entering payment information;

adding an item to a virtual shopping cart;

or

bidding on an auction listing.

7. The system of claim 1, wherein the target dimension has a characteristic comprising one or more of:

a page identifier;

a configuration identifier;

a search term;

a keyword;

a sort order;

or

a category identifier.

8. The system of claim 1 wherein the extracted subset of the data includes a plurality of success metrics and a plurality of associated target dimensions,

wherein the data compression module aggregates the plurality of success metrics into a set of aggregated key value pairs, and

wherein each key represents a unique combination of the associated target dimensions and the value presents an aggregation of the plurality of success metrics.

9. The system of claim 1, wherein attributing each of the identified operations comprises:

attributing each of the identified operations to at least one of the extracted events from the data attributed to the individual user, wherein the at least one of the extracted events from the data attributed to the individual user is a parent of the identified operations within a logical session of interaction of the user.

10. The system of claim 1, wherein the operation defined by the success metric is one from the following group:

a product purchase, a service purchase, a photo upload, and an electronic funds transfer.

11. A computer-implemented method for extracting targeted transactional data, the method comprising:

receiving a definition of an extraction scheme to extract a subset of data maintained in a data store, the extraction scheme including;

a computation block defining a logical session of interaction by a user with an application;

a target dimension defining a programmable characteristic of the application;

a success metric defining an operation performed by the user within the application and stored within the transactional data store; and

a list of tracked events, each tracked event identifying a specific operation performed by a user while interacting with the application and stored within the data store;

accessing, using one or more processors, a plurality of transaction records within the data store according to the extraction scheme to generate a plurality of target dimension combinations associated with the success metric, wherein generating the plurality of target dimension combinations includes;

filtering the data according to the computation block to obtain data attributed to the individual user,

extracting events from the data attributed to the individual user based on the list of tracked events,

identifying within the extracted events from the data attributed to the individual user operations defined by the success metric, and

attributing each of the identified operations to at least one of the extracted events from the data attributed to the individual user and wherein each of the plurality of target dimension combinations is associated with one or more of the attributed extracted events from the data attributed to the individual user;

aggregating, using the one or more processors, the plurality of target dimension combinations across all of the attributed extracted events from the data attributed to the individual user into key value pairs, wherein a unique target dimension combination represents a key and an aggregation of the success metric represents the value; and

storing the key value pairs in a denormalized database.

12. The computer-implemented method of claim 11,

wherein the accessing the plurality of transaction records uses the plurality of tracked events to reduce the data in the data store prior to extracting the plurality of target dimension combinations and success metric.

13. The computer-implemented method of claim 12, wherein the list of tracked events can include one or more of:

viewing a page;

entering a search request;

uploading content;

downloading content;

entering a comment;

or

entering a tag.

14. The computer-implemented method of claim 13, wherein the success metric is selected from the list of tracked events.

15. The computer-implemented method of claim 11, wherein the aggregation of success metrics includes counting the number of success metrics associated with a unique target dimension combination.

16. The computer-implemented method of claim 11, wherein the aggregation of success metrics includes averaging the success metrics over a time period.

17. The computer-implemented method of claim 11, wherein the aggregation of success metrics is performed according to an attribution plan, the attribution plan defining how the success metric will be scored for a unique target dimension combination.

18. The computer-implemented method of claim 11, wherein the computation block includes a user session with a start defined by a user logging into the online application and an end defined by an event comprising:

the user logging out of the online application;

the user navigating away from the online application;

or

the user becoming inactive for a pre-determined amount of time.

19. The computer-implemented method of claim 11, wherein the success metric includes one or more of:

completing a purchase transaction;

uploading content;

downloading content;

entering payment information;

adding an item to a virtual shopping cart;

or

bidding on an auction listing.

20. The computer-implemented method of claim 11, wherein a target dimension of the plurality of target dimension combinations has one or more characteristics comprising one or more of:

a page identifier;

a configuration identifier;

a search term;

a keyword;

a sort order;

or

a category identifier.

21. A tangible non-transitory computer-readable medium storing instructions, which when executed on one or more processors cause the one or more processors to perform operations to:

receive a definition of an extraction scheme to extract a subset of data maintained in a data store, the extraction scheme including;

a computation block defining a logical session of interaction by a user with an application;

a target dimension defining a programmable characteristic of the application;

a success metric defining an operation performed by the user within the application and stored within the transactional data store; and

a list of tracked events, each tracked event identifying a specific operation performed by a user while interacting with the application and stored within the data store;

access a plurality of transaction records within the data store according to the extraction scheme to generate a plurality of target dimension combinations associated with the success metric, wherein generating the plurality of target dimension combinations includes;

filtering the data according to the computation block to obtain data attributed to the individual user;

extracting events from the data attributed to the individual user based on the list of tracked events,

identifying within the extracted events from the data attributed to the individual user operations defined by the success metric, and

attributing each of the identified operations to at least one of the extracted events from the data attributed to the individual user and wherein each of the plurality of target dimension combinations is associated with one or more of the attributed extracted events from the data attributed to the individual user;

Aggregate the plurality of target dimension combinations across all of the attributed extracted events from the data attributed to the individual user into key value pairs, wherein a unique target dimension combination represents a key and an aggregation of the success metric represents the value; and

store the key value pairs in a denormalized database.