System and method to measure effectiveness and consumption of editorial content

• US 10,600,089 B2
• Filed: 03/14/2014
• Issued: 03/24/2020
• Est. Priority Date: 03/14/2013
• Status: Active Grant

• Current Assignee: Oracle America, Inc. (Oracle Corporation)
• Original Assignee: Oracle America, Inc. (Oracle Corporation)
• Inventors: Fichter, Daniel E., Tsoufakis, Christopher R., Gomez, Nikki K., Rahman, Aniq, Goodhart, Jonah, Seiler, Michael Garrett, Agarwal, Suryansh, McConnell, Theodore V.
• Primary Examiner(s): Somers, Marc S
• Application Number:
  US14/213,360
• Publication Number:
  US 20170316092A1
• Time in Patent Office: 2,202 Days
• CPC Class Codes: G06F 16/954 Navigation, e.g. using categorised browsing
• CPC Class Codes: G06F 16/958 Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
• CPC Class Codes: G06Q 30/0254 based on statistics
• CPC Class Codes: G06Q 30/0255 based on user history
• CPC Class Codes: G06Q 30/0277 Online advertisement

Abstract

A system for analyzing user consumption of content on web pages or determining user engagement with content comprising a data collection script and an content-engagement-inference engine. The data collection script determines one or more content elements to be measured from a web page and collects description data associated with the one or more content elements. The description data describes information about the one or more content elements. The content-engagement-inference engine creates an inference or analytics result based on the description data. The inference or analytics result describes how the one or more content elements have been consumed by users and then provides the inference or analytics result for users to designated entities.

Claims

1. A processor-based method comprising:

identifying one or more content elements of a web page by;

retrieving a set of rules specific to a publisher of the web page; and

applying the set of rules to the web page to detect the one or more content elements;

creating a data collection script according to the set of rules, the data collection script being configured to collect description data relative to the one or more content elements while the web page is rendered by a web browser on a client device of a first user;

transmitting the data collection script to the web browser on the client device, the data collection script configured to be executable by the web browser;

receiving, from the client device, one or more communications transmitted in response to execution of the data collection script by the web browser, wherein the one or more communications comprise the description data collected at the client device, wherein the description data includes interaction data that describes interactions of the first user with the one or more content elements, and wherein the description data further includes metadata that identifies attributes associated with the one or more content elements;

generating indications that the one or more content elements were read, viewed, or otherwise consumed by the first user based on the description data;

generating indications of a level of engagement of the first user with the one or more content elements based on the description data;

learning how various indications correlate with one another to derive conclusions on behavior of the first user directed at the one or more content elements, wherein the learning comprises applying an algorithm to the various indications, and the various indications include the indications that the one or more content elements were read, viewed, or otherwise consumed and the indications of the level of engagement of the first user with the one or more content elements;

analyzing the indications that the one or more content elements were read, viewed, or otherwise consumed, the indications of the level of engagement of the first user with the one or more content elements, and the derived conclusions on the behavior of the first user to estimate;

(i) at least one content element of the one or more contents elements was consumed by the first user, and (ii) an extent to which the at least one content element was consumed by the first user; and

transmitting a level of client engagement with the at least one content element to a second user based on the estimate that;

(i) the at least one content element of the one or more contents elements was consumed by the first user, and (ii) the extent to which the at least one content element was consumed by the first user.

2. The processor-based method of claim 1, wherein:

the analyzing comprises deriving inferences on the behavior of the first user from the indications that the one or more content elements were read, viewed, or otherwise consumed and the indications of the level of engagement of the first user with the one or more content elements based on the set of rules specific to the publisher;

the set of rules define the inferences on the behavior of the first user based on the interactions of the first user with the one or more content elements; and

the analyzing further comprises estimating (i) the at least one content element of the one or more contents elements was consumed by the first user, and (ii) the extent to which the at least one content element was consumed by the first user based on the derived inferences and the derived conclusions.

3. The processor-based method of claim 2, wherein the set of rules determine the level of client engagement with the at least one content element by determining the first user was engaged at a first level when a cursor activity of a cursor included no movement for a period of time greater than a first threshold over the at least one content element when the at least one content element comprises a video presentation;

and

the set of rules determine the level of client engagement with the at least one content element by determining the first user was engaged at a second level when a cursor activity of the cursor included scrolling at a pace greater than a second threshold but slower than a third threshold over the at least one content element when the at least one content element comprises a textual presentation.

4. The processor-based method of claim 3, wherein the set of rules determine the level of client engagement with the one or more content elements by determining the first user was engaged at a third level when a cursor activity of the cursor included clicking on a navigational link embedded in the textual presentation.

5. The processor-based method of claim 1, wherein the set of rules comprises at least a div tag, a content type, a content boundary, a frame definition, or a headline identifier for the one or more content elements.

6. The processor-based method of claim 1, wherein the interactions of the first user with the one or more content elements comprises cursor movement that includes scrolling behavior, scrolling velocity, scrolling direction, clicking on a comment link, clicking on a sharing link, or clicking on a navigational link.

7. The processor-based method of claim 1, wherein the metadata comprises article titles, a page name, video titles, or content thumbnails.

8. A non-transitory computer readable medium storing instructions, which when executed one or more processors, cause the one or more processors to perform operations including:

identifying one or more content elements of a web page by;

retrieving a set of rules specific to a publisher of the web page; and

applying the set of rules to the web page to detect the one or more content elements;

creating a data collection script according to the set of rules, the data collection script being configured to collect description data relative to the one or more content elements while the web page is rendered by a web browser on a client device of a first user,

transmitting the data collection script to the web browser on the client device, the data collection script configured to be executable by the web browser;

receiving, from the client device, one or more communications transmitted in response to execution of the data collection script by the web browser, wherein the one or more communications comprise the description data collected at the client device, wherein the description data includes interaction data that describes interactions of the first user with the one or more content elements, and wherein the description data further includes metadata that identifies attributes associated with the one or more content elements;

generating indications that the one or more content elements were read, viewed, or otherwise consumed by the first user based on the description data;

generating indications of a level of engagement of the first user with the one or more content elements based on the description data;

learning how various indications correlate with one another to derive conclusions on behavior of the first user directed at the one or more content elements, wherein the learning comprises applying an algorithm to the various indications, and the various indications include the indications that the one or more content elements were read, viewed, or otherwise consumed and the indications of the level of engagement of the first user with the one or more content elements;

analyzing the indications that the one or more content elements were read, viewed, or otherwise consumed, the indications of the level of engagement of the first user with the one or more content elements, and the derived conclusions on the behavior of the first user to estimate;

(i) at least one content element of the one or more contents elements was consumed by the first user, and (ii) an extent to which the at least one content element was consumed by the first user; and

transmitting a level of client engagement with the at least one content element to a second user based on the estimate that;

(i) the at least one content element of the one or more contents elements was consumed by the first user, and (ii) the extent to which the at least one content element was consumed by the first user.

9. The non-transitory computer readable medium of claim 8, wherein:

the analyzing comprises deriving inferences on the behavior of the first user from the indications that the one or more content elements were read, viewed, or otherwise consumed and the indications of the level of engagement of the first user with the one or more content elements based on the set of rules specific to the publisher;

the set of rules define the inferences on the behavior of the first user based on the interactions of the first user with the one or more content elements; and

the analyzing further comprises estimating (i) the at least one content element of the one or more contents elements was consumed by the first user, and (ii) the extent to which the at least one content element was consumed by the first user based on the derived inferences and the derived conclusions.

10. The non-transitory computer readable medium of claim 9, wherein the set of rules determine the level of client engagement with the at least one content element by determining the first user was engaged at a first level when a cursor activity of a cursor included no movement for a period of time greater than a first threshold over the at least one content element when the at least one content element comprises a video presentation;

and

the set of rules determine the level of client engagement with the at least one content element by determining the first user was engaged at a second level when a cursor activity of the cursor included scrolling at a pace greater than a second threshold but slower than a third threshold over the at least one content element when the at least one content element comprises a textual presentation.

11. The non-transitory computer readable medium of claim 10, wherein the set of rules determine the level of client engagement with the one or more content elements by determining the first user was engaged at a third level when a cursor activity of the cursor included clicking on a navigational link embedded in the textual presentation.

12. The non-transitory computer readable medium of claim 8, wherein the set of rules comprises at least a div tag, a content type, a content boundary, a frame definition, or a headline identifier for the one or more content elements.

13. The non-transitory computer readable medium of claim 8, wherein the interactions of the first user with the one or more content elements comprises cursor movement that includes scrolling behavior, scrolling velocity, scrolling direction, clicking on a comment link, clicking on a sharing link, or clicking on a navigational link.

14. The non-transitory computer readable medium of claim 8, wherein the metadata comprises article titles, a page name, video titles, or content thumbnails.

15. A system comprising one or more processors to configured for:

one or more processors; and

a memory coupled to the one or more processors, the memory storing a plurality of instructions executable by the one or more processors, the plurality of instructions comprising instructions that when executed by the one or more processors cause the one or more processors to perform processing comprising;

identifying one or more content elements of a web page by;

retrieving a set of rules specific to a publisher of the web page; and

applying of the set of rules to the web page to detect the one or more content elements;

creating a data collection script according to the set of rules, the data collection script being configured to collect description data relative to the one or more content elements while the web page is rendered by a web browser on a client device of a first user;

transmitting the data collection script to the web browser on the client device, the data collection script configured to be executable by the web browser;

receiving, from the client device, one or more communications transmitted in response to execution of the data collection script by the web browser, wherein the one or more communications comprise the description data collected at the client device, wherein the description data includes interaction data that describes interactions of the first user with the one or more content elements, and wherein the description data further includes metadata that identifies attributes associated with the one or more content elements;

generating indications that the one or more content elements were read, viewed, or otherwise consumed by the first user based on the description data;

generating indications of a level of engagement of the first user with the one or more content elements based on the description data;

learning how various indications correlate with one another to derive conclusions on behavior of the first user directed at the one or more content elements, wherein the learning comprises applying an algorithm to the various indications, and the various indications include the indications that the one or more content elements were read, viewed, or otherwise consumed and the indications of the level of engagement of the first user with the one or more content elements;

analyzing the indications that the one or more content elements were read, viewed, or otherwise consumed, the indications of the level of engagement of the first user with the one or more content elements, and the derived conclusions on the behavior of the first user to estimate;

(i) at least one content element of the one or more contents elements was consumed by the first user, and (ii) an extent to which the at least one content element was consumed by the first user; and

transmitting a level of client engagement with the at least one content element to a second user based on the estimate that;

(i) the at least one content element of the one or more contents elements was consumed by the first user, and (ii) the extent to which the at least one content element was consumed by the first user.

16. The system of claim 15, wherein:

the analyzing comprises deriving inferences on the behavior of the first user from the indications that the one or more content elements were read, viewed, or otherwise consumed and the indications of the level of engagement of the first user with the one or more content elements based on the set of rules specific to the publisher;

the set of rules define the inferences on the behavior of the first user based on the interactions of the first user with the one or more content elements; and

the analyzing further comprises estimating (i) the at least one content element of the one or more contents elements was consumed by the first user, and (ii) the extent to which the at least one content element was consumed by the first user based on the derived inferences and the derived conclusions.

17. The system of claim 16, wherein the set of rules determine the level of client engagement with the at least one content element by determining the first user was engaged at a first level when a cursor activity of a cursor included no movement for a period of time greater than a first threshold over the at least one content element when the at least one content element comprises a video presentation;

and

the set of rules determine the level of client engagement with the at least one content element by determining the first user was engaged at a second level when a cursor activity of the cursor included scrolling at a pace greater than a second threshold but slower than a third threshold over the at least one content element when the at least one content element comprises a textual presentation.

18. The system of claim 17, wherein the set of rules determine the level of client engagement with the one or more content elements by determining the first user was engaged at a third level when a cursor activity of the cursor included clicking on a navigational link embedded in the textual presentation.

19. The system of claim 15, wherein the set of rules comprises at least a div tag, a content type, a content boundary, a frame definition, or a headline identifier for the one or more content elements.

20. The system of claim 15, wherein the interactions of the first user with the one or more content elements comprises cursor movement that includes scrolling behavior, scrolling velocity, scrolling direction, clicking on a comment link, clicking on a sharing link, or clicking on a navigational link.

21. The system of claim 15, wherein the metadata comprises article titles, a page name, video titles, or content thumbnails.