Data mining to determine online user responses to broadcast messages

• US 10,248,960 B2
• Filed: 11/11/2011
• Issued: 04/02/2019
• Est. Priority Date: 11/16/2010
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Anderson, John, Witkin, Andrew P., Davies, Cameron, Dange, Anjali, Campo-Rembado, Miguel
• Primary Examiner(s): Feacher, Renae
• Application Number:
  US13/294,992
• Publication Number:
  US 20120123854A1
• Time in Patent Office: 2,699 Days
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0244 Optimization
• CPC Class Codes: G06Q 30/0246 Traffic

Abstract

In a networked system, a method and apparatus for measuring advertising effectiveness involves broadcasting an advertisement at a broadcast time, monitoring at least one social network or other online forum following the broadcast time, filtering social network traffic to identify messages, search engine queries to identify queries, or other traffic entries, that appear to be responsive to the broadcast advertisement, determining a time of initiation for at least a plurality of entries that appear to be responsive to the broadcast advertisement, and determining at least one demographic for at least the plurality of the entries that appear to be responsive to the broadcast advertisement.

Claims

1. In a system wherein computing devices are used by users at a plurality of nodes at least to initiate messages that flow between nodes of the plurality of nodes, a method of measuring advertising effectiveness, the method comprising:

monitoring, by a processor, at least one aggregate activity stream following a broadcast time of a broadcast advertisement, wherein the at least one aggregate activity stream represents one or more messages sent by an aggregation of users that form a statistical group over which advertising effectiveness is to be measured, wherein each of the one or more messages representing the at least one aggregate activity stream do not include the broadcast advertisement;

filtering, by the processor, the one or more messages representing the at least one aggregate activity stream to identify messages that appear to be responsive to the broadcast advertisement, wherein filtering the one or more messages includes determining which of the identified messages were sent before the broadcast time of the broadcast advertisement, and determining which of the identified messages are mechanically generated and which of the identified messages are not mechanically generated, wherein a mechanically generated message includes a message that was previously sent at a prior time and that is resent following the broadcast time, and wherein a non-mechanically generated message includes a message that is sent following the broadcast time and that is based on a human response from an observation of the broadcast advertisement;

assigning, by the processor, a first weight value to the identified messages that are mechanically generated and assigning a second weight value to the identified messages that are non-mechanically generated, wherein the first weight value is less than the second weight value;

classifying, by the processor, the identified messages into a plurality of groups, wherein the identified messages are classified into a group based at least in part on a word pattern or phrase associated with an author;

determining, by the processor, a demographic for each of the plurality of groups; and

for each demographic;

determining, by the processor, a measure of favorability based on the identified messages in one or more groups corresponding to the demographic and the weight values assigned to the identified messages in the one or more groups.

2. The method of claim 1, wherein the at least one aggregate activity stream comprises messages that are associated with social network traffic, and wherein monitoring comprises monitoring at least one social network following the broadcast time.

3. The method of claim 1, wherein the at least one aggregate activity stream comprises messages that are search queries, and wherein monitoring comprises monitoring at least one search engine query stream following the broadcast time.

4. The method of claim 1, wherein filtering the at least one aggregate activity stream to identify messages that appear to be responsive to the broadcast advertisement comprises:

determining a time of initiation for a message of the one or more messages; and

filtering the message as not responsive to the broadcast advertisement based on determining that the time of initiation for the message occurs earlier than the broadcast time.

19. The method of claim 1, wherein the mechanically generated message includes a short message system (SMS) message resent following the broadcast time or a social network message resent following the broadcast time.

20. The method of claim 1, wherein filtering the identified messages comprises parsing the identified messages into words, and applying a weight value to each particular word of the words based at least in part on a frequency of occurrence of the particular word.

21. The method of claim 1, wherein the first weight value is different from the second weight value, and wherein the at least one demographic is based at least in part on content of the identified messages that are non-mechanically generated.

22. The method of claim 1, wherein determining which of the identified messages are mechanically generated includes determining which of the identified messages have similar phrases.

23. The method of claim 1, wherein the identified messages are further classified into a group based on a word distribution of a set of words associated with the author corresponding to the group.

24. The method of claim 1, further comprising:

generating a plurality of seed terms; and

computing a value by comparing each of the identified messages to the plurality of seed terms, the computed value indicating a measure of closeness of an identified message to the plurality of seed terms;

wherein a weight value is assigned to each of the identified messages based on the computed value for each of the identified messages.

25. The method of claim 1, wherein the broadcast advertisement has a first format type that is different from a second format type of each of the one or more messages representing the at least one aggregate activity stream.

5. An advertising management system, comprising:

one or more processors; and

memory coupled to the one or more processors, wherein the memory stores instructions configured to cause the one or more processors to perform a method comprising;

filtering user messages following a broadcast time of a broadcast advertisement to identify the user messages that appear to be responsive to the broadcast advertisement, wherein each of the user messages do not include the broadcast advertisement, wherein filtering the user messages includes determining which of the identified user messages were sent before the broadcast time of the broadcast advertisement, and determining which of the identified user messages are mechanically generated and which of the identified user messages are not mechanically generated, wherein a mechanically generated message includes a message that was previously sent at a prior time and that is resent following the broadcast time, and wherein a non-mechanically generated message includes a message that is sent following the broadcast time and that is based on a human response from an observation of the broadcast advertisement;

assigning a first weight value to the filtered messages that are mechanically generated and assigning a second weight value to the filtered messages that are non-mechanically generated, wherein the first weight value is less than the second weight value; and

classifying the identified messages into a plurality of groups, wherein the identified messages are classified into a group based at least in part on a word pattern or phrase associated with an author;

determining a demographic for each of the plurality of groups; and

for each demographic;

determining a measure of favorability based on the identified messages in one or more groups corresponding to the demographic and the weight values assigned to the identified messages in the one or more groups.

6. The advertising management system of claim 5, wherein the user messages comprise messages in a social network system.

7. The advertising management system of claim 5, wherein the user messages comprise search queries submitted to a search engine.

8. The advertising management system of claim 5, wherein the user messages comprise one or more of social media messages, search queries, or requests for one or more web assets.

9. The advertising management system of claim 8, wherein the requests for one or more web assets comprise requests for web assets identified by URLs stored in a list of monitored URLs, wherein at least one of the monitored URLs is associated with at least one previously broadcast advertisement.

10. The advertising management system of claim 9, wherein the previously broadcast advertisement is an advertisement for movie and the list of monitored URLs includes URLs for web pages containing informational materials, promotional materials, or a trailer related to the advertised movie.

11. The advertising management system of claim 5, wherein a broadcast during the broadcast time of the broadcast advertisement is conveyed over one or more of television, radio, a billboard, or Internet advertising media.

12. The advertising management system of claim 5, wherein the method further comprises:

analyzing the user messages and determining a source class of each of the user messages, the source class chosen from a plurality of source classes based on one or both of message content or a message header.

13. The advertising management system of claim 5, wherein the method further comprises:

selecting one of a plurality of broadcast advertisements based on the filtered messages.

14. An advertising management system, comprising:

a memory storage device to store extracted user activity that includes data about user messages;

one or more processors; and

memory coupled to the one or more processors, wherein the memory stores instructions configured to cause the one or more processors to perform a method comprising;

filtering user messages following a broadcast time of a broadcast advertisement to identify user messages that appear to be responsive to the broadcast advertisement, wherein each of the user messages do not include the broadcast advertisement, wherein filtering the user messages includes determining which of the identified user messages were sent before the broadcast time of the broadcast advertisement, and determining which of the identified user messages are mechanically generated and which of the identified user messages are not mechanically generated, wherein a mechanically generated message includes a message that was previously sent at a prior time and that is resent following the broadcast time, and wherein a non-mechanically generated message includes a message that is sent following the broadcast time and that is based on a human response from an observation of the broadcast advertisement;

assigning a first weight value to the filtered messages that are mechanically generated and assigning a second weight value to the filtered messages that are non-mechanically generated, wherein the first weight value is less than the second weight value;

classifying the identified messages into a plurality of groups, wherein the identified messages are classified into a group based at least in part on a word pattern or phrase associated with an author;

determining a demographic for each of the plurality of groups; and

for each demographic;

determining a measure of favorability based on the identified messages in one or more groups corresponding to the demographic and the weight values assigned to the identified messages in the one or more groups.

15. The advertising management system of claim 14, wherein the user messages comprise one or more of social media messages, search queries, or requests for one or more web assets.

16. The advertising management system of claim 14, wherein filtering of the user messages is adjusted based at least in part on a demographic determined for a group.

17. The advertising management system of claim 14, wherein the method further comprises:

analyzing the user messages and determining a source class of each of the user messages, the source class chosen from a plurality of source classes based on one or both of message content or a message header.

18. An advertising management system, comprising:

one or more processors; and

memory coupled to the one or more processors, wherein the memory stores instructions configured to cause the one or more processors to perform a method comprising;

filtering user messages following a broadcast time of a broadcast advertisement to identify the user messages that appear to be responsive to the broadcast advertisement, wherein each of the user messages do not include the broadcast advertisement;

wherein filtering the user messages includes determining which of the identified user messages were sent before the broadcast time of the broadcast advertisement, and determining which of the identified user messages are mechanically generated and which of the identified user messages are not mechanically generated, wherein a mechanically generated message includes a message that was previously sent at prior time and that is resent following the broadcast time, and wherein a non-mechanically generated message includes a message that is sent following the broadcast time and that is based on a human response from an observation of the broadcast advertisement; and

adjusting content in the broadcast advertisement based at least in part on results of identified user messages responsive of a broadcast advertisement.