Determining information inter-relationships from distributed group discussions

• US 9,450,771 B2
• Filed: 10/24/2014
• Issued: 09/20/2016
• Est. Priority Date: 11/20/2013
• Status: Active Grant

• Current Assignee: Deloitte Development LLC
• Original Assignee: Blab Incorporated
• Inventors: Browning, Randolph, Papp, Stefan Nicholas, Zimmermann, Bernhard G., Bressler, Benjamin Ralph
• Primary Examiner(s): Bengzon, Greg C
• Application Number:
  US14/523,746
• Publication Number:
  US 20150142888A1
• Time in Patent Office: 697 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/285 Clustering or classification
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/02 Knowledge representation; Symbolic representation
• CPC Class Codes: G06N 5/046 Forward inferencing; Production systems
• CPC Class Codes: G06N 5/048 Fuzzy inferencing
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: H04L 12/1831 Tracking arrangements for later retrieval, e.g. recording contents, participants activities or behavior, network status
• CPC Class Codes: H04L 43/045 for graphical visualisation of monitoring data
• CPC Class Codes: H04L 51/52 for supporting social networking services
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

Techniques are described for analyzing user-supplied information and/or predicting future aspects of additional information that will be supplied by users. The analyzed information may include distributed group discussions involving numerous users, occurring via user comments and other content items supplied to social networking site(s) and/or other sources. Analysis of user-supplied information may, for example, include determining particular topics and/or categories of interest during one or more time periods and quantifying corresponding amounts of user interest; determining trends for, relevant terms and attributes for, and inter-relationships between the topics and categories; predicting future amounts of user interest in the topics and categories; tracking user interactions with information about the topics and/or categories; and taking further actions based on the analyzed and/or predicted information.

Claims

1. A computer-implemented method comprising:

obtaining, by one or more configured computing systems of a group discussion analysis service, information about a distributed group discussion that involves a plurality of users submitting a plurality of textual comments on a plurality of topics for the distributed group discussion;

analyzing, by the one or more configured computing systems, the plurality of textual comments to identify the plurality of topics;

generating, by the one or more configured computing systems, a plurality of comment groups based on the identified plurality of topics, wherein each of the generated comment groups is associated with one of the identified topics and includes one or more of the plurality of textual comments that have contents mentioning the associated identified topic;

identifying, by the one or more configured computing systems and for each of a plurality of specified content categories, a distinct subset of the plurality of textual comments that are associated with the specified content category based on contents of the textual comments of the identified subset, the textual comments of the identified subset being included in multiple of the generated comment groups;

determining, by the one or more configured computing systems and for each of the plurality of specified content categories, a distinct subset of the multiple generated comment groups that correspond to the specified content category for the specified prior time period;

determining, by the one or more configured computing systems, multiple terms of interest that are distinct from the plurality of topics, and assigning each of the multiple terms to one of multiple graph axes;

determining, by the one or more configured computing systems, and for each of at least some of the plurality of topics, a relevance of the topic to the multiple terms based at least in part on the textual comments included in one of the generated comment groups that is associated with the topic; and

providing, by the one or more configured computing systems, information to initiate display of a multi-dimensional graph that includes the multiple graph axes and that includes visual indicators for multiple of the at least some topics on the multi-dimensional graph, wherein a position of each of the multiple topics on the displayed multi-dimensional graph is based on the determined relevance of the topic to the multiple terms and is based on the assignment of the multiple terms to the multiple graph axes, to enable a visual determination from the displayed multi-dimensional graph of relative relevance of the multiple topics to the multiple terms.

2. The method of claim 1 wherein the providing of the information to initiate the display of the multi-dimensional graph includes providing the information to a client device of a user of the group discussion analysis service, and wherein display of the multi-dimensional graph occurs on the client device for the user.

3. The method of claim 2 wherein the visual indicators for the multiple topics that are included on the displayed multi-dimensional graph are each selectable by the user, and wherein the method further comprises:

receiving, by the one or more configured computing systems, an indication of one of the multiple topics selected by the user via an interaction with the visual indicator for the selected one topic; and

providing, by the one or more configured computing systems and in response to the received indication of the selected one topic, additional information for display to the user regarding the selected one topic, wherein the additional information includes at least one of information about the textual comments associated with the selected one topic, information about one or more attributes determined to be relevant to the selected one topic based at least in part on an analysis of the textual comments associated with the selected one topic, or information about one or more inter-relationships determined between the selected one topic and one or more other topics of the plurality based at least in part on an analysis of information about the selected one topic and about the one or more other topics.

4. The method of claim 2 wherein the determining of the multiple terms of interest includes receiving, by the one or more configured computing systems, selections from the user of the multiple terms.

5. The method of claim 2 wherein the assigning of each of the multiple terms to one of multiple graph axes includes receiving, by the one or more configured computing systems, indications from the user of a type of multi-dimensional graph and of, for each of the multiple terms, one of the multiple graph axes to which the term is to be assigned.

6. The method of claim 2 further comprising, before the providing of the information:

receiving, by the one or more configured computing systems, one or more criteria from the user for use in selecting a subset of the at least some topics to display on the multi-dimensional graph; and

selecting, by the one or more configured computing systems, the multiple topics based at least in part on the one or more criteria, wherein the multiple topics are a subset of the at least some topics.

7. The method of claim 2 further comprising, after the providing of the information:

receiving, by the one or more configured computing systems, one or more additional terms of interest from the user;

determining, by the one or more configured computing systems, and for each of the multiple topics, a relevance of the topic to the one or more additional terms; and

providing, by the one or more configured computing systems, additional information to initiate an updated display of the multi-dimensional graph in which the one or more additional terms are each assigned to one of the multiple graph axes and in which the positions of the visual indicators for the multiple topics are updated to reflect the determined relevances of the multiple topics to the one or more additional terms.

8. The method of claim 2 further comprising, under control of the client device, receiving the provided information, and displaying the multi-dimensional graph to the user on one or more display screens associated with the client device.

9. The method of claim 1 wherein each of the multiple graph axes has a single assigned term and includes a plurality of positions that each corresponds to a distinct degree of relevance with respect to the single assigned term, wherein the determining of the relevance of each of the at least some topics to the multiple terms includes, for each of the at least some topics and each of the multiple terms, determining a relevance score of the topic to the term that reflects a degree of relevance of the term to the topic, and wherein, for each of the multiple topics, the position of the visual indicator for the topic is selected for each of the multiple graph axes to correspond to the degree of relevance for the topic to the single assigned term for the graph axis.

10. The method of claim 1 wherein one of the multiple graph axes is assigned at least two of the multiple terms and includes a plurality of positions that each corresponds to a distinct degree of relevance with respect to a combination of the assigned at least two terms, wherein the determining of the relevance of each of the at least some topics to the multiple terms includes, for each of the at least some topics and the assigned at least two terms, determining degree of relevance of the topic to a combination of the assigned at least two terms, and wherein, for each of the multiple topics, the position of the visual indicator for the topic is selected for the one graph axis to correspond to the degree of relevance for the topic to the combination of the assigned at least two terms.

11. The method of claim 1 wherein one of the multiple graph axes is assigned to one or more attributes distinct from the multiple terms and includes a plurality of positions that each corresponds to a distinct degree of relevance with respect to the assigned one or more attributes, wherein the determining of the relevance of each of the at least some topics to the multiple terms further includes, for each of the at least some topics, determining a degree of relevance of the topic to the assigned one or more attributes, and wherein, for each of the multiple topics, the position of the visual indicator for the topic is selected for the one graph axis to correspond to the degree of relevance for the topic to the assigned one or more attributes.

12. The method of claim 1 wherein the plurality of textual comments are supplied by the plurality of users during a specified prior time period

and wherein the visual indicators for the multiple topics correspond to multiple comment groups that are a subset of the plurality of comment groups.

13. The method of claim 12 wherein the multiple comment groups are assigned to multiple content categories of the plurality of specified content categories, wherein the plurality of content categories are specified by a client of the group discussion analysis service, and wherein the providing of the information is performed in response to a request from the client, to further enable a visual determination by the client from the displayed multi-dimensional graph of relative relevance of the multiple comment groups from the multiple content categories.

14. A non-transitory computer-readable medium having stored contents that configure a computing system to perform a method, the method comprising:

analyzing, by the configured computing system, a plurality of textual comments being supplied by human users during a prior time period and being related to a plurality of topics that are each mentioned in contents of one or more of the plurality of textual comments and that are identified from the analyzing;

generating, by the configured computing system, a plurality of comment groups based on the identified topics, each of the generated comment groups being associated with one of the identified topics and including one or more of the plurality of textual comments that have contents mentioning the associated identified topic;

determining, by the configured computing system, multiple terms of interest that are distinct from the plurality of topics, and assigning each of the multiple terms to one of multiple graph axes;

determining, by the configured computing system, and for each of at least some of the plurality of topics, a relevance of one of the generated comment groups associated with the topic to the multiple terms based at least in part on the textual comments included in the one generated comment group;

providing, by the configured computing system, information to initiate display of a multi-dimensional graph that includes the multiple graph axes and that includes visual indicators for multiple of the at least some topics on the multi-dimensional graph, wherein a position of each of the multiple topics on the displayed multi-dimensional graph is based on the determined relevance of the one generated comment group associated with the topic to the multiple terms and is based on the assignment of the multiple terms to the multiple graph axes, to enable a visual determination from the displayed multi-dimensional graph of relative relevance of the multiple topics to the multiple terms;

identifying, by the configured computing system, a subset of the plurality of textual comments that are associated with a specified content category based on contents of the textual comments of the identified subset, the textual comments of the identified subset being included in multiple of the generated comment groups;

determining, by the configured computing system, a subset of the multiple generated comment groups that correspond to the specified content category for the prior time period;

discovering, by the configured computing system, additional content categories that are related to the specified content category, wherein each of the additional content categories corresponds to the topic of at least one of the determined subset of comment groups; and

providing, by the configured computing system, indications of the determined additional content categories that are related to the specified content category.

15. The non-transitory computer-readable medium of claim 14 wherein the method further comprises providing, by the configured computing system, indications of multiple topics determined to correspond to the specified content category for the prior time period, the multiple topics including, for each of the generated comment groups in the determined subset, the associated identified topic for the generated comment group.

16. The non-transitory computer-readable medium of claim 14 wherein the determining of the subset includes at least one of excluding at least one first comment group whose included textual comments appear in the identified subset less than a determined minimum threshold or of excluding at least one second comment group whose included textual comments appear in the identified subset more than a determined maximum threshold.

17. The non-transitory computer-readable medium of claim 14 wherein the method further comprises:

determining, by the configured computing system, and for each of the comment groups of the determined subset, a quantity of the textual comments included in the comment group;

predicting, by the configured computing system, and for each of the comment groups of the determined subset, an expected quantity of future textual comments associated with the identified topic for the comment group that will be supplied by human users during a future time period, the predicting being based in part on the determined quantity for the comment group; and

providing, by the configured computing system, an indication of the predicted expected quantity of future textual comments for one or more of the comment groups of the determined subset.

18. The non-transitory computer-readable medium of claim 14 wherein the method further comprises:

determining, by the configured computing system, and for each of the comment groups of the determined subset, a quantity of the textual comments included in the comment group;

predicting, by the configured computing system, an expected quantity of future textual comments associated with the specified content category that will be supplied by human users during a future time period, the predicting being based in part on a combination of the determined quantities for the comment groups of the determined subset; and

providing, by the configured computing system, an indication of the predicted expected quantity of future textual comments for the specified content category.

19. The non-transitory computer-readable medium of claim 14 wherein the plurality of textual comments are supplied by a plurality of human users at multiple geographical locations, and wherein the method further comprises:

determining, by the configured computing system, and for one of the comment groups of the determined subset, distinct subsets of the textual comments included in the one comment group that are associated with each of the multiple geographical locations;

determining, by the configured computing system, and for each of the multiple geographical locations, a quantity of the textual comments included in the subset associated with the geographical location; and

providing, by the configured computing system, information about the determined quantity for each of one or more of the multiple geographical locations.

20. The non-transitory computer-readable medium of claim 14 wherein the plurality of textual comments are posts supplied by a plurality of human users to one or more social networking services, and wherein the method further comprises:

determining, by the configured computing system, and for one of the comment groups of the determined subset, distinct subsets of the textual comments included in the one comment group that are associated with each of the plurality of human users;

determining, by the configured computing system, and for each of the plurality of human users, a quantity of the textual comments included in the subset associated with the human user; and

providing, by the configured computing system, information about the determined quantity for each of one or more of the plurality of human users.

21. The non-transitory computer-readable medium of claim 14 wherein the plurality of textual comments are supplied by the human users to a plurality of information sources, and wherein the method further comprises:

determining, by the configured computing system, and for one of the comment groups of the determined subset, distinct subsets of the textual comments included in the one comment group that are associated with each of the plurality of information sources;

determining, by the configured computing system, and for each of the plurality of information sources, a quantity of the textual comments included in the subset associated with the information source; and

providing, by the configured computing system, information about the determined quantity for each of one or more of the plurality of information sources.

22. The non-transitory computer-readable medium of claim 14 wherein the method further comprises:

determining, by the configured computing system, and for one of the comment groups of the determined subset, a quantity of the textual comments included in the one comment group;

determining, by the configured computing system and based at least in part on the determined quantity, one or more types of information to make available corresponding to at least one of the specified content category and the identified topic associated with the one comment group; and

supplying, by the configured computing system, one or more additional textual comments that are available to the human users and that have contents corresponding to the determined one or more types of information.

23. The non-transitory computer-readable medium of claim 22 wherein the plurality of textual comments are part of user posts to one or more social networking services by the human users, and wherein the supplying of the one or more additional textual comments includes supplying the additional textual comments to at least one of the social networking services as additional posts.

24. The non-transitory computer-readable medium of claim 14 wherein the method further comprises:

determining, for each of one or more additional prior time periods, a plurality of comment groups that have corresponding topics for the specified content category for the additional prior time period and that, wherein the corresponding topics for the determined plurality of comment groups of at least some of the additional prior time periods are distinct from the plurality of topics determined to correspond to the specified content category for the prior time period; and

providing indications of one or more topic changes for the specified content category between the prior time period and one or more of the additional prior time periods.

25. The non-transitory computer-readable medium of claim 14 wherein the method further comprises:

receiving, by the configured computing system, information about a succession of multiple content categories that one or more users select; and

providing, by the configured computing system, information to initiate display of information to at least one user about the succession of the multiple content categories.

26. The non-transitory computer-readable medium of claim 25 wherein the receiving of the information about the succession of the multiple content categories includes:

tracking, by the configured computing system, interactions of a plurality of users with a plurality of content categories; and

analyzing, by the configured computing system, the tracked interactions to identify the succession of multiple content categories, wherein the multiple content categories are a subset of the plurality of content categories, and wherein the one or more users include at least some of the plurality of users.

27. The non-transitory computer-readable medium of claim 14 wherein the method further comprises:

receiving, by the configured computing system, information about a succession of multiple comment groups that one or more users select; and

providing, by the configured computing system, information to initiate display of information to at least one user about the succession of the multiple comment groups.

28. The non-transitory computer-readable medium of claim 14 wherein the method further comprises:

receiving, by the configured computing system, a request from a user to display selected information that relates to at least one of one or more comment groups or one or more content categories;

determining, by the configured computing system, one or more pieces of promotional content that are not included in the selected information and that are relevant for the selected information; and

providing, by the configured computing system, information to initiate display to the user of the selected information and of at least one of the one or more pieces of promotional content.

29. The non-transitory computer-readable medium of claim 14 wherein the method further comprises:

analyzing, by the configured computing system and for each of the plurality of comment groups, the one or more textual comments included in the comment group to determine one or more relevant attributes for the comment group that are distinct from the associated topic for the comment group; and

providing, by the configured computing system, information about the determined one or more relevant attributes for one or more of the plurality of comment groups.

30. The non-transitory computer-readable medium of claim 29 wherein the method further comprises determining, by the configured computing system, one or more inter-relationships that are each between two or more comment groups associated with two or more content categories, wherein the determining of each of the inter-relationships is based at least in part on the determined one or more relevant attributes for each of the two or more comment groups of the inter-relationship, and wherein the providing of the information about the determined one or more relevant attributes includes providing information about at least one of the determined inter-relationships.

31. The non-transitory computer-readable medium of claim 30 wherein at least some of the determined relevant attributes for at least some of the plurality of comment groups are terms included in contents of the plurality of textual comments, and wherein the method further comprises displaying, by the configured computing system, information about at least one of the determined inter-relationships to at least one user.

32. The non-transitory computer-readable medium of claim 29 wherein the providing of the information about the determined one or more relevant attributes includes displaying, by the configured computing system, information to at least one user about the determined one or more relevant attributes for the one or more comment groups.

33. The non-transitory computer-readable medium of claim 14 wherein the method further comprises:

analyzing, by the configured computing system and for one or more of the content categories, information about the textual comments included in the subset of multiple generated comment groups corresponding to the one content category to determine one or more relevant attributes for the one content category that are distinct from any definition terms for the one content category; and

providing, by the configured computing system, information about the determined one or more relevant attributes for the one content category.

34. The non-transitory computer-readable medium of claim 33 wherein at least one of the determined relevant attributes for the one content category is a term included in contents of the textual comments included in the subset of multiple generated comment groups corresponding to the one content category, and wherein the method further comprises:

analyzing, by the configured computing system and for one or more of the plurality of comment groups that are not in the subset of multiple generated comment groups corresponding to the one content category, the one or more textual comments included in the comment group to determine one or more relevant attributes for the comment group that are distinct from the associated topic for the comment group; and

determining, by the configured computing system, one or more inter-relationships that are each between the one content category and at least one of the one or more comment groups that are not in the subset of multiple generated comment groups corresponding to the one content category, wherein the determining of each of the inter-relationships is based at least in part on the determined relevant attributes for the one content category and on the determined one or more relevant attributes for the at least one comment group of the inter-relationship,

and wherein the providing of the information about the determined one or more relevant attributes for the one content category includes displaying, by the configured computing system, information to at least one user about at least one of the determined one or more relevant attributes for the one content category or of the determined one or more inter-relationships.

35. The non-transitory computer-readable medium of claim 14 wherein the configured computing system is part of a group discussion analysis and prediction service, wherein the specified content category is indicated by a client of the group discussion analysis and prediction service, and wherein the providing of the indications of the determined additional content categories includes providing information to the client in accordance with criteria specified by the client.

36. The non-transitory computer-readable medium of claim 14 wherein the identified plurality of topics are each at least one of a term, of a phrase, or of a hashtag, wherein the computer-readable medium is a memory of the configured computing system, and wherein the stored contents are instructions that, when executed, program the configured computing system to perform the method.

37. The non-transitory computer-readable medium of claim 14 wherein the configured computing system is part of a group discussion analysis and prediction service, and wherein the providing of the information includes sending information over one or more computer networks from the group discussion analysis and prediction service to a client device of an end user of the group discussion analysis and prediction service for display on an application from the group discussion analysis and prediction service that is executing on the client device.

38. The non-transitory computer-readable medium of claim 14 wherein the configured computing system is part of a group discussion analysis and prediction service, wherein the stored contents further configure the computing system to provide additional information from the group discussion analysis and prediction service to one or more advertisers that includes at least one of discovered additional content categories or determined attributes for two or more of the generated comment groups or information about determined terms of interest, and wherein the providing of the information includes providing promotional information from at least one advertiser in response to instructions from the at least one advertiser based on the provided additional information.

39. A system, comprising:

one or more processors of one or more computer systems; and

one or more memories including instructions that, when executed by at least one of the one or more processors, configure the at least one processor to determine relevant information for a distributed group discussion that involves a plurality of users submitting a plurality of textual comments on a plurality of topics for the distributed group discussion, the determining of the relevant information including;

analyzing the plurality of textual comments to identify the plurality of topics, wherein each topic of the plurality is mentioned in contents of one or more of the plurality of textual comments;

generating a plurality of comment groups based on the identified topics, each of the generated comment groups being associated with one of the identified topics and including one or more of the plurality of textual comments that have contents mentioning the associated identified topic;

determining multiple terms of interest that are distinct from the plurality of topics, and assigning each of the multiple terms to one of multiple graph axes;

determining, for each of at least some of the plurality of topics, a relevance of one of the generated comment groups associated with the topic to the multiple terms based at least in part on the textual comments included in the one generated comment group;

analyzing, for each of the plurality of comment groups, the one or more textual comments included in the comment group to determine one or more relevant attributes for the comment group that are distinct from the associated topic for the comment group;

determining one or more inter-relationships that are each between two or more of the plurality of comment groups based at least in part on the determined one or more relevant attributes for each of the two or more comment groups of the inter-relationship; and

providing information to a user about at least one of the determined one or more inter-relationships and to initiate display of a multi-dimensional graph that includes the multiple graph axes and that includes visual indicators for multiple of the at least some topics on the multi-dimensional graph, wherein a position of each of the multiple topics on the displayed multi-dimensional graph is based on the determined relevance to the multiple terms of the one generated comment group associated with the topic and is based on the assignment of the multiple terms to the multiple graph axes, to enable a visual determination from the displayed multi-dimensional graph of relative relevance of the multiple topics to the multiple terms.

40. The system of claim 39 wherein at least some of the determined relevant attributes for at least some of the plurality of comment groups are terms included in contents of the plurality of textual comments, and wherein the instructions further configure the at least one processor to display information to at least one user about one or more determined relevant attributes for one or more of the plurality of comment groups.

41. The system of claim 39 wherein the determining of the relevant information further includes:

determining, for each of multiple content categories, multiple comment groups that are a subset of the plurality of generated comment groups and that correspond to one or more definition terms for the content category;

analyzing, for each of the multiple content categories, the textual comments included in the multiple comment groups corresponding to the content category to determine one or more relevant attributes for the content category that are distinct from the definition terms for the content category; and

providing information about at least one of the determined relevant attributes for at least one of the multiple content categories.

42. The system of claim 41 wherein at least some of the determined relevant attributes for at least some of the multiple content categories are terms included in contents of the plurality of textual comments, and wherein the at least one determined inter-relationship is between two or more comment groups associated with two or more of the multiple content categories.

43. The system of claim 39 wherein the one or more computer systems are part of a group discussion analysis and prediction service and the user is an end-user of the group discussion analysis and prediction service, and wherein the providing of the information to the user includes sending information over one or more computer networks from the group discussion analysis and prediction service to a client device of the user for display on an application from the group discussion analysis and prediction service that is executing on the client device.

44. The system of claim 39 wherein the one or more computer systems are part of a group discussion analysis and prediction service, wherein the instructions further configure the at least one processor to provide, by the group discussion analysis and prediction service, additional information to one or more advertisers from the determining of the relevant information, the provided additional information including at least one of determined relevant attributes for two or more of the comment groups or information about at least one determined inter-relationship between two or more of the plurality of comment groups, and wherein the providing of the information to the user includes providing promotional information from at least one advertiser in response to instructions from the at least one advertiser based on the provided additional information.