Processing qualitative responses and visualization generation

• US 10,360,226 B2
• Filed: 07/22/2016
• Issued: 07/23/2019
• Est. Priority Date: 07/24/2015
• Status: Active Grant

• Current Assignee: Fulcrum Management Solutions Ltd.
• Original Assignee: Fulcrum Management Solutions Ltd.
• Inventors: Michelson, Natalie Sara, Jeffery, Mark John, MacLeod, David Anthony
• Primary Examiner(s): Allen, Brittany N
• Application Number:
  US15/216,828
• Publication Number:
  US 20170024395A1
• Time in Patent Office: 1,096 Days
• CPC Class Codes: G06F 16/24578 using ranking
• CPC Class Codes: G06F 16/287 Visualization; Browsing
• CPC Class Codes: G06F 16/355 Class or cluster creation or modification
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15

Abstract

Aspects of this disclosure generally are related to significant systems and methods of processing qualitative, comment-style, responses provided by a population, and for corresponding production of significant visualizations configured to efficiently facilitate insight into population characteristics and matters important to the population. According to some embodiments, qualitative, or open-ended, comment-style responses are assigned priority values by participants. Based at least on an analysis of these priority values, groups of participants, qualitative responses, or both, are identified, according to some embodiments. In some embodiments, a significant visualization is generated that visually presents the groups at least in part via clusters of visual representations of participants, qualitative responses, or both based at least on results of the analysis.

Claims

1. An inquiry-response evaluation device system comprising:

a data processing device system;

an input-output device system communicatively connected to the data processing device system; and

a processor-accessible memory device system communicatively connected to the data processing device system and storing a program executable by the data processing device system,

wherein the data processing device system is configured by the program at least to;

access a plurality of qualitative responses to open-ended inquiries from the processor-accessible memory device system;

access a plurality of priority values from the processor-accessible memory device system, each priority value of the plurality of priority values associated with a respective participant of a plurality of participants and associated with a respective qualitative response of the plurality of qualitative responses, and each priority value of the plurality of priority values indicating a priority assigned by the respective participant to the respective qualitative response;

determine, for each respective participant of the plurality of participants, a respective individual priority pattern associated with the respective participant based at least on an analysis of priority values of the plurality of priority values associated with the respective participant;

determine, as part of a first iteration of a participant-to-group assignment, a first group priority pattern associated with a first group of participants based at least on an analysis of priority values of the plurality of priority values associated with a first set of unassigned participants of the plurality of participants that have not yet been assigned to a group of participants;

determine, for each respective participant in the first set of unassigned participants and as part of the first iteration of the participant-to-group assignment, whether the respective individual priority pattern associated with the respective participant is similar to the first group priority pattern within a first threshold range, and assign the respective participant to the first group of participants in a state in which it is determined that the respective individual priority pattern is similar to the first group priority pattern within the first threshold range;

determine, as part of a second iteration of the participant-to-group assignment, a second group priority pattern associated with a second group of participants based at least on an analysis of priority values of the plurality of priority values associated with a second set of unassigned participants of the plurality of participants that have not yet been assigned to a group of participants, the second set of unassigned participants excluding participants assigned to the first group of participants in the first iteration of the participant-to-group assignment;

determine, for each respective participant in the second set of unassigned participants and as part of the second iteration of the participant-to-group assignment, whether the respective individual priority pattern associated with the respective participant is similar to the second group priority pattern within a second threshold range providing for less similarity than the first threshold range, and assign the respective participant to the second group of participants in a state in which it is determined that the respective individual priority pattern is similar to the second group priority pattern within the second threshold range;

identify a plurality of groups of participants of the plurality of participants based at least on the first group of participants and the second group of participants; and

facilitate, via the input-output device system, visual presentation of at least one group of the identified plurality of groups of participants, the visual presentation including at least a cluster of visual representations of the participants in the one group and including at least a visual representation of at least one participant of the plurality of participants that is not in the one group separated from the cluster.

2. The system of claim 1, wherein each qualitative response of at least some of the plurality of qualitative responses is associated in the processor-accessible memory device system with a respective participant of a plurality of participants, and each qualitative response of the plurality of qualitative responses indicates a response provided by the respective participant to an open-ended inquiry.

3. The system of claim 1, wherein each qualitative response of at least some of the plurality of qualitative responses represents multiple individual qualitative responses from a plurality of participants.

4. The system of claim 1, wherein the data processing device system is configured by the program at least to determine the first group priority pattern based at least on an analysis of priority values of the plurality of priority values associated with at least one of the plurality of qualitative responses and the first set of unassigned participants.

5. The system of claim 4, wherein the data processing device system is configured by the program at least to determine the respective individual priority pattern for each respective participant of the plurality of participants based at least on an analysis of priority values of the plurality of priority values associated with the at least one of the plurality of qualitative responses and the respective participant.

6. The system of claim 1, wherein, for each respective participant of at least some of the plurality of participants, the data processing device system is configured by the program at least to store a group-association-value set in the processor-accessible memory device system, the group-association-value set indicating a degree of association between the respective participant and a group of the plurality of groups.

7. The system of claim 1, wherein the data processing device system is configured by the program at least to:

associate the plurality of qualitative responses with the identified plurality of groups;

determine, for each respective qualitative response of the plurality of qualitative responses, and with respect to each respective group of the identified plurality of groups, a respective resonance value indicating an amount that the respective qualitative response resonates with at least some of the participants in the respective group;

identify a first set of the plurality of qualitative responses associated with multiple of the identified plurality of groups, the determined respective resonance values associated with the first set within a first range;

identify a second set of the plurality of qualitative responses associated with multiple of the identified plurality of groups, the determined respective resonance values associated with the second set within a second range different than the first range; and

facilitate, via the input-output device system, visual presentation of one or more indications (a) that at least some of the qualitative responses in the first set represent relative participant agreement, (b) that at least some of the qualitative responses in the second set represent relative participant tension, or both (a) and (b).

8. The system of claim 7, wherein the first range, the second range, or both, are derived from a standard deviation.

9. The system of claim 7,

wherein the visual presentation of the at least one group further includes a visual representation of the one group of the plurality of groups and a visual representation of an agreement group of the plurality of groups, and

wherein the data processing device system is configured by the program at least to facilitate, via the input-output device system, visual presentation of visual representations of the at least some of the qualitative responses in the first set within the visual representation of the agreement group, and visual presentation of visual representations of the at least some of the qualitative responses in the second set within the visual representation of the one group.

10. The system of claim 1, wherein the data processing device system is configured by the program at least to:

determine, for each respective participant of the plurality of participants, and with respect to each respective group of the identified plurality of groups, a respective resonance value indicating an amount that priority values assigned by the respective participant resonate with at least some of the participants in the respective group;

identify a first set of the plurality of participants associated with multiple of the identified plurality of groups, the determined respective resonance values associated with the first set of the plurality of participants within a first range;

identify a second set of the plurality of participants associated with multiple of the identified plurality of groups, the determined respective resonance values associated with the second set of the plurality of participants within a second range different than the first range; and

facilitate, via the input-output device system, visual presentation of one or more indications (a) that at least some of the participants in the first set of the plurality of participants represent relative participant agreement, (b) that at least some of the participants in the second set of the plurality of participants represent relative participant tension, or both (a) and (b).

11. The system of claim 10, wherein the first range, the second range, or both, are derived from a standard deviation.

12. The system of claim 10,

wherein the visual presentation of the at least one group further includes a visual representation of the one group of the plurality of groups and an agreement group of the plurality of groups, and

wherein the data processing device system is configured by the program at least to facilitate, via the input-output device system, visual presentation of visual representations of the at least some of the participants in the first set of the plurality of participants within the visual representation of the agreement group, and visual presentation of visual representations of the at least some of the participants in the second set of the plurality of participants within the visual representation of the one group.

13. The system of claim 1, wherein the data processing device system is configured by the program at least to:

associate the plurality of qualitative responses with the identified plurality of groups based at least on an analysis of the plurality of priority values associated with the plurality of qualitative responses; and

facilitate, via the input-output device system, visual presentation of at least one of the plurality of qualitative responses with respect to the at least one group of the identified plurality of groups in a manner consistent with one or more results of the analysis of the plurality of priority values associated with the plurality of qualitative responses.

14. The system of claim 13, wherein the visual presentation of the at least one of the plurality of qualitative responses with respect to the at least one group includes a visual representation of the one group as a closed shape.

15. The system of claim 14, wherein the visual presentation of the at least one of the plurality of qualitative responses with respect to the at least one group includes a visual representation of the at least one of the plurality of qualitative responses within the closed shape.

16. The system of claim 15, wherein the analysis of the plurality of priority values associated with the plurality of qualitative responses includes determining, for each respective qualitative response of the plurality of qualitative responses, and with respect to each respective group of the identified plurality of groups, a respective resonance value indicating an amount that the respective qualitative response resonates with at least the participants in the respective group, and wherein the one or more results of the analysis of the plurality of priority values associated with the plurality of qualitative responses indicate at least that the respective resonance values for each of the at least one of the plurality of qualitative responses most closely align the at least one of the plurality of qualitative responses with the one group as compared to each other group of the identified plurality of groups.

17. The system of claim 14, wherein the closed shape is circular.

18. The system of claim 17, wherein the closed shape is visually presented with a color gradient that decreases in color intensity from a geometric center of the closed shape towards an exterior edge of the closed shape.

19. The system of claim 13, wherein the second analysis includes determining, for each respective qualitative response of the plurality of qualitative responses, and with respect to each respective group of the identified plurality of groups, a respective resonance value indicating an amount that the respective qualitative response resonates with at least the participants in the respective group, and wherein the visual presentation of the one of the plurality of qualitative responses is in a spaced relationship with the one group, the spaced relationship consistent at least with the respective resonance value for the one of the plurality of qualitative responses with respect to the one group.

20. The system of claim 19, wherein the spaced relationship results in a representation of the one of the plurality of qualitative responses located within the cluster of visual representations of the participants in the one group.

21. The system of claim 19, wherein the spaced relationship results in a representation of the one of the plurality of qualitative responses located outside the cluster of visual representations of the participants in the one group.

22. The system of claim 1, wherein the data processing device system is configured by the program at least to:

associate, in a first process, each respective qualitative response in a first set of the plurality of qualitative responses with each respective group of at least one of the plurality of identified groups based at least on an analysis of the priority values associated with the respective qualitative response, the first set of the plurality of qualitative responses associated with a particular participant of the plurality of participants;

identify a first particular group of the plurality of identified groups based at least on an analysis of one or more results of the first process;

identify a second particular group of the plurality of identified groups based at least on an analysis of priority values associated with the particular participant with respect to each of the identified plurality of groups; and

facilitate, via the input-output device system, visual presentation of at least visual representations of the particular participant, the first particular group, and the second particular group.

23. The system of claim 22, wherein the identifying of the first particular group based at least on the analysis of the one or more results of the first process includes:

identifying the first particular group as most closely related, as compared to each other of the plurality of identified groups, to the first set of the plurality of qualitative responses.

24. The system of claim 22, wherein the identifying of the second particular group based at least on the analysis of priority values associated with the particular participant with respect to each of the identified plurality of groups includes:

identifying the second particular group as most closely related, as compared to each other of the plurality of identified groups, to the priority values associated with the particular participant.

25. The system of claim 22, wherein visual presentation of at least visual representations of the particular participant, the first particular group, and the second particular group includes a graphical animation including movement of a visual representation of the particular participant between a visual representation of the first particular group and a visual representation of the second particular group.

26. The system of claim 25, wherein the graphical animation including movement of the visual representation of the particular participant between the visual representation of the first particular group and the visual representation of the second particular group includes movement of the visual representation of the particular participant only within an intermediate display region that does not contact the visual representation of the first particular group and the visual representation of the second particular group.

27. The system of claim 1, wherein the visual presentation of the at least one group includes a visual representation of the one group as a closed shape.

28. The system of claim 27, wherein the cluster of visual representations of the participants in the one group are visually represented within the closed shape.

29. The system of claim 28,

wherein the cluster of visual representations of the participants in the one group is represented in a manner consistent with one or more results of the analysis of the plurality of priority values,

wherein the analysis of the plurality of priority values includes determining, for each respective participant of the plurality of participants, and with respect to each respective group of the identified plurality of groups, a respective resonance value indicating an amount that the respective participant resonates with at least the participants in the respective group, and

wherein the one or more results of the analysis of the plurality of priority values indicate at least that the respective resonance values for each of the participants in the one group most closely align the participants in the one group with the one group as compared to each other group of the identified plurality of groups.

30. The system of claim 27, wherein the closed shape is circular.

31. The system of claim 30, wherein the closed shape is visually presented with a color gradient that decreases in color intensity from a geometric center of the closed shape towards an exterior edge of the closed shape.

32. The system of claim 1,

wherein the data processing device system is configured by the program at least to determine, for each particular group of the plurality of groups, a group alignment value with respect to the particular group and each respective other group of the plurality of groups, thereby determining a plurality of group alignment values associated with each particular group of the plurality of groups,

wherein the visual presentation of the at least one group includes a visual representation of the one group, a visual representation of a second particular group of the plurality of groups, and a visual representation of a third particular group of the plurality of groups, the visual representation of the one group closer to the visual representation of the second particular group than the visual representation of the third particular group, and

wherein at least one of the plurality of group alignment values associated with the one group, at least one of the plurality of group alignment values associated with the second particular group, and at least one of the plurality of group alignment values associated with the third particular group indicate that the one group is more closely related to the second particular group than the third particular group.

33. The system of claim 32, wherein the visual presentation of the at least one group further includes a visual representation of an agreement group of the plurality of groups, the agreement group different than the one group, the second particular group, and the third particular group.

34. The system of claim 33, wherein the visual representation of the agreement group is surrounded, at least in part, by the visual representations of the one group, the second particular group, and the third particular group.

35. The system of claim 1, wherein the visual presentation of the at least one group includes a visual representation of the one group, a visual representation of a second particular group of the plurality of groups, and a visual representation of a third particular group of the plurality of groups, the visual representation of the one group closer to the visual representation of the second particular group than the visual representation of the third particular group.

36. The system of claim 1, wherein the visual presentation of the at least one group includes visual representations of at least four groups of the plurality of groups, the visual representations of the at least four groups exhibiting distance relationships therebetween that correspond to relatedness between the at least four groups.

37. The system of claim 1, wherein the visual presentation of the at least one group includes visual representations of at least four groups of the plurality of groups, the visual representations of the at least four groups circumferentially arranged in a circumferential order with at least two most closely related groups of the at least four groups in adjacent positions in the circumferential order and at least two least closely related groups of the plurality of groups in non-adjacent positions in the circumferential order.

38. The system of claim 1, wherein the data processing device system is configured by the program at least to:

determine a plurality of group alignment values for each of a plurality of pairs of at least four groups of the plurality of groups; and

determine a circumferential order of visual representations of the at least four groups that places at least two most closely related groups of the at least four groups in adjacent positions in the circumferential order and places at least two least closely related groups of the at least four groups in opposite positions in the circumferential order based at least on an analysis of the plurality of group alignment values,

wherein the visual presentation of the at least one group includes visual representations of the at least four groups, the visual representations of the at least four groups circumferentially arranged in the circumferential order with at least two most closely related groups of the at least four groups in adjacent positions in the circumferential order and at least two least closely related groups of the plurality of groups in non-adjacent positions in the circumferential order.

39. The system of claim 1, wherein the one group is a first particular group, the cluster is a first cluster, and the visual presentation includes a second particular group of the identified plurality of groups of participants, wherein the visual presentation includes at least a second cluster of visual representations of the participants in the second particular group, and wherein the visual representation of the one participant of the plurality of participants that is not in the first particular group is visually located between the first cluster and the second cluster.

40. The system of claim 39,

wherein the visual representation of the one participant is visually located between the first cluster and the second cluster at a position, with respect to the first cluster and the second cluster, that is consistent at least with:

(a) a first relationship between a participant priority pattern associated with the one participant and a first group priority pattern associated with the first particular group, and (b) a second relationship between the participant priority pattern and a second group priority pattern associated with the second particular group,

wherein the participant priority pattern is a priority pattern of a first set of priority values of the plurality of priority values associated with at least some of the plurality of qualitative responses and the one participant,

wherein the first group priority pattern is a priority pattern of a second set of priority values of the plurality of priority values associated with at least some of the plurality of qualitative responses associated with the first particular group, and

wherein the second group priority pattern is a priority pattern of a third set of priority values of the plurality of priority values associated with at least some of the plurality of qualitative responses associated with the second particular group.

41. The system of claim 1, wherein the visual representations of the participants in the one group are presented with a first set of visual characteristics, and wherein the at least the visual representation of the at least one participant of the plurality of participants that is not in the one group is presented with a second set of visual characteristics different than the first set of visual characteristics.

42. The system of claim 41, wherein the first set of visual characteristics includes a first color, and the second set of visual characteristics includes a second color different than the first color.

43. The system of claim 1, wherein the visual presentation includes a visual representation of a first qualitative response of the plurality of qualitative responses visually located within one of the groups of the identified plurality of groups.

44. The system of claim 1, wherein the visual presentation includes a visual representation of a first qualitative response of the plurality of qualitative responses visually located between at least two of the groups of the identified plurality of groups.

45. The system of claim 44, wherein the data processing device system is configured by the program at least to associate each of at least some of the qualitative responses with each of at least some of the identified plurality of groups.

46. The system of claim 45, wherein the associating of each of the at least some of the qualitative responses with each of the at least some of the identified plurality of groups includes determining at least a first distance between the first qualitative response and a first particular group of the two of the groups and a second distance between the first qualitative response and a second particular group of the two of the groups based at least on the analysis of the priority values associated with the first qualitative response, and wherein the visual representation of the first qualitative response is visually located between at least the two of the groups consistent with the determined first distance and the determined second distance.

47. The system of claim 46, wherein the at least two of the groups is at least three of the groups of the identified plurality of groups, wherein the associating of each of the at least some of the qualitative responses with each of the at least some of the identified plurality of groups includes determining at least a third distance between the first qualitative response and a third particular group of the three of the groups based at least on the analysis of the priority values associated with the first qualitative response, and wherein the visual representation of the first qualitative response is visually located between at least the three of the groups consistent with the determined first distance, the determined second distance, and the determined third distance.

48. The system of claim 1, wherein each of at least some of the priority values indicates a category of emotion exhibited by the respective participant in response to perceiving the respective qualitative response.

49. The system of claim 48, wherein each category of emotion is represented by a distinct numerical value.

50. The system of claim 1, wherein each of at least some of the priority values indicates a number of stars assigned by the respective participant to the respective qualitative response.

51. An inquiry-response evaluation method executed by a data processing device system according to a program stored by a processor-accessible memory device system communicatively connected to the data processing device system, the data processing device system further communicatively connected to an input-output device system, and the method comprising:

accessing a plurality of qualitative responses to open-ended inquiries from the processor-accessible memory device system;

accessing a plurality of priority values from the processor-accessible memory device system, each priority value of the plurality of priority values associated with a respective participant of a plurality of participants and associated with a respective qualitative response of the plurality of qualitative responses, and each priority value of the plurality of priority values indicating a priority assigned by the respective participant to the respective qualitative response;

determining, for each respective participant of the plurality of participants, a respective individual priority pattern associated with the respective participant based at least on an analysis of priority values of the plurality of priority values associated with the respective participant;

determining, as part of a first iteration of a participant-to-group assignment, a first group priority pattern associated with a first group of participants based at least on an analysis of priority values of the plurality of priority values associated with a first set of unassigned participants of the plurality of participants that have not yet been assigned to a group of participants;

determining, for each respective participant in the first set of unassigned participants and as part of the first iteration of the participant-to-group assignment, whether the respective individual priority pattern associated with the respective participant is similar to the first group priority pattern within a first threshold range, and assigning the respective participant to the first group of participants in a state in which it is determined that the respective individual priority pattern is similar to the first group priority pattern within the first threshold range;

determining, as part of a second iteration of the participant-to-group assignment, a second group priority pattern associated with a second group of participants based at least on an analysis of priority values of the plurality of priority values associated with a second set of unassigned participants of the plurality of participants that have not yet been assigned to a group of participants, the second set of unassigned participants excluding participants assigned to the first group of participants in the first iteration of the participant-to-group assignment;

determining, for each respective participant in the second set of unassigned participants and as part of the second iteration of the participant-to-group assignment, whether the respective individual priority pattern associated with the respective participant is similar to the second group priority pattern within a second threshold range providing for less similarity than the first threshold range, and assigning the respective participant to the second group of participants in a state in which it is determined that the respective individual priority pattern is similar to the second group priority pattern within the second threshold range;

identifying a plurality of groups of participants of the plurality of participants based at least on the first group of participants and the second group of participants; and

facilitating, via the input-output device system, visual presentation of at least one group of the identified plurality of groups of participants, the visual presentation including at least a cluster of visual representations of the participants in the one group and including at least a visual representation of at least one participant of the plurality of participants that is not in the one group separated from the cluster.

52. One or more non-transitory computer-readable storage mediums storing a program executable by one or more data processing devices of a data processing device system communicatively connected to an input-output device system, the program comprising:

first access instructions that cause the data processing device system to access a plurality of qualitative responses to open-ended inquiries from a processor-accessible memory device system;

second access instructions that cause the data processing device system to access a plurality of priority values from the processor-accessible memory device system, each priority value of the plurality of priority values associated with a respective participant of a plurality of participants and associated with a respective qualitative response of the plurality of qualitative responses, and each priority value of the plurality of priority values indicating a priority assigned by the respective participant to the respective qualitative response;

first determination instructions that cause the data processing device system to determine, for each respective participant of the plurality of participants, a respective individual priority pattern associated with the respective participant based at least on an analysis of priority values of the plurality of priority values associated with the respective participant;

second determination instructions that cause the data processing device system to determine, as part of a first iteration of a participant-to-group assignment, a first group priority pattern associated with a first group of participants based at least on an analysis of priority values of the plurality of priority values associated with a first set of unassigned participants of the plurality of participants that have not yet been assigned to a group of participants;

third determination instructions that cause the data processing device system to determine, for each respective participant in the first set of unassigned participants and as part of the first iteration of the participant-to-group assignment, whether the respective individual priority pattern associated with the respective participant is similar to the first group priority pattern within a first threshold range, and to assign the respective participant to the first group of participants in a state in which it is determined that the respective individual priority pattern is similar to the first group priority pattern within the first threshold range;

fourth determination instructions that cause the data processing device system to determine, as part of a second iteration of the participant-to-group assignment, a second group priority pattern associated with a second group of participants based at least on an analysis of priority values of the plurality of priority values associated with a second set of unassigned participants of the plurality of participants that have not yet been assigned to a group of participants, the second set of unassigned participants excluding participants assigned to the first group of participants in the first iteration of the participant-to-group assignment;

fifth determination instructions that cause the data processing device system to determine, for each respective participant in the second set of unassigned participants and as part of the second iteration of the participant-to-group assignment, whether the respective individual priority pattern associated with the respective participant is similar to the second group priority pattern within a second threshold range providing for less similarity than the first threshold range, and to assign the respective participant to the second group of participants in a state in which it is determined that the respective individual priority pattern is similar to the second group priority pattern within the second threshold range;

identification instructions that cause the data processing device system to identify a plurality of groups of participants of the plurality of participants based at least on the first group of participants and the second group of participants; and

visual presentation instructions that cause the data processing device system to facilitate, via the input-output device system, visual presentation of at least one group of the identified plurality of groups of participants, the visual presentation including at least a cluster of visual representations of the participants in the one group and including at least a visual representation of at least one participant of the plurality of participants that is not in the one group separated from the cluster.