Methods and systems for conducting surveys and processing survey data to generate a collective outcome

• US 9,727,883 B2
• Filed: 04/12/2013
• Issued: 08/08/2017
• Est. Priority Date: 10/14/2011
• Status: Active Grant

• Current Assignee: Ethelo Decisions Inc.
• Original Assignee: Ethelo Decisions Inc.
• Inventors: Richardson, John
• Primary Examiner(s): Koneru, Sujay
• Application Number:
  US13/862,326
• Publication Number:
  US 20130226665A1
• Time in Patent Office: 1,579 Days
• Field of Search: 705 732
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 30/0203 Market surveys; Market polls

Abstract

Machine-implemented methods and computer systems are disclosed for determining a collective outcome for a survey from a plurality of potential outcomes. Each outcome is a particular combination of options. Each survey participant ranks the options to develop an influent function. For each participant, the influent function is applied to each potential outcome to determine a satisfaction score for the outcome. A collective dissonance score is calculated based on the participants'"'"' satisfaction scores for the potential outcome. The potential outcomes are ordered based on a weighted combination of the satisfaction scores and the collective dissonance score for each of the potential outcomes. In particular embodiments, the methods and systems aim to determine a collective outcome that achieves greatest satisfaction and minimizes dissonance among the participants.

Claims

1. A machine-implemented method of determining a collective outcome for a survey from a plurality of potential outcomes, the method comprising the following steps:

(a) defining a plurality of options, wherein each one of the plurality of options is associated with one of a plurality of issues;

(b) automatically determining the plurality of potential outcomes based on the plurality of options, and applying one or more predefined constraints to the plurality of potential outcomes to determine a plurality of actionable outcomes;

(c) storing the plurality of actionable outcomes on a data storage module of a server;

(d) for each one of the plurality of options, generating and displaying a visual prompt for a ranking indicative of a value accorded to the option, the visual prompt comprising a graphical control element in the form of one or more slider widgets each having a movable indicator, and presenting the visual prompt to a plurality of survey participants via one or more client devices, wherein a position of the indicator on one of the one or more slider widgets is automatically constrained and adjusted based on the values accorded to the other options;

(e) for each of the plurality of options, and for each of the plurality of survey participants, receiving at the server a value for the ranking, wherein the value for the ranking is based on a position of the movable indicator on each of the one or more slider widgets;

(f) for each of the plurality of survey participants, applying, by a data processor of the server, the values for the rankings accorded to the plurality of options by the survey participant to determine an influent function associated with the survey participant;

(g) for each of the survey participants, and for each of the actionable outcomes retrieved from the data storage module, calculating a satisfaction score by the data processor applying to the actionable outcome the influent function associated with the survey participant, and storing the satisfaction score on the data storage module;

(h) for each of the actionable outcomes retrieved from the data storage module, calculating a collective dissonance score by the data processor based on the satisfaction scores for the actionable outcome calculated in step (g) and storing the collective dissonance score on the data storage module, wherein the collective dissonance score is indicative of a measure of difference in the satisfaction scores between the survey participants;

(i) ordering the actionable outcomes stored in the data storage module based on a weighted combination of the satisfaction scores and the collective dissonance score for each of the actionable outcomes stored in the data storage module;

(j) selecting a collective outcome to the survey stored in the data storage module based at least in part on the order of the actionable outcomes determined in step (i);

(k) displaying via the one or more client devices a graphical representation that presents the collective outcome, and the satisfaction scores and the collective dissonance score for each of the actionable outcomes; and

(l) automatically repeating steps (f) through (j) to determine a new collective outcome each time a value for a ranking is modified by movement of an indicator on one of the one or more sliding widgets, and, in response to determining the new collective outcome, updating the graphical user representation to display the new collective outcome.

2. The method of claim 1 comprising, for each of the actionable outcomes, calculating a collective satisfaction score by the data processor based on a weighted combination of the satisfaction scores, wherein each of the satisfaction scores is weighted in accordance with an influent level of the survey participant associated with the satisfaction score, and wherein ordering the actionable outcomes comprises determining a weighted combination of the collective satisfaction score and the collective dissonance score.

3. The method of claim 2 comprising normalizing the influent levels of the survey participants by ensuring that 0<

Sat(O, x)<

1 for all of the actionable outcomes and the survey participants, where Sat(O, x) defines a normalized satisfaction for a survey participant x to an outcome O and is given by;

Sat(O,x)=Sat′

(O,x)^Px

where P^x is determined by taking n=|O^a| and finding P^x for each x in P such that;

4. The method of claim 2 comprising, for one or more of the survey participants, adjusting the influent levels of each survey participant based on the survey participant'"'"'s satisfaction score for a collective outcome from a previous survey, for use in determining a weighted combination of satisfaction scores in the current survey.

5. The method of claim 2 wherein the weighted combination of the collective satisfaction score and the collective dissonance score is determined based on one or more explicit or assumed collective agreements between the survey participants relating to the acceptability of the actionable outcome.

6. The method of claim 1, wherein for each option the ranking comprises an indication of a degree of support accorded to the option and an indication of the importance of the issue associated with the option.

7. The method of claim 6 wherein one of the influent functions is in a first issue domain, and the method comprises translating the one of the influent functions into an equivalent influent function in a second issue domain different from the first issue domain.

8. The method of claim 6 wherein the ranking from one of the survey participants for one of the options defines the degree of support accorded to the option as a function of a characteristic over a predefined range of the characteristic.

9. The method of claim 1 wherein calculating the collective dissonance score for each actionable outcome is based on at least one of:

a standard deviation of satisfaction scores for the survey participants;

for each of the survey participants, a number of survey participants having a higher satisfaction score than the survey participant and a number of survey participants having a lower satisfaction score than the survey participant;

for each of the survey participants, a difference in satisfaction scores associated with a preferred outcome of the survey participant for the survey and the collective outcome for the survey;

a sum of differences in satisfaction scores for the actionable outcome; and

for each of the survey participants and for each of the actionable outcomes, an influent level associated with one or more of;

the survey participant and the actionable outcome.

10. The method of claim 1 comprising determining initial values of the rankings of the options for each of the survey participants based on a starting influent function retrieved from a repository of influent functions.

11. The method of claim 10 comprising modifying the starting influent function in accordance with instructions from the survey participant, wherein the modifying comprises at least one of:

varying a ranking of an option in the starting influent function, and applying a rule to the starting influent function.

12. The method of claim 1 comprising determining the rankings of the options for each of the survey participants by:

obtaining an initial influent function;

applying the initial influent function to each of the actionable outcomes to rank the actionable outcomes, and displaying to the survey participant a ranked list of the actionable outcomes; and

receiving from the survey participant a revised ranking for one or more of the actionable outcomes.

13. The method of claim 1 comprising determining initial values of the rankings of the options for a survey participant by merging first and second starting influent functions retrieved from a repository of influent functions to determine a new influent function in accordance with instructions from the survey participant, the new influent function stipulating a ranking for each of the options, wherein the merging comprises a weighted combination of the first and second starting influent functions.

14. The method of claim 1 comprising determining a recommended ranking for one of the survey participants for at least one of the options based on influent functions associated with one or more other survey participants, and displaying the recommended ranking to the one of the survey participants.

15. The method of claim 1 wherein if one of the survey participants does not indicate a ranking for one of the options, the data processor generates a ranking for the option in determining the influent function associated with the participant.

16. A system for determining a collective outcome for a survey from a plurality of potential outcomes, the system comprising:

a survey conductor having a first data processor and a first data storage module, the first data processor configured to;

define a plurality of options, wherein each one of the plurality of options is associated with one of a plurality of issues;

automatically determine the plurality of potential outcomes based on the plurality of options, and apply one or more predefined constraints to the plurality of potential outcomes to determine a plurality of actionable outcomes, and cause the plurality of actionable outcomes to be stored on the data storage module;

for each one of the plurality of options, generate and display a visual prompt for a ranking indicative of a value accorded to the option, the visual prompt comprising a graphical control element in the form of one or more slider widgets each having a movable indicator, wherein a position of the indicator on one of the one or more slider widgets is automatically constrained and adjusted based on the values accorded to the other options, and present the visual prompt to a plurality of survey participants via one or more client devices;

and

a collective outcome generator in communication with the survey conductor, the collective outcome generator having a second data processor and a second data storage module, the second data processor configured to;

for each of the plurality of options, and for each of the plurality of survey participants, receive a value for the ranking obtained by the survey conductor, determine an influent function associated with the survey participant based on the value for the ranking and cause the influent function to be stored on the second data storage module;

for each of the survey participants, and for each of the actionable outcomes retrieved from the data storage module, apply to the actionable outcome the influent function associated with the survey participant to calculate a satisfaction score and cause the satisfaction score to be stored on the second data storage module;

for each of the actionable outcomes retrieved from the data storage module, calculate a collective dissonance score based on the calculated satisfaction scores for the actionable outcome, wherein the collective dissonance score is indicative of a measure of difference in the satisfaction scores between the survey participants and case the collective dissonance score to be stored on the second data storage module;

order the actionable outcomes based on a weighted combination of the satisfaction scores and the collective dissonance score for each of the actionable outcomes;

select a collective outcome to the survey based at least in part on the order of the actionable outcomes;

present to the one or more client devices a graphical representation of the collective outcome, and the satisfaction scores and the collective dissonance score for each of the actionable outcomes; and

automatically determine a new collective outcome each time a value for a ranking is modified by movement of an indicator on the one or more sliding widgets, and, in response to determining the new collective outcome, update the graphical representation to display the new collective outcome.

17. The system of claim 16 wherein the second data processor of the collective outcome generator is configured to, for each of the actionable outcomes, calculate a collective satisfaction score based on a weighted combination of the satisfaction scores, wherein each of the satisfaction scores is weighted in accordance with an influent level of the survey participant associated with the satisfaction score, and wherein the second data processor of the collective outcome generator is configured to order the actionable outcomes by determining a weighted combination of the collective satisfaction score and the collective dissonance score.

18. The system of claim 17 wherein the first data processor of the collective outcome generator is configured to normalize the influent levels of the survey participants by ensuring that 0<

Sat(O, x)<

1 for all of the actionable outcomes and the survey participants, where Sat(O, x) defines a normalized satisfaction for a survey participant x to an outcome O and is given by;

Sat(O,x)=Sat′

(O,x)^Px

where P^x is determined by taking n=|O^a| and finding P^x for each x in P such that;

19. The system of claim 17 wherein the first data processor of the collective outcome generator is configured to, for one or more of the survey participants, adjust the influent levels of each survey participant based on the survey participant'"'"'s satisfaction score for the collective outcome from a previous survey, for use in determining a weighted combination of satisfaction scores in the current survey.

20. The system of claim 16 wherein each of the options is associated with one of a plurality of issues, wherein for the ranking of each option the first data processor of the survey collector is configured to receive an indication of a degree of support accorded to the option and an indication of the importance of the issue associated with the option.

21. The system of claim 20 wherein one of the influent functions is in a first issue domain, and the second data processor of the collective outcome generator is configured to translate the one of the influent functions into an equivalent influent function in a second issue domain different from the first issue domain.

22. The system of claim 16 wherein the second data processor of the collective outcome generator is configured to calculate the collective dissonance score for each actionable outcome based on at least one of:

a standard deviation of satisfaction scores for the survey participants;

for each of the survey participants, a number of survey participants having a higher satisfaction score than the survey participant and a number of survey participants having a lower satisfaction score than the survey participant;

for each of the survey participants, a difference in satisfaction scores associated with a preferred outcome of the survey participant for the survey and the collective outcome for the survey;

a sum of differences in satisfaction scores for the actionable outcome; and

for each of the survey participants and for each of the actionable outcomes, an influent level associated with one or more of;

the survey participant and the actionable outcome.

23. The system of claim 16 wherein the first data processor of the survey conductor is configured to determine initial values of the rankings of the options for each of the survey participants based on a starting influent function retrieved from a repository of influent functions.

24. The system of claim 23 wherein the first data processor of the survey conductor is configured to modify the starting influent function in accordance with instructions from the survey participant, by at least one of:

varying a ranking of an option in the starting influent function, and applying a rule to the starting influent function.

25. The system of claim 16 wherein the first data processor of the survey conductor is configured to determine the rankings of the options for each of the survey participants by merging first and second starting influent functions retrieved from a repository of influent functions to determine a new influent function in accordance with instructions from the survey participant, the new influent function stipulating a ranking for each of the options, wherein the merging comprises a weighted combination of the first and second starting influent functions.

26. A machine-implemented method of determining a collective outcome for a survey from a plurality of potential outcomes, the method comprising the following steps:

(a) defining a plurality of options, wherein each one of the plurality of options is associated with one of a plurality of issues;

(b) automatically determining the plurality of potential outcomes based on the plurality of options, and applying one or more predefined constraints to the plurality of potential outcomes to determine a plurality of actionable outcomes;

(c) for each one of the plurality of options, generating and displaying a visual prompt for a ranking indicative of a value accorded to the option, the visual prompt comprising a graphical control element in the form of one or more slider widgets each having a movable indicator, and presenting the visual prompt to a plurality of survey participants via one or more client devices, wherein a position of the indicator on one of the one or more slider widgets is automatically constrained and adjusted based on the values accorded to the other options;

(d) for each of the plurality of options, and for each of the plurality of survey participants, receiving a value for the ranking, wherein the value of the ranking is based on a position of the movable indicator on each of the one or more slider widgets;

(e) for each of the plurality of survey participants, applying the values for the rankings by the survey participants to determine an influent function associated with the survey participant;

(f) for each of the survey participants, and for each of the actionable outcomes, calculating by a data processor of a server a satisfaction score by applying to the actionable outcome the influent function associated with the survey participant and storing the satisfaction score on the data storage module;

(g) for each of the actionable outcomes, calculating by the data processor a collective dissonance score based on the satisfaction scores for the actionable outcome calculated in step (f) and storing the collective dissonance score on the data storage module;

(h) ordering the actionable outcomes stored on the data storage module based on a weighted combination of the satisfaction scores and the collective dissonance score stored on the data storage module for each of the actionable outcomes;

(i) selecting a collective outcome to the survey stored in the data storage module based at least in part on the order of the actionable outcomes determined in step (h); and

(j) displaying via the one or more client devices a graphical representation that presents the collective outcome, and the satisfaction scores and the collective dissonance score for each of the actionable outcomes; and

(k) automatically repeating steps (e) through (i) to determine a new collective outcome each time a value for a ranking is modified by movement of an indicator on one of the one or more sliding widgets, and, in response to determining the new collective outcome, updating the graphical user representation to display the new collective outcome.

27. A computer program product comprising a non-transitory computer readable medium having instructions recorded thereon for execution by a processor on a computer to determine a collective outcome for a survey from a plurality of potential outcomes, wherein the instructions comprise the steps of:

(a) defining a plurality of options, wherein each one of the plurality of options is associated with one of a plurality of issues;

(b) automatically determining the plurality of potential outcomes based on the plurality of options, and applying one or more predefined constraints to the plurality of potential outcomes to determine a plurality of actionable outcomes;

(c) storing the plurality of actionable outcomes on a data storage module of a server;

(d) for each one of the plurality of options, generating and displaying a visual prompt for a ranking indicative of a value accorded to the option, the visual prompt comprising a graphical control element in the form of one or more slider widgets each having a movable indicator, and presenting the visual prompt to a plurality of survey participants via one or more client devices, wherein a position of the indicator on one of the one or more slider widgets is automatically constrained and adjusted based on the values accorded to the other options;

(e) for each of the plurality of options, and for each of the plurality of survey participants, receiving at the server a value for the ranking, wherein the value for the ranking is based on a position of the movable indicator on each of the one or more slider widgets;

(f) for each of the plurality of survey participants, applying, by a data processor of the server, the values for the rankings accorded to the plurality of options by the survey participant to determine an influent function associated with the survey participant;

(g) for each of the survey participants, and for each of the actionable outcomes, calculating a satisfaction score by applying to the actionable outcome the influent function associated with the survey participant;

(h) for each of the actionable outcomes, calculating a collective dissonance score based on the satisfaction scores for the actionable outcome calculated in step (g);

(i) ordering the actionable outcomes based on a weighted combination of the satisfaction scores and the collective dissonance score for each of the actionable outcomes; and

(j) selecting a collective outcome to the survey based at least in part on the order of the actionable outcomes determined in step (i),

(k) displaying via the one or more client devices a graphical representation that presents the collective outcome, and the satisfaction scores and the collective dissonance score for each of the actionable outcomes; and

(l) automatically repeating steps (f) through (j) to determine a new collective outcome each time a value for a ranking is modified by movement of an indicator on one of the one or more sliding widgets, and, in response to determining the new collective outcome, updating the graphical user representation to display the new collective outcome.

28. Apparatus for determining a collective outcome for a survey from a plurality of potential outcomes, the apparatus comprising:

(a) means for determining a plurality of potential outcomes based on a plurality of options, and means for applying one or more predefined constraints to the plurality of potential outcomes to determine a plurality of actionable outcomes;

(b) means for generating and displaying a visual prompt for a ranking indicative of a value accorded to the option, the visual prompt comprising a graphical control element in the form of one or more slider widgets each having a movable indicator, and presenting the visual prompt to a plurality of survey participants via one or more client devices, wherein a position of the indicator on one of the one or more slider widgets is automatically constrained and adjusted based on the values accorded to the other options;

(c) means for receiving a value for the ranking for each of the plurality of options, and for each of the plurality of survey participants, wherein the value for the ranking is based on a position of the movable indicator on each of the one or more slider widgets;

(d) means for applying the values for the rankings accorded to the plurality of options by the survey participant to determine an influent function associated with the survey participant;

(e) means for calculating a satisfaction score for each of the survey participants and for each of the actionable outcomes based on the influent function associated with the survey participant;

(f) means for calculating a collective dissonance score for each of the actionable outcomes based on the satisfaction scores for the actionable outcome calculated in step (e);

(g) means for ordering the actionable outcomes based on a weighted combination of the satisfaction scores and the collective dissonance score for each of the actionable outcomes;

(h) means for selecting a collective outcome to the survey based at least in part on the order of the actionable outcomes determined in step (g);

(i) means for displaying via the one or more client devices a graphical representation that presents the collective outcome, and the satisfaction scores and the collective dissonance score for each of the actionable outcomes; and

(j) means for automatically repeating steps (d) through (h) to determine a new collective outcome each time a value for a ranking is modified by movement of an indicator on one of the one or more sliding widgets, and, in response to determining the new collective outcome, updating the graphical user representation to display the new collective outcome.