SURVEY INSIGHT REPORTING SYSTEM AND METHOD

• US 20180225602A1
• Filed: 04/03/2018
• Published: 08/09/2018
• Est. Priority Date: 11/04/2014
• Status: Active Grant

• Current Assignee: Energage, LLC
• Original Assignee: Energage, LLC
• Inventors: Joi, Richard Johnson, Leaman, Zachary Jordan, Claffey, Douglas John, Salk, Ethan, Smith, Kinsey
• Granted Patent: US 10,055,701 B1
• Time in Patent Office: Days
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06Q 10/063 Operations research, analysis or management
• CPC Class Codes: G06T 11/206 Drawing of charts or graphs
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]

Abstract

In one embodiment, the invention can be a method of displaying survey results. The method can include, for each of a plurality of survey items, receiving survey responses from survey respondents, wherein each response is chosen from response options, the response options corresponding to an ordinal scale; and each respondent is associated with a collection of respondent segments. The method can further include, for each response to each item by each survey respondent, determining a calibrated score, the calibrated score based on a probability that a person having the collection of respondent segments associated with the survey respondent would provide the response to the item or any of the response options to the item that are lower on the ordinal scale.

Claims

1. A method of displaying survey results, the method comprising:

a) for each of a plurality of survey items, receiving survey responses from survey respondents, wherein;

i) each response is chosen from response options, the response options corresponding to an ordinal scale; and

ii) each respondent is associated with a collection of respondent segments; and

b) for each response to each item by each survey respondent, determining a calibrated score, the calibrated score based on a probability that a person having the collection of respondent segments associated with the survey respondent would provide either;

i) the response to the item or any of the response options to the item that are lower on the ordinal scale;

or

ii) the response to the item or any of the response options to the item that are higher on the ordinal scale;

c) for each item, determining a composite score based on the calibrated score for each response to the item;

d) for each respondent segment, determining a composite score based on the calibrated score for each response associated with the respondent segment; and

e) providing a display of response groupings, the displayed response groupings comprising at least a portion of the plurality of items and the collection of respondent segments;

f) wherein the displayed response groupings are located on the display along a first axis; and

g) wherein the display includes, for each response grouping, a corresponding image, the corresponding image being located with respect to the first axis or a distinct second axis based on the composite score for the response grouping.

4. The method of claim 1 wherein the calibrated scored is further based on benchmark data, and the benchmark data comprises a calibrated score for each response option for each item for each possible collection of respondent segments.

5. The method of claim 1 wherein each calibrated score is a logit-based score.

6. The method of claim 1,

a) wherein the collection of respondent segments comprises demographic segments and department segments;

and

b) wherein the demographic segments are associated with at least one of respondent tenure, respondent hours worked, respondent salary range, respondent management level, and respondent role.

9. The method of claim 1 wherein:

a) each item composite score is the average of the calibrated scores for the responses to the item; and

b) each segment composite score is the average of the calibrated scores for the responses associated with the respondent segment.

10. The method of claim 1 wherein the displayed response groupings are ordered along the first axis according to a noteworthiness ranking of each response grouping to an organization to which the survey respondents belong.

11. The method of claim 10 further comprising providing, on a user interface, user-selectable options for determining one or more factors upon which the noteworthiness ranking of each response grouping is based.

12. The method of claim 10 wherein the noteworthiness ranking of each response grouping is based on the composite score for each response grouping.

13. The method of claim 10 wherein the noteworthiness ranking of each response grouping is based on either:

a) a change in the composite score for the response grouping from a previous composite score for the response grouping;

or

b) where a previous composite score is unavailable, an average change of a related response grouping.

15. The method of claim 10 wherein the noteworthiness ranking of each response grouping is based on, for each respondent segment, a size of the respondent segment.

16. The method of claim 15 wherein the size of the respondent segment is based on a total number of survey respondents associated with the respondent segment.

18. The method of claim 16 wherein each survey respondent or possible survey respondent is assigned a weight, and the noteworthiness ranking of each response grouping is based on a sum of the weights for each respondent segment.

23. The method of claim 10 wherein:

a) the collection of respondent segments comprises department segments, and the department segments comprise a parent-department segment having a first child-department segment and a second child-department segment; and

b) the noteworthiness ranking of the first child-department segment is further based on a comparison of the composite score for the first child-department segment with the composite score for the second child-department segment.

26. The method of claim 10 wherein the noteworthiness ranking of each response grouping is based on an actionability index, the actionability index indicative of an ease or difficulty in addressing the response grouping.

27. The method of claim 10 further comprising:

a) among the respondent segments, identifying a first segment and a second segment as having overlapping respondents;

b) identifying the first segment respondents as being a proper subset of the second segment respondents;

c) reducing the noteworthiness ranking of the second segment if a second segment noteworthiness score is closer to an average noteworthiness score than a noteworthiness score for survey respondents belonging to the second segment but not the first segment; and

d) reducing the noteworthiness ranking of the first segment if a first segment noteworthiness score is closer to an average noteworthiness score than a noteworthiness score for survey respondents belonging to the second segment but not the first segment.

28. The method of claim 10 further comprising:

a) if the first segment respondents are a not a proper subset of the second segment respondents, then;

i) if a first segment noteworthiness score is close to a second segment noteworthiness score, then reducing the noteworthiness ranking of the less actionable of the first segment and the second segment; and

ii) if the first segment noteworthiness score is not close to the second segment noteworthiness score, then reducing the noteworthiness ranking of the segment having a noteworthiness score closer to an average noteworthiness score. b) wherein the first noteworthiness score and the second noteworthiness score are close if either;

i) the scores differ by a predetermined percentage or amount;

or

ii) a distance between the scores exceeds a difference between a benchmark first percentile score and a benchmark second percentile score; and

c) wherein the actionability of a segment is based on an ease or difficulty in addressing the response grouping.

29. The method of claim 28 wherein, prior to identifying the overlapping respondents and reducing the noteworthiness ranking, determining the noteworthiness ranking based on:

a) the composite score for each response grouping;

b) a change in the composite score for the response grouping from a previous composite score; and

c) the actionability of the response grouping.

30. The method of claim 10 wherein at least one response grouping of the displayed response groupings comprises a combination of two or more of the respondent segments and the plurality of survey items.

31. The method of claim 27 wherein the second segment is a combination of two or more sequenced segments.

32. The method of claim 27 wherein the second segment comprises a combination of all but one segment belonging to a category of segments.

33. The method of claim 10 wherein the noteworthiness ranking of a response grouping is further based on whether an aspect of the response grouping falls below or above a predetermined threshold.

34. The method of claim 1 wherein:

a) each corresponding image has a size that is based on;

i) for each respondent segment, at least one of a total number of the survey respondents that correspond with the respondent segment, a total number of possible survey respondents that correspond with the respondent segment, and an aggregate salary associated with the respondent segment; and

ii) for each of the plurality of items, an item multiplier.

35. The method of any of the preceding claims claim 1 wherein each item has a corresponding change indicator that indicates a magnitude of change in the composite score.

37. The method of claim 1 wherein, for at least one of the displayed response groupings, the display includes:

a) a polarization indicator indicating a polarization in responses for the response grouping;

or

b) a coalescence indicator indicating a coalescence in responses for the response grouping.

38. The method of claim 1 further comprising:

a) receiving additional responses;

b) for each item and respondent segment, automatically determining new composite scores based on the additional responses; and

c) changing the display in accordance with the new composite scores.

39. The method of claim 1 wherein the display further includes, for each item, a plurality of corresponding images and related composite scores for different times when survey responses were received, each of the plurality of corresponding images being located:

a) with respect to the first axis, based on the composite score for the response grouping; and

b) with respect to the second axis, based on the time when the survey responses related to the corresponding image were received.

2. (canceled)

3. (canceled)

7. (canceled)

8. (canceled)

14. (canceled)

17. (canceled)

19. (canceled)

20. (canceled)

21. (canceled)

22. (canceled)

24. (canceled)

25. (canceled)

36. (canceled)

40. A method of displaying survey results, the method comprising:

a) for each of a plurality of survey items, receiving survey responses from survey respondents, wherein;

i) each response is chosen from response options, the response options corresponding to an ordinal scale; and

ii) each respondent is associated with a collection of respondent segments; and

b) for each response to each item by each survey respondent, determining a calibrated score, the calibrated score based on a probability that a person having the collection of respondent segments associated with the survey respondent would provide either;

i) the response to the item or any of the response options to the item that are lower on the ordinal scale;

or

ii) the response to the item or any of the response options to the item that are higher on the ordinal scale;

c) determining at least one of;

i) for each item, a composite score based on the calibrated score for each response to the item; and

ii) for each respondent segment, a composite score based on the calibrated score for each response associated with the respondent segment; and

d) providing a display of images, each image corresponding with one of the item or respondent segments;

e) wherein each image is located with respect to a first axis based on the composite score associated with the image and its corresponding item or segment.

41. The method of claim 40 wherein each image is located with respected to a second axis based on a change in the composite score associated with the image and its corresponding item or segment.

42. (canceled)

43. (canceled)

44. (canceled)

45. (canceled)

46. A system for displaying survey results, the system comprising:

a) respondent devices configured to receive survey items and communicate survey responses to the items; and

b) a server configured to carry out the steps of;

i) for each item, receiving survey responses from respondent devices, wherein;

(1) each response is chosen from response options, the response options corresponding to an ordinal scale; and

(2) each respondent is associated with a collection of respondent segments; and

ii) for each response to each item by each survey respondent, determining a calibrated score, the calibrated score based on a probability that a person having the collection of respondent segments associated with the survey respondent would provide either;

(1) the response to the item or any of the response options to the item that are lower on the ordinal scale;

or

(2) the response to the item or any of the response options to the item that are higher on the ordinal scale;

iii) for each item, determining a composite score based on the calibrated score for each response to the item;

iv) for each respondent segment, determining a composite score based on the calibrated score for each response associated with the respondent segment; and

v) providing a display of response groupings, the displayed response groupings comprising at least a portion of the plurality of items and the collection of respondent segments;

vi) wherein the response groupings are located on the display along a first axis; and

vii) wherein the display includes, for each response grouping, a corresponding image, the corresponding image being located with respect to the first axis or a distinct second axis based on the composite score for the response grouping.

47-53. -53. (canceled)