Systems and methods for assessment and optimization of real-time collaborative intelligence systems

• US 10,122,775 B2
• Filed: 09/18/2015
• Issued: 11/06/2018
• Est. Priority Date: 03/26/2014
• Status: Active Grant

• Current Assignee: Unanimous A.I., Inc.
• Original Assignee: Unanimous A.I., Inc.
• Inventors: Rosenberg, Louis B.
• Primary Examiner(s): Levy, Amy M
• Application Number:
  US14/859,035
• Publication Number:
  US 20160057182A1
• Time in Patent Office: 1,145 Days
• CPC Class Codes: E21B 47/13 by electromagnetic energy, e.g. radio frequency
• CPC Class Codes: E21B 47/18 through the well fluid , e.g. mud pressure pulse telemetry
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: H04L 65/403 Arrangements for multi-party communication, e.g. for conferences data switching systems for conference H04L12/18; arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities H04M3/56; television conferencing systems H04N7/15
• CPC Class Codes: H04L 67/10 in which an application is distributed across nodes in the network software deployment G06F8/60; multiprogramming arrangements G06F9/46
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04L 67/125 involving control of end-device applications over a network
• CPC Class Codes: H04L 67/14 Session management for real-time applications in data packet communications networks H04L65/1066

Abstract

Systems and methods for real-time collaborative computing and collective intelligence are disclosed. A collaborative application runs on a collaborative server connected to a plurality of computing devices. Collaborative sessions are run wherein a group of independent users, networked over the internet, collaboratively answer questions in real-time, thereby harnessing their collective intelligence. A user performance value is determined for each of a plurality of independent users in the group based on each user'"'"'s participation as compared to other users in the collaborative group. A group cohesiveness score is determined that quantifies the group'"'"'s overall collaborative effectiveness. In some embodiments, the group cohesiveness score and user performance values are used to adjust weighting factors that affect the relative impact of each of the plurality of users in the collaborative group. In some embodiments, adjusting the weighting factors is performed with the objective of increasing the effectiveness of the resulting collective intelligence.

Claims

1. A method for determining a user performance value of each user of a group of users participating in a plurality of real-time collaborative sessions over a time period, wherein each user of the group is associated with a computing device in networked communication with a collaboration server, wherein during each collaborative session each computing device in real-time repeatedly determines a user intent from user input to the computing device, the user intent having an angular direction, and sends the user intent to the collaboration server, the collaboration server performing the steps of:

receiving, during a first collaborative session, in real-time from each computing device the corresponding user intent;

determining, during the first collaborative session, in real-time a group intent, having an angular direction, from the plurality of real-time user intents received from the plurality of computing devices;

determining, during the first collaborative session, a degree of alignment between the angular direction of each user intent and the angular direction of the group intent;

determining, during the first collaborative session, the user performance value for each user based at least in part upon the degree of alignment between the user intent associated with that user and the group intent, whereby a user performance value is determined for each user for each of the multiple time steps;

determining, during the first collaborative session, a weighting value for each of the plurality of users based at least in part on the user performance value associated with the user;

receiving, during a second collaborative session including display on a display of each computing device of a graphical object in a location relative to a set of targets, in real-time from each computing device the corresponding user intent;

applying, during the second collaborative session, the weighting value to the corresponding user intent;

determining, during the second collaborative session, the real-time group intent based on the weighted user intents;

determining, during the second collaborative sessions, an updated graphical object location based on the plurality of weighted user inputs; and

sending, during the additional collaborative session, the updated graphical object location to the plurality of computing devices, whereby the updated graphical object location is displayed on each of the plurality of computing devices.

2. The method for determining the user performance value of claim 1, wherein the user performance value has an upper bound, wherein the user performance value at the upper bound indicates that the user intent was substantially aligned with the group intent.

3. The method for determining the user performance value of claim 1, wherein the user weighting value is determined such that the user weighting value increases the contribution of the user when the user intents and the group intents used to determine the user performance value are more aligned and decreases the contribution of the user when the user intents and group intents are less aligned.

4. The method for determining the user performance value of claim 1, wherein the user performance value has a lower bound, wherein the user performance value at the lower bound indicates that the user intent was substantially misaligned with the group intent.

5. The method for determining the user performance value of claim 4, wherein the user intent is determined to be substantially misaligned with the group intent when the degree of alignment is such that the user intent and group intent are substantially opposite.

6. The method for determining the user performance value of claim 1, wherein an average user performance value for one user over a second time period including a plurality of selected time steps is computed by finding the mean of the user performance values for the plurality of selected time steps.

7. The method for determining the user performance value of claim 6, wherein the second time period substantially coincides with the collaborative session time period over which the group collaboratively moves at least one of the at least one graphical object to a select an answer from a plurality of displayed choices in response to a displayed question.

8. The method for determining the user performance value of claim 6, wherein the average user performance value is time-weighted such that selected time steps near the end of the second time period are more heavily weighted than selected time steps near the start of the second time period.

9. The method for determining the user performance value of claim 1, wherein a number of credits are awarded to at least one user based at least in part on the user performance value of that user.

10. The method for determining the user performance value of claim 9, wherein the credits are redeemable by the user for the right to ask a question to the group, said question to be answered through an additional collaborative session.

11. he method for determining the user performance value of claim 1, further comprising the step of determining by the collaboration server of an ordered ranking of the plurality of users, the ordered ranking based at least in part upon at least one user performance value associated with each of the plurality of users.

12. The method for determining the user performance value of claim 1, wherein an assessment of the at least one user is determined based at least in part upon the user performance value of each user assessed.

13. The method for determining the user performance value of claim 12, wherein the assessment indicates where each assessed user is one of at least “

flexible”

or “

entrenched”

.

14. The method for determining the user performance value of claim 12, wherein the assessment indicates where each assessed user is one of at least “

constructive”

or “

destructive”

.

15. The method for determining the user performance value of claim 1, comprising ejecting at least one user from the group based at least in part upon one user performance value associated with that user.

16. The method for determining the user performance value of claim 1, further comprising selectively allowing a new user to join the group based at least in part upon a previously generated user performance value associated with the new user, said previously generated user performance value indicating a performance level of the new user in at least one prior collaborative session.

17. A method for determining a group cohesiveness score of a group of individual users participating in a collaborative control session over a time period, wherein each user of the group is associated with an individual computing device in networked communication with a collaboration server, wherein during the collaborative session each computing device in real-time repeatedly determines a user intent from user input to the computing device, the user intent having an angular direction, and sends the user intent to the collaboration server, the collaboration server during each of multiple time steps of the collaborative session performing the steps of:

receiving in real-time from each of the computing device the corresponding user intent;

determining in real-time a group intent, having an angular direction, from the plurality of real-time user intents received from the plurality of computing devices;

determining in real-time a degree of alignment between the angular direction of each user intent and the group intent;

determining an average of the degree of alignment for the group;

determining a running average of the degree of alignment for the collaborative session over a selection of the multiple time steps;

determining in real-time a group cohesiveness score based at least in part on the running average of the degree of alignment; and

modulating the make-up of the group by at least one selected from the group of actions comprising ejecting at least one user from the group based at least in part upon a determination that the ejected user'"'"'s real-time user intents were poorly aligned with the group intents over a period of time and selectively allowing a new user to join the group based at least in part upon a previously generated user performance value associated with the new user, said previously generated user performance value indicating a performance level of the new user in at least one prior collaborative session.

18. The method for determining the group cohesiveness score of claim 17 wherein the collaborative session includes the group of individual users responding to a question by moving at least one graphical object to one of a plurality of input choices displayed on each computing device.

19. The method for determining the group cohesiveness score of claim 17 wherein the group cohesiveness score has an upper bound wherein the upper bound indicates that the plurality of real-time user intents are substantially aligned with each other.

20. The method for determining the group cohesiveness score of claim 17, wherein the group cohesiveness score has a lower bound wherein the lower bound that indicates that the plurality of real-time user intents are substantially misaligned with each other.

21. The method for determining the group cohesiveness score of claim 20, wherein the plurality of real-time user intents are determined to be substantially misaligned when a summation of the user intent directions results in a substantially zero resultant.

22. The method for determining the group cohesiveness score of claim 17, further comprising the step of determining an average group cohesiveness score for the group of users for a second time period including a plurality of selected time steps by finding the mean of the group cohesiveness scores during the plurality of selected time steps.

23. The method for determining the group cohesiveness score of claim 22, wherein the mean is time-weighted such that selected time steps near the end of the second time period are more heavily weighted than selected time steps near the start of the second time period.

24. The method for determining the group cohesiveness score of claim 22, wherein the mean group cohesiveness score is included in a social media message that also includes an associated answer collaboratively selected by the group.

25. The method for determining the group cohesiveness score of claim 17, wherein a number of credits are awarded to each of the individual users based at least in part on the group cohesiveness score.

26. The method for determining the group cohesiveness score of claim 25, wherein the credits are redeemable by each of the users for the right to ask a question to the group of users, said question to be answered through an additional collaborative session.

27. The method for determining the group cohesiveness score of claim 17, further comprising the step of ranking of the group of users with respect to additional groups of users based at least in part upon at least one group cohesiveness score.

28. The method for determining the group cohesiveness score of claim 17, further comprising the step of providing an assessment of the group based at least in part on at least one of the group cohesiveness scores.

29. The method for determining the group cohesiveness score of claim 28, wherein the assessment indicates where the group is one of at least “

flexible”

or “

entrenched”

.

30. The method for determining the group cohesiveness score of claim 17, wherein the period of time includes the collaborative session and at least one additional collaborative session.

31. The method for determining the group cohesiveness score of claim 17, further comprising the step of repeatedly reporting and repeatedly displaying the group cohesiveness score to each of the plurality of computing devices.

32. The method for determining the group cohesiveness score of claim 17, wherein the group cohesiveness score is expressed as a percentage from 0% to 100%.