System and method for consensus-based knowledge validation, analysis and collaboration
First Claim
1. A method for consensus-based validation and analysis of response data gathered from a plurality of panelists in response to a plurality of items regarding a subject matter, the method comprising:
- a host device collecting a set of response data from the plurality of panelists for the plurality of items;
the host device formatting the set of response data from the plurality of panelists into a response data matrix in accordance with a predetermined data model;
the host device estimating from the response data matrix an empirical point estimate matrix indicative of the amount of agreement in the set of response data on the plurality of items between two or more panelists;
the host device determining a saliency of the subject matter to the plurality of panelists;
the host device estimating a competency of the plurality of panelists based on the empirical point estimate matrix; and
,the host device computing a consensus model of correct answers based on the estimated competency of the plurality of panelists and the set of response data for the plurality of items.
5 Assignments
0 Petitions
Accused Products
Abstract
A consensus-based knowledge validation and analysis system provides a way to increase use of collaboration tools among panels of experts by providing a system for analyzing and validating the responses of such experts to a set of questions. The system uses a set of response data input by a panel of experts with respect to a particular subject matter formatted in accordance with a data model as input. The response data set is used to estimate an empirical point estimate matrix indicative of the amount of agreement in the responses on all items between the panelists. The empirical point estimate matrix is used to estimate the saliency of the subject matter to panelists, the competency of each panelist and a consensus model of correct answers is based on the estimated competency of each panelist and the of responses for each item in the response data set. This consensus model is used to generate a knowledge map to aid visualization of the consensus data and encourage further collaboration and consensus building. The method is implemented in a web-based system that enables users of collaboration tools to send response data sets to the tool via the Internet or virtual private network and to likewise retrieve knowledge maps, panelist information and consensus data. An interactive feature enables users/panelists to collaborate with other panelists using the knowledge map as an interface to one or more collaboration tools such as instant messaging.
17 Citations
42 Claims
-
1. A method for consensus-based validation and analysis of response data gathered from a plurality of panelists in response to a plurality of items regarding a subject matter, the method comprising:
-
a host device collecting a set of response data from the plurality of panelists for the plurality of items; the host device formatting the set of response data from the plurality of panelists into a response data matrix in accordance with a predetermined data model; the host device estimating from the response data matrix an empirical point estimate matrix indicative of the amount of agreement in the set of response data on the plurality of items between two or more panelists; the host device determining a saliency of the subject matter to the plurality of panelists; the host device estimating a competency of the plurality of panelists based on the empirical point estimate matrix; and
,the host device computing a consensus model of correct answers based on the estimated competency of the plurality of panelists and the set of response data for the plurality of items. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8)
-
-
9. A method for consensus-based validation and analysis of one or more sets of response data gathered from a plurality of panelists in response to a plurality of items regarding a subject matter using at least one collaborative modeling tool, the method comprising:
-
a host device formatting at least one of the one or more sets of response data and inputting the at least one of the one or more sets of response data from the collaborative modeling tool into a response data matrix in accordance with a predetermined data model; the host device estimating from the response data matrix an empirical point estimate matrix indicative of an amount of agreement in responses on the plurality of items between two or more panelists; the host device determining a saliency of the subject matter to the plurality of panelists; the host device estimating a competency for the plurality of panelists based on the empirical point estimate matrix; and
,the host device computing a consensus model of correct answers based on the estimated competency of the plurality of panelists and responses for the plurality of items. - View Dependent Claims (10, 11, 12, 13, 14)
-
-
15. A system for the validation and analysis of knowledge from a set of response data collected from a plurality of panelists regarding a subject matter using a collaborative modeling tool, the system comprising:
-
means for formatting the set of response data from the collaborative modeling tool into a response data matrix in accordance with a predetermined data model; means for estimating from the response data matrix an empirical point estimate matrix indicative of an amount of agreement in the set of response data on the plurality of items between the plurality of panelists; means for determining a saliency of the subject matter to the plurality of panelists; means for estimating a competency of the plurality of panelists based on the empirical point estimate matrix; and
,means for computing a consensus model of correct answers based on the estimated competency of the plurality of panelists and the set of response data for the plurality of items. - View Dependent Claims (16, 17, 18)
-
-
19. A method to compare a plurality of knowledge maps derived from a consensus model generated from a set of response data collected from a plurality of panelists using a collaborative modeling tool over a period of time, the method comprising:
-
a host device inputting a first set of response data from the plurality of panelists using a collaborative modeling tool at a first time; the host device computing a first consensus model from the first set of response data; the host device developing a first knowledge map from the first consensus model; the host device inputting a second set of response data from the plurality of panelists using the collaborative modeling tool at a second time, wherein the second time is different than the first time; the host device computing a second consensus model from the second set of response data; the host device developing a second knowledge map from the second consensus model; the host device performing a procrustes analysis of the second knowledge map against the first knowledge map to generate a realigned second knowledge map; the host device enabling display of the first knowledge map and second realigned knowledge maps. - View Dependent Claims (20, 21, 22, 23, 24)
-
-
25. A method of knowledge-based collaboration comprising:
-
a host device deriving a knowledge map from a set of response data collected from a plurality of panelists in response to a plurality of items regarding a subject matter the knowledge map comprising a graphical display of a comparison of responses of the plurality of panelists; the host device presenting the knowledge map to one or more of the plurality of panelists; the host device enabling a first panelist from the plurality of panelists to initiate communication with a second panelist from the plurality of panelists by facilitating a selection of an identifier associated with the second panelist on the knowledge map. - View Dependent Claims (26, 27, 28, 30, 31, 32, 34, 35, 36, 37, 38, 40, 42)
-
-
29. A system for the validation and analysis of knowledge from a set of response data collected from a plurality of panelists regarding a subject matter using a collaborative modeling tool, the system comprising:
-
a computing system; an application implemented on the computing system configured to collect a set of response data from a plurality of panelists from a plurality of client modeling tools through a communication link with a plurality of client hosts; a statistical programming element implemented on the application comprising at least one script configured to; format the set of response data and input the set of response data into a response data matrix in accordance with a predetermined data model; estimate from the response data matrix an empirical point estimate matrix indicative of an amount of agreement in responses on the plurality of items between two or more panelists; determine a saliency of the subject matter to the plurality of panelists; estimate a competency for the plurality of panelists based on the empirical point estimate matrix; and compute a consensus model of correct answers based on the estimated competency of the plurality of panelists and responses for the plurality of items; a database configured to store results of the script.
-
-
33. A tangible computer readable medium having instructions stored thereon, that if executed by a device, cause the device to perform a method for consensus-based validation and analysis of response data gathered from a plurality of panelists in response to a plurality of items regarding a subject matter, the method comprising:
-
collecting a set of response data from the plurality of panelists for the plurality of items; formatting the response data from the plurality of panelists into a response data matrix in accordance with a predetermined data model; estimating from the response data matrix an empirical point estimate matrix indicative of the amount of agreement in the set of response data on the plurality of items between two or more panelists; determining a saliency of the subject matter to the plurality of panelists; estimating a competency of the plurality of panelists based on the empirical point estimate matrix; and computing a consensus model of correct answers based on the estimated competency of the plurality of panelists and the set of response data for the plurality of items.
-
-
39. A method for providing consensus-based validation and analysis of response data gathered from a plurality of panelists in response to a plurality of items regarding a subject matter in web service definition language, the method comprising:
-
receiving a set of response data from a plurality of client modeling tools operating on a plurality of remote client hosts at a server host; providing an applet including a statistical programming element implemented on the server host comprising at least one script configured to; format the set of response data and input the set of response data into a response data matrix in accordance with a predetermined data model; estimate from the response data matrix an empirical point estimate matrix indicative of an amount of agreement in responses on the plurality of items between two or more panelists; determine a saliency of the subject matter to the plurality of panelists; estimate a competency for the plurality of panelists based on the empirical point estimate matrix; and compute a consensus model of correct answers based on the estimated competency of the plurality of panelists and responses for the plurality of items; storing results of the at least one script on a database; the applet executing in conjunction with a graphic interface web browser adapted to present through the graphic interface web browser a knowledge map comprising a graphical display comparing the plurality of panelists based on at least the estimated competencies of and relative differences between the plurality of panelists. - View Dependent Claims (41)
-
Specification