User feedback in semi-automatic question answering systems
First Claim
1. A method performed by at least one computer processor executing computer program instructions tangibly stored on at least one non-transitory computer-readable medium,the method for use with a system including a data source and a first billing code,the method comprising using the at least one computer processor to perform operations of:
- (A) receiving input from a user, wherein the input represents a verification status of the first billing code;
(B) applying first inverse logic to the input, the billing code, and a set of forward logic, to identify first and second concept extraction components, wherein (B) comprises;
(B)(1) identifying a first logic component that generated the first billing code, wherein the first logic component comprises means for implementing first logic, wherein the first logic includes a first condition, wherein the first condition includes a first sub-condition and a second sub-condition; and
(B)(2) applying first inverse logic to the input received from the user to identify at least one of the first and second sub-conditions; and
(C) applying reinforcement to the first and second concept extraction components, comprising;
(B)(1) determining whether the verification status indicates that the first billing code is accurate;
(B)(2) if the verification status indicates that the first billing code is inaccurate, then applying negative reinforcement to the first and second concept extraction components, comprising apportioning the negative reinforcement between the first and second concept extraction components.
11 Assignments
0 Petitions
Accused Products
Abstract
A system applies rules to a set of documents to generate codes, such as billing codes for use in medical billing. A human operator provides input specifying whether the generated codes are correct. Based on the input from the human operator, the system attempts to identify which clause(s) in the rules which were relied on to generate the particular code are correct and which such clause(s) are incorrect. The system then assigns praise to components of the system responsible for generating codes in the correct clauses, and assigns blame to components of the system responsible for generating codes in the incorrect clauses. Such blame and praise may then be used to determine whether particular code-generating components are insufficiently reliable. The system may disable, or take other remedial action in response to, insufficiently reliable code-generating components.
-
Citations
20 Claims
-
1. A method performed by at least one computer processor executing computer program instructions tangibly stored on at least one non-transitory computer-readable medium,
the method for use with a system including a data source and a first billing code, the method comprising using the at least one computer processor to perform operations of: -
(A) receiving input from a user, wherein the input represents a verification status of the first billing code; (B) applying first inverse logic to the input, the billing code, and a set of forward logic, to identify first and second concept extraction components, wherein (B) comprises; (B)(1) identifying a first logic component that generated the first billing code, wherein the first logic component comprises means for implementing first logic, wherein the first logic includes a first condition, wherein the first condition includes a first sub-condition and a second sub-condition; and (B)(2) applying first inverse logic to the input received from the user to identify at least one of the first and second sub-conditions; and (C) applying reinforcement to the first and second concept extraction components, comprising; (B)(1) determining whether the verification status indicates that the first billing code is accurate; (B)(2) if the verification status indicates that the first billing code is inaccurate, then applying negative reinforcement to the first and second concept extraction components, comprising apportioning the negative reinforcement between the first and second concept extraction components. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. A non-transitory computer-readable medium comprising computer-readable instructions tangibly stored on the computer-readable medium, wherein the instructions are executable by at least one computer processor to perform a method for use with a system including a data source and a first billing code, the method comprising:
-
(A) receiving input from a user, wherein the input represents a verification status of the first billing code; (B) applying first inverse logic to the input, the billing code, and a set of forward logic, to identify first and second concept extraction components, wherein (B) comprises; (B)(1) identifying a first logic component that generated the first billing code, wherein the first logic component comprises means for implementing first logic, wherein the first logic includes a first condition, wherein the first condition includes a first sub-condition and a second sub-condition; and (B)(2) applying first inverse logic to the input received from the user to identify at least one of the first and second sub-conditions; and (C) applying reinforcement to the first and second concept extraction components, comprising; (B)(1) determining whether the verification status indicates that the first billing code is accurate; (B)(2) if the verification status indicates that the first billing code is inaccurate, then applying negative reinforcement to the first and second concept extraction components, comprising apportioning the negative reinforcement between the first and second concept extraction components. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19, 20)
-
Specification