Managing a portfolio of experts
First Claim
1. A computer-implemented method of selecting an expert, the method comprising the steps of:
- receiving at least one task feature describing a characteristic of a task to be assigned to an expert;
receiving, for each of a plurality of experts, at least one expert feature describing a characteristic of that expert;
arranging a machine learning system to access a mapping from expert features and task features to a multi-dimensional latent trait space, that mapping having been learnt by the machine learning system; and
arranging a selection engine to, for each expert, map the task features and expert features to the multi-dimensional trait space using the mapping to produce a latent task trait and a latent expert trait and to combine the latent task trait and latent expert trait using a similarity measure to obtain an estimate of a probability distribution over the expert'"'"'s performance on the task; and
arranging the selection engine to select one of the experts on the basis of the estimated probability distributions.
2 Assignments
0 Petitions
Accused Products
Abstract
Managing a portfolio of experts is described where the experts may be for example, automated experts or human experts. In an embodiment a selection engine selects an expert from a portfolio of experts and assigns the expert to a specified task. For example, the selection engine has a Bayesian machine learning system which is iteratively updated each time an experts performance on a task is observed. For example, sparsely active binary task and expert feature vectors are input to the selection engine which maps those feature vectors to a multi-dimensional trait space using a mapping learnt by the machine learning system. In examples, an inner product of the mapped vectors gives an estimate of a probability distribution over expert performance. In an embodiment the experts are automated problem solvers and the task is a hard combinatorial problem such as a constraint satisfaction problem or combinatorial auction.
-
Citations
20 Claims
-
1. A computer-implemented method of selecting an expert, the method comprising the steps of:
-
receiving at least one task feature describing a characteristic of a task to be assigned to an expert; receiving, for each of a plurality of experts, at least one expert feature describing a characteristic of that expert; arranging a machine learning system to access a mapping from expert features and task features to a multi-dimensional latent trait space, that mapping having been learnt by the machine learning system; and arranging a selection engine to, for each expert, map the task features and expert features to the multi-dimensional trait space using the mapping to produce a latent task trait and a latent expert trait and to combine the latent task trait and latent expert trait using a similarity measure to obtain an estimate of a probability distribution over the expert'"'"'s performance on the task; and arranging the selection engine to select one of the experts on the basis of the estimated probability distributions. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11)
-
-
12. A computer-readable memory having stored thereon computer executable instructions to implement a selection engine comprising:
-
receiving at least one task feature describing a characteristic of a task to be assigned to an expert, the task feature comprising a sparsely active binary vector; and receiving, for each of a plurality of experts, at least one expert feature describing a characteristic of that expert, the expert feature comprising a sparsely active binary vector; a machine learning system comprising a mapping from expert features and task features to a multi-dimensional latent trait space, that mapping having been learnt by the machine learning system; and instructions to arrange a processor to; for each expert, map the task features and expert features to the multi-dimensional trait space using the mapping to produce a latent task trait and a latent expert trait and to combine the latent task trait and latent expert trait using a similarity measure to obtain an estimate of a probability distribution over the expert'"'"'s performance on the task; and select one of the experts on the basis of the estimated probability distributions. - View Dependent Claims (13, 14, 15, 16, 17)
-
-
18. An apparatus comprising:
-
an input arranged to receive at least one task feature describing a characteristic of a task to be assigned to an automated problem solver, the task feature comprising sparsely active binary vector; and the input also arranged to receive, for each of a plurality of automated problem solvers, at least one automated problem solver feature describing a characteristic of that automated problem solver, the automated problem solver feature comprising a sparsely active binary vector; and a machine learning system comprising a mapping from automated problem solver features and task features to a multi-dimensional latent trait space; and a processor arranged to, for each automated problem solver, map the task features and automated problem solver features to the multi-dimensional trait space using the mapping to produce a latent task trait and a latent expert trait and to combine the latent task trait and latent expert trait using a similarity measure to obtain an estimate of a probability distribution over the automated problem solver'"'"'s performance on the task; and the processor being arranged to select one of the automated problem solvers on the basis of the estimated probability distributions. - View Dependent Claims (19, 20)
-
Specification