Improvements To Computer Based Reasoning and Artificial Intellignence Systems

• US 20190311220A1
• Filed: 04/05/2019
• Published: 10/10/2019
• Est. Priority Date: 04/09/2018
• Status: Active Grant

• Current Assignee: Howso Incorporated
• Original Assignee: Diveplane Corporation
• Inventors: Hazard, Christopher James, Fusting, Christopher, Resnick, Michael
• Granted Patent: US 10,817,750 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 18/2148 characterised by the process organisation or structure, e.g. boosting cascade
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks

Abstract

Techniques are provided herein for creating well-balanced computer-based reasoning systems and using those to control systems. The techniques include receiving a request to determine whether to use one or more particular data elements, features, cases, etc. in a computer-based reasoning model (e.g., as data elements, cases or features are being added, or as part of pruning existing features or cases). Conviction measures (such as targeted or untargeted conviction, contribution, surprisal, etc.) are determined and inclusivity conditions are tested. The result of comparing the conviction measure can be used to determine whether to include or exclude the feature, case, etc. in the computer-based reasoning model. A controllable system may then be controlled using the computer-based reasoning model. Examples controllable systems include self-driving cars, image labeling systems, manufacturing and assembly controls, federated systems, smart voice controls, automated control of experiments, energy transfer systems, health care systems, cybersecurity systems, and the like.

Claims

1. A method comprising:

receiving, at one or more computing devices, a request to determine whether to include one or more particular data elements in a computer-based reasoning model;

determining, at the one or more computing devices, one or more conviction scores for the one or more particular data elements;

determining, at the one or more computing devices, whether the one or more conviction scores meet one or more inclusivity conditions;

in response to determining that the one or more conviction scores meet the one or more inclusivity conditions;

including the one or more particular data elements in the computer-based reasoning model when the inclusivity conditions comprise an inclusion condition;

excluding the one or more particular data elements in the computer-based reasoning model when the inclusivity conditions comprise an exclusion condition;

causing, with a control system, control of a controllable system with the computer-based reasoning model,

wherein the method is performed on one or more computing devices.

2. The method of claim 1,

wherein the one or more particular data elements includes at least one label on at least one training image;

wherein causing control of the controllable system comprises causing control of a system that identifies elements of an image using the computer-based reasoning model by;

receiving an input image for labelling;

determining one or more labels for the input image based on the image and the computer-based reasoning model;

labelling the input image based on the one or more determined labels.

3. The method of claim 1,

wherein causing control of the controllable system comprises causing control of a vehicle using the computer-based reasoning model by:

receiving a current context for the vehicle, wherein the vehicle can be controlled using the computer-based reasoning model;

determining an action to take for the vehicle based on the current context for the vehicle and the computer-based reasoning model;

causing the vehicle to perform the determined action.

4. The method of claim 1,

wherein determining one or more conviction scores for the one or more particular data elements comprises determining a prediction conviction score for the one or more particular data elements and determining a familiarity conviction score for the one or more particular data elements;

wherein determining whether the one or more conviction scores meet the inclusivity conditions comprise determining that the one or more particular data elements meet the exclusion condition when the prediction conviction score is high and the familiarity conviction score is low.

5. The method of claim 1,

wherein determining one or more conviction scores for the one or more particular data elements comprises determining a prediction conviction score for the one or more particular data elements and determining a familiarity conviction score for the one or more particular data elements;

wherein determining whether the one or more conviction scores meet the inclusivity conditions comprises determining that the one or more particular data elements meet the exclusion condition when the prediction conviction score is high and the familiarity conviction score is high.

6. The method of claim 1,

wherein determining one or more conviction scores for the one or more particular data elements comprises determining a prediction conviction score for the one or more particular data elements and determining a familiarity conviction score for the one or more particular data elements;

wherein determining whether the one or more conviction scores meet the inclusivity conditions comprises determining that the one or more particular data elements meet the exclusion condition when the prediction conviction score is high and the familiarity conviction score is low.

7. The method of claim 1,

wherein determining one or more conviction scores for the one or more particular data elements comprises determining a prediction conviction score for the one or more particular data elements and determining a familiarity conviction score for the one or more particular data elements;

wherein determining whether the one or more conviction scores meet the inclusivity conditions comprises determining that the one or more particular data elements meet the inclusion condition when the prediction conviction score is high and the familiarity conviction score is high.

8. The method of claim 1,

wherein receiving a request to determine whether to include one or more particular data elements comprises receiving a request to reduce the computer-based reasoning model to a particular size;

and the method further comprises;

determining a number of data elements to include in the computer-based reasoning model to reduce the computer-based reasoning model to a particular size;

determining a subset of data elements to include, that includes the number of data elements, to include in the computer-based reasoning model based at least in part on the one or more conviction scores for data elements in the computer-based reasoning model; and

including only the subset of data elements to include in the computer-based reasoning model, and excluding data elements from the computer-based reasoning model that are not in the subset of data elements to include.

9. The method of claim 1,

wherein receiving a request to determine whether to include one or more particular data elements comprises receiving a request to reduce the computer-based reasoning model to a particular size;

and the method further comprises;

determining a number of cases to exclude in the computer-based reasoning model to reduce the computer-based reasoning model by the particular size;

determining a subset of cases to exclude, that includes the number of cases, to exclude in the computer-based reasoning model based at least in part on the one or more conviction scores for cases in the computer-based reasoning model; and

excluding the subset of cases to exclude from the computer-based reasoning model.

10. The method of claim 1, further comprising:

initially receiving the one or more particular data elements as part of training for the computer-based reasoning model;

in response to determining that the one or more conviction scores meet the inclusion condition, sending an indication to a trainer associated with the training for the computer-based reasoning model to continue to train related to the one or more particular data elements;

in response to determining that the one or more conviction scores meet the exclusion condition, sending the indication to the trainer associated with the training for the computer-based reasoning model that training is no longer needed related to the one or more particular data elements.

11. The method of claim 1, further comprising:

receiving a request for an action to take in a current context;

determining the action to take based on comparing the current context to contexts associated with cases in the computer-based reasoning model; and

responding to the request for the action to take with the determined action.

12. The method of claim 11, further comprising:

receiving an indication that there was an anomaly associated with the determined action;

removing one or more cases associated with the determined action from the computer-based reasoning model.

13. The method of claim 11, further comprising:

receiving an indication that there was an error associated with the determined action;

adding, to the computer-based reasoning model, one or more additional cases with contexts associated with the current context, wherein the one or more additional cases would cause a determination that the current context is associated with one of the one or more additional cases, and would cause determination that the current context would be associated with a different action than the determined action associated with the error.

14. The method of claim 1, further comprising:

continuing to determine the one or more conviction scores for new cases and including or excluding those cases based on whether the one or more conviction scores meet the inclusivity conditions until a termination condition for inclusion or exclusion is met.

15. A system for performing a machine-executed operation involving instructions, wherein said instructions are instructions which, when executed by one or more computing devices, cause performance of a process comprising:

receiving a request to determine whether to include one or more particular data elements in a computer-based reasoning model;

determining one or more conviction scores for the one or more particular data elements;

determining whether the one or more conviction scores meet inclusivity conditions;

in response to determining that the one or more conviction scores meet the inclusivity conditions;

including the one or more particular data elements in the computer-based reasoning model when the inclusivity conditions comprise an inclusion condition;

excluding the one or more particular data elements in the computer-based reasoning model when the inclusivity conditions is an exclusion condition;

causing, with a control system, control of a controllable system with the computer-based reasoning model.

16. The system of claim 15,

wherein the one or more particular data elements includes at least one label on at least one training image;

wherein causing control of the controllable system comprises causing control of a system that identifies elements of an image using the computer-based reasoning model by;

receiving an input image for labelling;

determining one or more labels for the input image based on the image and the computer-based reasoning model;

labelling the input image based on the one or more determined labels.

17. The system of claim 15,

wherein determining one or more conviction scores for the one or more particular data elements comprises determining a prediction conviction score for the one or more particular data elements and determining a familiarity conviction score for the one or more particular data elements;

wherein determining whether the one or more conviction scores meet the inclusivity conditions comprise determining that the one or more particular data elements meet the exclusion condition when the prediction conviction score is high and the familiarity conviction score is low.

18. A non-transitory computer readable medium storing instructions which, when executed by one or more computing devices, cause the one or more computing devices to perform a process of:

receiving a request to determine whether to include one or more particular data elements in a computer-based reasoning model;

determining one or more conviction scores for the one or more particular data elements;

determining whether the one or more conviction scores meet inclusivity conditions;

in response to determining that the one or more conviction scores meet the inclusivity conditions;

including the one or more particular data elements in the computer-based reasoning model when the inclusivity conditions comprise an inclusion condition;

excluding the one or more particular data elements in the computer-based reasoning model when the inclusivity conditions is an exclusion condition;

causing, with a control system, control of a controllable system with the computer-based reasoning model.

19. The non-transitory computer readable medium of claim 18,

wherein causing control of the controllable system comprises causing control of a vehicle using the computer-based reasoning model by:

receiving a current context for the vehicle, wherein the vehicle can be controlled using the computer-based reasoning model;

determining an action to take for the vehicle based on the current context for the vehicle and the computer-based reasoning model;

causing the vehicle to perform the determined action.

20. The non-transitory computer readable medium of claim 18,

wherein determining one or more conviction scores for the one or more particular data elements comprises determining a prediction conviction score for the one or more particular data elements and determining a familiarity conviction score for the one or more particular data elements;

wherein determining whether the one or more conviction scores meet the inclusivity conditions comprises determining that the one or more particular data elements meet the exclusion condition when the prediction conviction score is high and the familiarity conviction score is high.