Artificial intelligence engine configured to work with a pedagogical programming language to train one or more trained artificial intelligence models

• US 10,671,938 B2
• Filed: 01/26/2017
• Issued: 06/02/2020
• Est. Priority Date: 01/27/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Bonsai AI, Inc. (Microsoft Corporation)
• Inventors: Hammond, Mark Isaac, Browne, Keen McEwan, Adams, Megan
• Primary Examiner(s): Hicks, Austin
• Application Number:
  US15/417,056
• Publication Number:
  US 20170213126A1
• Time in Patent Office: 1,223 Days
• CPC Class Codes: G06F 15/80 comprising an array of processing units with common control, e.g. single instruction multiple data processors G06F15/82 takes precedence ; for correlation function computation G06F17/15
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 18/2148 characterised by the process organisation or structure, e.g. boosting cascade
• CPC Class Codes: G06F 3/03543 Mice or pucks G06F3/03541 takes precedence
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 30/20 Design optimisation, verification or simulation optimisation, verification or simulation of circuit designs G06F30/30
• CPC Class Codes: G06F 30/333 Design for testability [DFT], e.g. scan chain or built-in self-test [BIST]
• CPC Class Codes: G06F 40/166 Editing, e.g. inserting or deleting
• CPC Class Codes: G06F 8/31 Programming languages or programming paradigms
• CPC Class Codes: G06F 8/311 Functional or applicative languages; Rewrite languages
• CPC Class Codes: G06F 8/38 for implementing user interfaces
• CPC Class Codes: G06F 8/437 Type checking
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06F 9/4881 Scheduling strategies for dispatcher, e.g. round robin, multi-level priority queues
• CPC Class Codes: G06F 9/5066 Algorithms for mapping a plurality of inter-dependent sub-tasks onto a plurality of physical CPUs mappping at compile time, see G06F8/451
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 3/008 based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
• CPC Class Codes: G06N 3/04 Architecture, e.g. interconnection topology
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 : Learning methods
• CPC Class Codes: G06N 3/105 : Shells for specifying net layout
• CPC Class Codes: G06N 5/04 : Inference or reasoning models
• CPC Class Codes: G06Q 10/00 : Administration; Management
• CPC Class Codes: H04L 67/01 : Protocols
• CPC Class Codes: H04L 67/02 : based on web technology, e.g. hypertext transfer protocol [HTTP]

Abstract

Provided in some embodiments is an artificial intelligence (“AI”) engine configured to work with a pedagogical programming language configured to enable an author to 1) define a mental model to be learned by an AI model, the mental model including an input, one or more concept nodes, one or more stream nodes, and an output, as well as 2) define one or more curriculums for training the AI model respectively on the one or more concept nodes. A compiler can be configured to generate an assembly code from a source code authored in the pedagogical programming language. An architect module can be configured to propose a neural-network layout from the assembly code. A learner module can be configured to build the AI model the neural-network layout. An instructor module can be configured to train the AI model on the one or more concept nodes respectively with the one or more curriculums.

Claims

1. An artificial intelligence (“

AI”

) engine configured to work with a pedagogical programming language, comprising;

a compiler in a memory configured for execution by one or more processors to generate an assembly code from a source code written in the pedagogical programming language,

wherein the pedagogical programming language is configured to enable an author to define a mental model to be learned by an AI model, the mental model including an input, one or more concept nodes, and an output, and

wherein the pedagogical programming language is further configured to enable an author to define one or more curriculums for training the AI model respectively on the one or more concept nodes; and

one or more AI-engine modules in the memory including an architect module, an instructor module, and a learner module,

wherein the architect module upon execution of the architect module by the one or more processors is configured to propose a neural-network layout with one or more neural-network layers from the assembly code,

wherein the learner module upon execution of the learner module by the one or more processors is configured to build the AI model with the one or more neural-network layers from the neural-network layout proposed by the architect module,

wherein the instructor module upon execution of the instructor module by the one or more processors is configured to train the AI model built by the learner module on the one or more concept nodes respectively with the one or more curriculums, and

wherein the instructor module is further configured to analyze code in the pedagogical programming language in order to find a starting point among the one or more concept nodes.

2. The AI engine of claim 1,

wherein the architect module is further configured to parse scripted code in the pedagogical programming language in order to map the one or more concept nodes of the mental model on the one or more network layers of the AI model, and

wherein the instructor module is further configured to analyze the scripted code in the pedagogical programming language in order to find the starting point among the one or more concept nodes and the corresponding one or more curriculums for training the AI model.

3. The AI engine of claim 1,

wherein the instructor module is further configured to train the AI model in one or more training cycles with training data from one or more training-data sources selected from a simulator, a training-data generator, a training-data database, or a combination thereof based on training parameters in the pedagogical programming language, and

wherein the instructor module is further configured to terminate neural-network training upon an instructor module-determined terminating condition.

4. The AI engine of claim 1,

wherein the pedagogical programming language is further configured to enable an author to define schemas describing one or more data types to be streamed through the mental model including the one or more concept nodes and one or more stream nodes in the mental model.

5. The AI engine of claim 4,

wherein the compiler is further configured to match-check the schemas and report one or more errors if the schemas expected to match do not match, and

wherein the match-check is structural in that any pair of the schemas with different schema names match if the pair of the schemas define a same list of field types in a same order.

6. The AI engine of claim 4,

wherein the data types include primitive data types selected from strings, integers, floats, and Booleans;

structured data types;

constrained data types with range expressions limiting the data of the constrained data types; and

combinations thereof.

7. The AI engine of claim 1,

wherein the pedagogical programming language is further configured to enable an author to define each of the one or more concept nodes by name;

prediction type;

output; and

one or more connecting concept nodes in the mental model including leading, input-providing concept nodes and following, output-accepting concept nodes.

8. The AI engine of claim 1,

wherein the pedagogical programming language is further configured to enable an author to define each of the one or more curriculums by name;

the concept node to be trained;

training data-source type;

training data-source name;

one or more lessons of the curriculum for the concept node to be trained; and

objective including a termination condition for the concept node to be trained.

9. The AI engine of claim 8,

wherein the pedagogical programming language is further configured to enable an author to define each of the one or more lessons by name;

a leading lesson to follow;

data configuration for the lesson; and

the termination condition for the objective for the concept node to be trained.

10. The AI engine of claim 1,

wherein the pedagogical programming language is further configured to enable an author to use and reuse one or more blocks in the mental model,

wherein each of the one or more blocks includes one or more block-encapsulated concept nodes, optionally one or more block-encapsulated stream nodes, and one or more block-encapsulated schemas without any curriculums for training the AI model, and

wherein each of the one or more blocks is configured to

accept input from any one or more of the one or more concept nodes or the one or more stream nodes of the mental model and

provide output to any other one of the one or more concept nodes or the one or more stream nodes of the mental model.

11. An artificial intelligence (“

AI”

) system configured to work with a pedagogical programming language, comprising;

one or more remote servers including

i) a compiler in a server memory configured for execution by one or more server processors to generate an assembly code from a source code written in the pedagogical programming language,

wherein the pedagogical programming language is configured to enable an author to define a mental model to be learned by an AI model, the mental model including an input, one or more concept nodes, and an output, and

wherein the pedagogical programming language is further configured to enable an author to define one or more curriculums for training the AI model respectively on the one or more concept nodes;

ii) one or more AI-engine modules in the server memory including an architect module, an instructor module, and a learner module,

wherein the architect module upon execution of the architect module by the one or more processors is configured to propose a neural-network layout with one or more neural-network layers from the assembly code,

wherein the learner module upon execution of the learner module by the one or more processors is configured to build the AI model with the one or more neural-network layers from the neural-network layout proposed by the architect module, and

wherein the instructor module upon execution of the instructor module by the one or more processors is configured to train the AI model built by the learner module on the one or more concept nodes respectively with the one or more curriculums; and

iii) one or more server-side client-server interfaces configured to enable client interactions with the one or more AI engine modules;

wherein the one or more remote servers are configured to interact with

one or more local clients including

i) a coder in a client memory configured for execution by one or more client processors to enable an author to generate the source code written in the pedagogical programming language; and

ii) one or more client-side client-server interfaces configured to enable the client interactions with the AI engine in one or both client interactions selected from

submitting the source code for training the AI model and

using a trained AI model for one or more predictions based upon training data,

wherein the AI system includes at least one server-side training-data source or at least one client-side training-data source.

12. A machine-readable storage device configured to store data and instructions, which, when executed by one or more processors on a computing device, causes the following operations, comprising:

enabling an author to define a mental model to be learned by an AI model using a pedagogical programming language, the mental model including an input, one or more concept nodes, and an output, wherein the pedagogical programming language is further configured to enable an author to use and reuse one or more blocks in the mental model, wherein each of the one or more blocks includes one or more block-encapsulated concept nodes;

enabling an author to define one or more curriculums for training the AI model respectively on the one or more concept nodes using the pedagogical programming language;

executing a compiler in a memory by one or more processors to generate an assembly code from a source code written in the pedagogical programming language; and

executing one or more artificial intelligence (“

AI”

)-engine modules including an architect module, an instructor module, and a learner module in the memory by the one or more processors to

propose by the architect module a neural-network layout with one or more neural-network layers from the assembly code,

build by the learner module the AI model with the one or more neural-network layers from the neural-network layout proposed by the architect module, and

train by the instructor module the AI model built by the learner module on the one or more concept nodes respectively with the one or more curriculums.

13. The machine-readable storage device of claim 12,

wherein, upon executing the architect module, the architect module further

parses scripted code in the pedagogical programming language and maps the one or more concept nodes of the mental model on the one or more network layers of the AI model, and

wherein, upon executing the instructor module, the instructor module further

analyzes the scripted code in the pedagogical programming language and finds a starting point among the one or more concept nodes and the corresponding one or more curriculums for training the AI model.

14. The machine-readable storage device of claim 12,

wherein, upon executing the instructor module, the instructor module further

trains the AI model in one or more training cycles with training data from one or more training-data sources selected from a simulator, a training-data generator, a training-data database, or a combination thereof based upon training parameters in the pedagogical programming language, and

terminates neural-network training upon an instructor module-determined terminating condition.

15. The machine-readable storage device of claim 12, further comprising

enabling an author to define schemas using the pedagogical programming language, the schemas describing one or more data types to be streamed through the mental model including the one or more concept nodes and one or more stream nodes.

16. The machine-readable storage device of claim 15,

wherein, upon executing the compiler, the compiler further

match-checks the schemas and reports one or more errors if the schemas expected to match do not match, and

wherein the match-check is structural in that any pair of the schemas with different schema names match if the pair of the schemas define a same list of field types in a same order.

17. The machine-readable storage device of claim 12, further comprising

enabling an author to define each of the one or more concept nodes by name;

prediction type;

output; and

one or more connecting concept nodes in the mental model including leading, input-providing concept nodes and following, output-accepting concept nodes using the pedagogical programming language.

18. The machine-readable storage device of claim 12, further comprising

enabling an author to define each of the one or more curriculums by name;

the concept node to be trained;

training data-source type;

training data-source name;

one or more lessons of the curriculum for the concept node to be trained; and

objective including a termination condition for the concept node to be trained using the pedagogical programming language.

19. The machine-readable storage device of claim 18, further comprising

enabling an author to define each of the one or more lessons by name;

a leading lesson to follow;

data configuration for the lesson; and

the termination condition for the objective for the concept node to be trained using the pedagogical programming language.

20. The machine-readable storage device of claim 12, further comprising

enabling an author to use and reuse one or more blocks in the mental model using the pedagogical programming language,

wherein each of the one or more blocks includes one or more block-encapsulated concept nodes, one or more block-encapsulated stream nodes, and one or more block-encapsulated schemas without any curriculums for training the AI model, and

wherein each of the one or more blocks is configured to

accept input from any one or more of the one or more concept nodes or the one or more stream nodes of the mental model and

provide output to any other one of the one or more concept nodes or the one or more stream nodes of the mental model.