MONITORING AN ARTIFICIAL INTELLIGENCE (AI) BASED PROCESS

• US 20200133816A1
• Filed: 01/22/2019
• Published: 04/30/2020
• Est. Priority Date: 10/30/2018
• Status: Active Grant

• Current Assignee: Accenture Global Solutions Limited (Accenture PLC)
• Original Assignee: Accenture Global Solutions Limited (Accenture PLC)
• Inventors: SONI, Soujanya, V V GANESHAN, Kavita, KALIKI, Aishwarya, SHIVARAM, Madhura, PATIL, Mandar Mohan
• Granted Patent: US 11,188,436 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 11/302 where the computing system component is a software system
• CPC Class Codes: G06F 11/3051 Monitoring arrangements for monitoring the configuration of the computing system or of the computing system component, e.g. monitoring the presence of processing resources, peripherals, I/O links, software programs verification or detection of system hardware configuration G06F11/2247
• CPC Class Codes: G06F 11/3086 where the reporting involves the use of self describing data formats, i.e. metadata, markup languages, human readable formats
• CPC Class Codes: G06F 11/328 Computer systems status display G06F11/327 takes precedence
• CPC Class Codes: G06F 11/3409 for performance assessment
• CPC Class Codes: G06F 11/3447 Performance evaluation by modeling
• CPC Class Codes: G06F 11/3466 Performance evaluation by tracing or monitoring
• CPC Class Codes: G06F 2201/81 Threshold
• CPC Class Codes: G06F 2201/865 Monitoring of software
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 8/427 Parsing
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: G06N 5/046 Forward inferencing; Production systems

Abstract

An Artificial Intelligence (AI)-based automated process is monitored via a process monitoring system that identifies components used in the execution of the sub-processes of the automated process. Various metrics are selected for collection prior to or during the execution of the AI-based automated process. The values of the metrics are collected as step outputs corresponding to the sub-processes. A final output generated upon the execution of the automated process is also collected. The step outputs can be used to determine the reason why the automated process produced a certain final output.

Claims

1. An artificial intelligence (AI)-based automated process monitoring system comprising:

at least one processor;

a non-transitory processor readable medium storing machine-readable instructions that cause the at least one processor to;

access code associated with an AI-based automated process that includes one or more sub-processes that implement at least one process component wherein the process produces a detectable output;

automatically identify the at least one process component via a component identification tool;

retrieve one or more metrics associated with the at least one process component, the metrics correspond to respective attributes of the at least one process component during an implementation of the process;

parse the code associated with the at least one process component;

collect values of the one or more metrics via parsing the code; and

generate an introspection output for the process, the introspection output including a respective step output for each of the one or more sub-processes, the respective step output including the values.

2. The AI-based automated process monitoring system of claim 1 wherein the at least one component includes a plurality of process components wherein each of the plurality of process components is associated with a respective set of metrics.

3. The AI-based automated process monitoring system of claim 2, wherein the instructions to retrieve one or more metrics cause the processor to:

select one of the respective set of metrics associated with the at least one process component.

4. The artificial AI-based automated process monitoring system of claim 1, wherein the component identification tool includes a component-specific wrapper Application Programming Interface (API).

5. The AI-based automated process monitoring system of claim 1, wherein the instructions to collect values of the one or more metrics cause the processor to:

execute code that includes access functions that collect the values of the one or more metrics, wherein the access functions are based on the at least one process component.

6. The AI-based automated process monitoring system of claim 1, wherein the instructions to collect values of the one or more metrics cause the processor to:

monitor one or more rule events fired by one or more process rules during an execution of the process; and

collect the values of the one or more metrics based on the rule events.

7. The AI-based automated process monitoring system of claim 1, wherein the instructions to collect values of the one or more metrics cause the processor to:

receive output values from explicitly configured functions in the code, wherein the output values correspond to the values of the metrics.

8. The AI-based automated process monitoring system of claim 1, wherein the instructions to generate the introspection output cause the processor to:

enable display of code that generates the introspection output.

9. The AI-based automated process monitoring system of claim 1, wherein the instructions to generate the introspection output cause the processor to:

access a predetermined template that includes natural language content conveying the values of the one or more metrics.

10. The AI-based automated process monitoring system of claim 1, wherein the non-transitory processor readable medium stores further instructions that cause the processor to:

compare the values of the metrics to predetermined thresholds; and

generate alerts based on the comparisons of the values.

11. A method of monitoring an automated artificial intelligence (AI)-based process comprising:

receiving process monitoring instructions for the AI-based automated process wherein the automated process includes at least one sub-process that implements one or more process components;

identifying the process components by employing respective component identification tools;

selecting one or more sets of metrics, wherein each set of metrics corresponds to attributes of a respective process component of the one or more process components;

collecting values associated with the sets of metrics during an execution of the AI-based automated process; and

generating an introspection output for the process, the introspection output including a respective step output for each of the sub-processes, the respective step output including the values.

12. The method of claim 11, wherein generating the introspection output further comprises:

generating within the introspection output, a summary which provides a final output based on the respective step outputs of each of the sub-processes.

13. The method of claim 11, wherein receiving the process monitoring instructions further comprises:

downloading a library that includes at least the component identification tools.

14. The method of claim 13, wherein downloading the library further comprises:

including wrapper application programming interfaces (APIs) specific to a respective process component of the one or more process components as the component identification tools.

15. The method of claim 11, wherein the AI-based automated process includes conversion of a textual manual into an executable set of instructions.

16. The method of claim 15, wherein the sub-processes include structural analysis of a textual manual, semantic analysis of textual content in the textual manual, obtaining domain specific knowledge, retrieving logic, generating process flow based on the logic and the domain specific knowledge and platform-specific code generation.

17. A non-transitory computer-readable storage medium comprising machine-readable instructions that cause a processor to:

access code associated with an AI-based process that includes one or more sub-processes that implement at least one process component wherein the process produces a detectable output;

automatically identify the at least one process component via a component identification tool;

retrieve one or more metrics associated with the at least one process component, the metrics correspond to respective attributes of the at least one process component during an implementation of the process;

parse the code associated with the at least one process component;

collect values of the one or more metrics via parsing the code; and

generate an introspection output for the process, the introspection output including a respective step output for each of the or more sub-processes, the respective step output including the values.

18. The non-transitory computer-readable storage medium of claim 17, wherein the instructions for generating the introspection output further comprising instructions that cause the processor to:

generate within the introspection output, a summary which provides a final output based on the respective step outputs of each of the sub-processes.

19. The non-transitory computer-readable storage medium of claim 17, further comprising instructions that cause the processor to:

download process monitoring instructions in a library that includes at least the component identification tools.

20. The non-transitory computer-readable storage medium of claim 19, wherein the instructions for receiving the process monitoring instructions further comprise instructions that cause the processor to:

include wrapper application programming interfaces (APIs) specific to a respective process component of the at least one process component as the component identification tools.