System, method, and recording medium for validating computer documentation

• US 10,318,246 B2
• Filed: 01/12/2018
• Issued: 06/11/2019
• Est. Priority Date: 08/19/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Castelli, Vittorio, Florian, Radu, Moon, Taesun, Sil, Avirup
• Primary Examiner(s): Khatri, Anil
• Application Number:
  US15/870,021
• Publication Number:
  US 20180136908A1
• Time in Patent Office: 515 Days
• Field of Search: 717101-103, 717109-113, 717136-137, 717140-141
• CPC Class Codes: G06F 11/36 Preventing errors by testing or debugging software
• CPC Class Codes: G06F 11/3604 Software analysis for verifying properties of programs byte-code verification G06F9/44589
• CPC Class Codes: G06F 40/10 Text processing natural language analysis G06F40/20; semantic analysis G06F40/30; processing or translation of natural language G06F40/40
• CPC Class Codes: G06F 8/20 Software design
• CPC Class Codes: G06F 8/73 Program documentation

Abstract

A computer-implemented computer documentation validation method, system, and computer program product for discovering problems and errors in an entire set of program/software documentation written by humans and flagging the documentation errors to suggest corrections to humans.

Claims

1. A computer-implemented computer documentation validation method, the method comprising:

translating a natural language of a computer software documentation into a machine instruction that manipulates a user interface of an operating system by taking control of a user input device to execute a command based on the natural language of the computer software documentation on behalf of the user;

executing the translated machine instruction on a software product configured for the computer software documentation; and

highlighting a location in the computer software documentation where an error occurs during the executing the translated machine instruction,

wherein the machine instruction includes a back pointer indicating the location in the computer software documentation from which the machine instruction is translated, and

wherein the error comprises a failure in the natural language of the computer software documentation being executed as the machine instruction.

2. The computer-implemented method of claim 1, wherein the natural language of the computer software documentation comprises text written by a human.

3. The computer-implemented method of claim 1, wherein the translating translates the computer software documentation into the machine instruction by processing the computer software documentation through a natural language processing pipeline.

4. The computer-implemented method of claim 3, wherein the natural language processing pipeline process is selected from a group consisting of:

tokenization;

tagging;

parsing; and

mental detection.

5. The computer-implemented method of claim 1, wherein the translating performs the translating by a reinforcement learning agent that learns an accuracy of the translating through an iterative process of repeatedly executing the machine instruction on the software that has been previously translated.

6. The computer-implemented method of claim 5, wherein the highlighting highlights the back pointer when the detecting detects the error such that the location of the error can be back traced by a human.

7. The computer-implemented method of claim 5, wherein the executing occurs within an environment by executing each of the machine instructions as a model returned to the reinforcement learning agent, and

wherein the reinforcement learning agent receives feedback from the environment each time the model of the machine instruction is executed such that the accuracy is learned from multiple executions of the translated machine instruction.

8. The computer-implemented method of claim 7, wherein the reinforcement agent repeats the execution of the machine instruction to receive additional feedback to refine the accuracy of the translating, and

wherein the reinforcement learning agent stops repeating the execution of the machine instruction when the machine instruction converges in the environment or a predetermined number of iterations of the reinforcement learning agent has occurred.

9. The computer-implemented method of claim 7, wherein the translating is made more robust by learning a background agent that:

accumulates each model for a plurality of different software and computer software documentation pairs;

adapts the accumulated set of models to a new software and computer software documentation pair; and

incorporates the adapted models into a background model for the new software and computer software documentation pair.

10. The computer-implemented method of claim 1, wherein the software comprises software that the computer software documentation was written for by a human.

11. The computer-implemented method of claim 1, wherein the error comprises:

a captured exception in the execution of the machine instruction; and

a sustained inactivity in the software during the execution of the machine instruction.

12. The computer-implemented method of claim 1, embodied in a cloud-computing environment.

13. The method of claim 1, wherein the back pointers includes which part of the computer software documentation was translated to the machine instruction such that the source of an error is identified.

14. A computer program product for computer documentation validation, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to cause the computer to perform:

translating a natural language of a computer software documentation into a machine instruction that manipulates a user interface of an operating system by taking control of a user input device to execute a command based on the natural language of the computer software documentation on behalf of the user;

executing the translated machine instruction on a software product configured for the computer software documentation; and

highlighting a location in the computer software documentation where an error occurs during the executing the translated machine instruction,

wherein the machine instruction includes a back pointer indicating the location in the computer software documentation from which the machine instruction is translated, and

wherein the error comprises a failure in the natural language of the computer software documentation being executed as the machine instruction.

15. The computer program product of claim 14, wherein the natural language of the computer software documentation comprises text written by a human.

16. The computer program product of claim 14, wherein the translating performs the translating by a reinforcement learning agent that learns an accuracy of the translating through an iterative process of repeatedly executing the machine instruction on the software that has been previously translated.

17. The computer program product of claim 14, wherein the natural language processing pipeline process is selected from a group consisting of:

tokenization;

tagging;

parsing; and

mental detection.

18. A computer documentation validation system, said system comprising:

a processor, and

a memory, the memory storing instructions to cause the processor to;

translating a natural language of a computer software documentation into a machine instruction that manipulates a user interface of an operating system by taking control of a user input device to execute a command based on the natural language of the computer software documentation on behalf of the user;

executing the translated machine instruction on a software product configured for the computer software documentation; and

highlighting a location in the computer software documentation where an error occurs during the executing the translated machine instruction,

wherein the machine instruction includes a back pointer indicating the location in the computer software documentation from which the machine instruction is translated, and

wherein the error comprises a failure in the natural language of the computer software documentation being executed as the machine instruction.

19. The system of claim 18, embodied in a cloud-computing environment.