MODULAR MEMOIZATION, TRACKING AND TRAIN-DATA MANAGEMENT OF FEATURE EXTRACTION

• US 20180025092A1
• Filed: 07/21/2016
• Published: 01/25/2018
• Est. Priority Date: 07/21/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Aharonov, Ranit, Goldschmidt, Yaara, Ozery-Flato, Michal, Yanover, Chen
• Granted Patent: US 10,572,822 B2
• Time in Patent Office: Days
• CPC Class Codes: G06F 16/81 Indexing, e.g. XML tags; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/9024 Graphs; Linked lists G06F16/9027 takes precedence
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/01 Dynamic search techniques; Heuristics; Dynamic trees; Branch-and-bound

Abstract

There is provided, in accordance with some embodiments, a method for receiving electronic documents representing a dependency graph comprising feature extractors at each graph node and directed edges corresponding to computational dependencies of the feature extractors. For at least some feature extractors, extractor defining data, comprising extractor data and computational dependencies of the graph node in the dependency graph are determined, and a node lookup key based on the extractor defining data is computed. When the node lookup key is associated with a stored set of output feature values, the stored set is assigned as output values of the feature extractor. When node lookup key is not associated with a stored set of output feature values, a new set of output feature values is computed, stored, and associated the node lookup key. The one set of output feature values are sent as an output feature set.

Claims

1. A method comprising using at least one hardware processor for:

receiving at least one electronic document representing a dependency graph comprising feature extractors at each graph node and directed edges corresponding to computational dependencies of the feature extractors;

for at least some feature extractors;

i) determining extractor defining data, comprising extractor data and computational dependencies of said graph node in said dependency graph,

ii) computing a node lookup key based on said extractor defining data,

iii) when said node lookup key is associated with stored data, said stored data may be retrieved, wherein a non-transitory computer-readable storage medium has stored therein previously-computed node lookup keys and associated previously-computed data, and

iv) when node lookup key is not associated with stored data, computing new data, storing said new data on the non-transitory computer-readable storage medium, and associating said node lookup key with said new data; and

sending some of said stored data as an output set of said dependency graph.

2. The method of claim 1, wherein said stored data is one member from the group consisting of feature data, train data, and analysis results.

3. The method of claim 1, further comprising receiving a node lookup key and retrieving a dependency graph for said node lookup key.

4. The method of claim 1, further comprising when said node lookup key is associated with stored data, computing said new data.

5. The method of claim 1, wherein said extractor defining data comprises at least one child node lookup key.

6. The method of claim 1, wherein said output feature values comprise at least one member from a group consisting of metadata names, metadata types, and metadata descriptions.

7. The method of claim 1, wherein said extractor data comprises at least one member from a group consisting of an extractor class, extractor parameters, a cohort class, cohort parameters, and a cohort index-list.

8. A computer program product comprising a non-transitory computer-readable storage medium having program code embodied therewith, the program code being executable by at least one hardware processor to:

receive at least one electronic document representing a dependency graph comprising feature extractors at each graph node and directed edges corresponding to computational dependencies of the feature extractors;

for at least some feature extractors;

i) determine extractor defining data, comprising extractor data and computational dependencies of said graph node in said dependency graph,

ii) compute a node lookup key based on said extractor defining data,

iii) when said node lookup key is associated with stored data, said stored data may be retrieved, wherein a non-transitory computer-readable storage medium has stored therein previously-computed node lookup keys and associated previously-computed data, and

iv) when node lookup key is not associated with stored data, compute new data, store said new data on the non-transitory computer-readable storage medium, and associate said node lookup key with said new data; and

send some of said stored data as an output set of said dependency graph.

9. The computer program product of claim 8, wherein said stored data is one member from the group consisting of feature data, train data, and analysis results.

10. The computer program product of claim 8, further comprising receiving a node lookup key and retrieving a dependency graph for said node lookup key.

11. The computer program product of claim 8, further comprising when said node lookup key is associated with stored data, computing said new data.

12. The computer program product of claim 8, wherein said extractor data comprises at least one child node extractor lookup key.

13. The computer program product of claim 8, wherein said output feature values comprise at least one member from a group consisting of metadata names, metadata types, and metadata descriptions.

14. The computer program product of claim 8, wherein said extractor data comprises at least one member from a group consisting of an extractor class, extractor parameters, a cohort class, cohort parameters, and a cohort index-list.

15. A computerized system, comprising:

(a) a non-transitory computer-readable storage medium having stored thereon program code for;

receiving at least one electronic document representing a dependency graph comprising feature extractors at each graph node and directed edges corresponding to computational dependencies of the feature extractors;

for at least some feature extractors;

(1) determining extractor defining data, comprising extractor data and computational dependencies of said graph node in said dependency graph,

(2) computing a node lookup key based on said extractor defining data,

(3) when said node lookup key is associated with stored data, said stored data may be retrieved, wherein a non-transitory computer-readable storage medium has stored therein previously-computed node lookup keys and associated previously-computed data, and

(4) when node lookup key is not associated with stored data, computing new data, storing said new data on the non-transitory computer-readable storage medium, and associating said node lookup key with said new data;

sending stored data as an output set of said dependency graph, and

(b) at least one hardware processor configured to execute said program code.

16. The computerized system of claim 15, wherein said stored data is one member from the group consisting of feature data, train data, and analysis results.

17. The computerized system of claim 15, further comprising receiving a node lookup key and retrieving a dependency graph for said node lookup key.

18. The computerized system of claim 15, further comprising when said node lookup key is associated with stored data, computing said new data.

19. The computerized system of claim 15, wherein said extractor defining data comprises at least one child node extractor identification value.

20. The computerized system of claim 15, wherein said output feature values comprise at least one member from a group consisting of metadata names, metadata types, and metadata descriptions.

21. The computerized system of claim 15, wherein said extractor data comprises at least one member from a group consisting of an extractor class, extractor parameters, a cohort class, cohort parameters, and a cohort index-list.