Data integration tool

• US 9,984,152 B2
• Filed: 12/15/2014
• Issued: 05/29/2018
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: Palantir Technologies Incorporated
• Original Assignee: Palantir Technologies Incorporated
• Inventors: Nassar, Anthony Albert
• Primary Examiner(s): Dao, Thuy
• Application Number:
  US14/571,006
• Publication Number:
  US 20150100559A1
• Time in Patent Office: 1,261 Days
• CPC Class Codes: G06F 11/362 Software debugging
• CPC Class Codes: G06F 11/3688 for test execution, e.g. scheduling of test suites
• CPC Class Codes: G06F 11/3692 for test results analysis
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 16/288 Entity relationship models
• CPC Class Codes: G06F 16/289 Object oriented databases
• CPC Class Codes: G06F 16/367 Ontology

Abstract

Computer-implemented systems and methods are disclosed for providing proactive validations of transformation scripts. In one implementation, a method is provided that includes associating, with at least one processor, the transformation script with ontology parameters. The method also includes initiating a debugging operation of the transformation script having at least one condition and importing, from a data source, at least one data item for transformation. The method further includes determining, as part of the debugging operation, whether the at least one condition that uses the at least one data item is valid based on the ontology parameters and notifying a user, on a display device, of a result associated with the determination.

Claims

1. A method for providing a proactive validation of a transformation script, the method being performed by one or more processors and comprising:

associating the transformation script with ontology parameters, wherein the ontology parameters include parameters that assign an entity as being an object or a property of an object;

initiating a debugging operation of the transformation script having at least one condition, wherein the transformation script uses a builder that defines a definition of the entity as the entity being an object or a property of an object;

importing, from a data source, at least one data item for transformation;

determining, as part of the debugging operation, whether the at least one condition that uses the at least one data item is valid if the assignment of the entity associated with the transformation script is consistent with the definition of the entity, the determination whether the at least one condition that uses the at least one data item is valid being based on the ontology parameters; and

providing an indication of a result associated with the determination, wherein the result is at least one of an expressed result or an implicit result.

2. The method of claim 1, wherein providing the indication of the result associated with the determination comprises:

if the at least one condition is determined to be not valid, providing the indication of the result to a display device for displaying an expressed result indicating that the at least one condition is not valid.

3. The method of claim 1, wherein providing the indication of the result associated with the determination comprises:

if the at least one condition is determined to be valid and the transformation script includes at least one subsequent condition to be debugged, providing an implicit result indicating that the at least one condition is valid; and

if the at least one condition is determined to be valid and if the transformation script does not include any subsequent conditions to be debugged, displaying on the display device an expressed result indicating that the transformation script has been validated.

4. The method of claim 1, wherein providing the indication of the result to the display device for displaying comprises:

displaying the expressed result upon determining that the at least one condition is not valid.

5. The method of claim 1, wherein determining comprises:

determining that the at least one condition is not valid if the assignment of the entity is inconsistent with the definition of the entity.

6. The method of claim 1, further comprising:

associating the transformation script with ontology parameters, the determining whether the at least one condition that uses the at least one data item is valid being based on the ontology parameters;

wherein the ontology parameters assigns a link between two entities; and

further wherein the transformation script uses a builder that creates a link between two entities.

7. The method of claim 6 wherein determining comprises:

determining that the at least one condition is not valid if the assigned link in the ontology parameters is inconsistent with the created link in the builder.

8. The method of claim 1, wherein the data source includes unstructured data or the data source includes structured data.

9. The method of claim 1, wherein the transformation script includes information for mapping at least a portion of the at least one data item to at least one of the ontology parameters.

10. An apparatus for providing a proactive validation of a transformation script, the apparatus comprising:

a memory device that stores a set of instructions;

one or more processors that execute the set of instructions to configure the one or more processors to;

associate the transformation script with ontology parameters, wherein the ontology parameters include parameters that assign an entity as being an object or a property of an object;

initiate a debugging operation of the transformation script having at least one condition, wherein the transformation script uses a builder that defines a definition of the entity as the entity being an object or a property of an object;

import, from a data source, at least one data item for transformation;

determine, as part of the debugging operation, whether the at least one condition that uses the at least one data item is valid if the assignment of the entity associated with the transformation script is consistent with the definition of the entity, the determination whether the at least one condition that uses the at least one data item is valid being based on the ontology parameters; and

provide an indication of a result associated with the determination, wherein the result is at least one of an expressed result or an implicit result.

11. The apparatus of claim 10, wherein the one or more processors determine that at least one condition is not valid if the assignment of the entity is inconsistent with the definition of the entity.

12. The apparatus of claim 10, wherein the data source includes unstructured data or the data source includes structured data.

13. The apparatus of claim 10, wherein the at least one condition is based on the data source.

14. The apparatus of claim 10, wherein the transformation script includes information for mapping at least a portion of the at least one data item to at least one of the ontology parameters.

15. A non-transitory computer readable medium that stores a set of instructions that are executable by at least one processor to cause the at least one processor to perform a method for providing a proactive validation of a transformation script, the method comprising:

associating the transformation script with ontology parameters, wherein the ontology parameters include parameters that assign an entity as being an object or a property of an object;

initiating a debugging operation of the transformation script having at least one condition, wherein the transformation script uses a builder that defines a definition of the entity as the entity being an object or a property of an object;

importing, from a data source, at least one data item for transformation;

determining, as part of the debugging operation, whether the at least one condition that uses the at least one data item is valid if the assignment of the entity associated with the transformation script is consistent with the definition of the entity, the determination whether the at least one condition that uses the at least one data item is valid being based on the ontology parameters; and

providing an indication of a result associated with the determination, wherein the result is at least one of an expressed result or an implicit result.

16. The non-transitory computer readable medium of claim 15, wherein the data source includes unstructured data or the data source includes structured data.

17. The non-transitory computer readable medium of claim 15, wherein the transformation script includes information for mapping at least a portion of the at least one data item to at least one of the ontology parameters.