Systems and methods for importing data from electronic data files

• US 9,514,205 B1
• Filed: 04/07/2016
• Issued: 12/06/2016
• Est. Priority Date: 09/04/2015
• Status: Active Grant

• Current Assignee: Palantir Technologies Incorporated
• Original Assignee: Palantir Technologies Incorporated
• Inventors: Yazicioglu, Stephen, Luck, Christopher, Giardina, Robert, Streufert, Justin, O'Conner, Gregory, Marc-Aurele, Brandon, Zhu, Olivia, Schindel, Howard, Tung, Henry, Ray, Lucas, Leech, Christopher, Jeney, Eric, Negrus, Stefan, Lee, Jason, Doyle, John McKinstry, Pitelka, Hunter, Lozano, Ethan, Ossher, Joel, Fedderly, Matthew, Slatcher, Timothy, Mingione, Alessandro
• Primary Examiner(s): Le, Hung
• Application Number:
  US15/092,814
• Time in Patent Office: 243 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/162 Delete operations erasing in storage systems G06F3/0652
• CPC Class Codes: G06F 16/22 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/235 Update request formulation
• CPC Class Codes: G06F 16/245 Query processing
• CPC Class Codes: G06F 16/254 Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database
• CPC Class Codes: G06F 16/86 Mapping to a database
• CPC Class Codes: G06F 40/186 Templates

Abstract

Computer implemented systems and methods are disclosed for importing data from electronic data files. In accordance with some embodiments, source electronic data files are received at a data importation system and managed by the data importation system. The data importation system may apply detector/transformer plugins to the received source electronic data files to transform the files for importation into one or more data analysis systems and/or databases. The data importation system may also receive user inputs for mapping source electronic data files to transformation templates. The inputs may include, for example, an assignment of a file format to the source electronic data file, identification of a file type identifier associated with the source electronic data file, and a mapping of a the source electronic data file to a transformation template. The data importation system may store the received inputs as a file type profile in a database.

Claims

1. A system for importing data from electronic data files, the system comprising:

a memory device that stores a set of instructions;

at least one processor that executes the instructions to;

receive a source electronic data file;

generate instructions for displaying a visual indicator for indicating that a file type associated with the source electronic data file could not be detected;

receive an assignment of a file format to the source electronic data file;

receive identification of a file type identifier associated with the source electronic data file;

receive a mapping of the source electronic data file to a transformation template; and

store the file format, file type identifier, and mapping as a file type profile associated with the source electronic data file in a database.

2. The system of claim 1, wherein the file type identifier associated with the source electronic data file includes at least one of a structure of the source electronic data file, a combination of file headers included in the source electronic data file, and an order of file headers included in the source electronic data file.

3. The system of claim 1, wherein the file type identifier associated with the source electronic data file includes at least one of a string of text included in the source electronic data file and a type of data included in the source electronic data file.

4. The system of claim 1, wherein the mapping of the source electronic data file to a transformation template includes mappings to conform a format of data fields included in the source electronic data file to a canonical format of data fields included in the transformation template.

5. The system of claim 1, wherein the mapping of the source electronic data file to a transformation template includes mappings to conform data included in the source electronic data file to a canonical format of data included in the transformation template.

6. A method of importing data from electronic data files, the method performed by at least one processor and comprising:

receiving a source electronic data file;

generating instructions for displaying a visual indicator for indicating that a file type associated with the electronic data file could not be detected;

receiving an assignment of a file format to the electronic data file;

receiving identification of a file type identifier associated with the source electronic data file;

receiving a mapping of the source electronic data file to a transformation template; and

storing the file format, file type identifier, and mapping as a file type profile associated with the source electronic data file in a database.

7. The method of claim 6, wherein the file type identifier associated with the source electronic data file includes at least one of a structure of the source electronic data file, a combination of file headers included in the source electronic data file, and an order of file headers included in the source electronic data file.

8. The method of claim 6, wherein the file type identifier associated with the source electronic data file includes at least one of a string of text included in the source electronic data file and a type of data included in the source electronic data file.

9. The method of claim 6, wherein the mapping of the source electronic data file to a transformation template includes mappings to conform a format of data fields included in the source electronic data file to a canonical format of data fields included in the transformation template.

10. The method of claim 6, wherein the mapping of the source electronic data file to a transformation template includes mappings to conform data included in the source electronic data file to a canonical format of data included in the transformation template.