Data import system for data analysis system

• US 6,718,336 B1
• Filed: 09/29/2000
• Issued: 04/06/2004
• Est. Priority Date: 09/29/2000
• Status: Active Grant

• Current Assignee: Battelle Memorial Institute
• Original Assignee: Battelle Memorial Institute
• Inventors: Crow, Vernon L., Chen, Guang, McCall, Jonathon D., Saffer, Jeffrey D., Thurston, Sarah J., Zabriskie, Sean J., Miller, Nancy E., Payne, Deborah A., Scarberry, Randall E., Stillwell, Lisa C., Nowell, Lucille T., Monroe, Philip J., Williams, Leigh K., Calaprist, Augustin J.
• Primary Examiner(s): Metjahic, Safet
• Assistant Examiner(s): Alaubaidi, Haythim J.
• Application Number:
  US09/672,622
• Time in Patent Office: 1,285 Days
• Field of Search: 707/3, 707/4, 707/100, 707/102
• US Class Current: 1/1
• CPC Class Codes: G06F 16/25 Integrating or interfacing systems involving database management systems
• CPC Class Codes: Y10S 707/99943 Generating database or data structure, e.g. via user interface

Abstract

A data import system enables access to data of multiple types from multiple data sources of different formats and provides an interface for importing data into a data analysis system. The interface enables a user to customize the formatting of the data as the data is being imported into a data analysis system. A user may select first user defined options for operating on a first data set received during a data importation process. An intermediate representation of the data set is generated based on the user first defined options. A user may specify second user defined options based on the intermediate representation during the data importation process. The second user defined options are processed to produce a final data representation of the data set to be used for analysis of the data. The intermediate representation may be a data table. The processing of a data set may include merging a first and second data set to produce the final data representation. The second user defined options may enable a user to select a basic operation for merging the data sets or to select a non-basic operation for merging the data sets. The basic operation may combine data sets in response to a user'"'"'s selection of a first graphical interface control, and the non-basic operation may combine the data sets based on user selection of at least two graphical interface controls from a group of graphical interface controls.

Claims

1. A method of importing data into a data analysis system, comprising the steps of:

enabling a user to specify first user defined options for operating on a first data set;

generating an intermediate representation of the first data set based on said user first defined options;

enabling a user to specify second user defined options for operating on a second data set;

generating an intermediate representation of the second data set based on said user second defined options;

enabling a user to select a basic operation for combining the data sets or to select a non-basic operation for combining the data sets;

said basic operation being operative to combine said data sets upon user selection of a first graphical interface control, and

said non-basic operation being operative to combine said data sets based on user selection of at least two graphical interface controls from a group of graphical interface controls; and

processing said basic or non-basic selected operation to merge said data sets and produce a final data representation to be used for analysis of said data.

2. The method of claim 1 wherein said data sets comprise data of different types.

3. The method of claim 1 wherein said intermediate representation is a data table.

4. The method of claim 3 wherein said user defined options enable a user to select particular rows and columns of the data set for inclusion in the final data representation.

5. The method of claim 1 wherein said final data representation is a data table.

6. The method of claim 1 wherein said first and second user defined options include an option to extract relevant data from the first and second data sets respectively.

7. The method of claim 1 wherein said first and second user defined options include an option to standardize the format of data from the first and second data sets respectively.

8. A method of creating a data set for use in a data analysis system, comprising the steps of:

enabling a user to access first and second data sets for data analysis;

enabling the user to select a basic operation for combining the data sets or to select a non-basic operation for combining the data sets,

said basic operation being operative to combine said data sets upon user selection of a first graphical interface control, and

said non-basic operation being operative to combine said data sets based on user selection of at least two graphical interface controls from a group of graphical interface controls.

9. The method of claim 8 wherein said basic operation combines said data sets by concatenating said data sets.

10. The method of claim 9 wherein said basic operation may concatenate said data sets in a head-to-tail manner or in a side-by-side manner.

11. The method of claim 8 wherein said non-basic operation is defined by operation on a first characteristic of said data sets and by operation on a second characteristic of said data sets.

12. The method of claim 11 further comprising providing a plurality of first characteristic graphical interface controls for specifying the manner of operation on said first characteristic when said data sets are combined and a plurality of second characteristic graphical interface controls for specifying the manner of operation on said second characteristic when said data sets are combined.

13. The method of claim 12 wherein said data sets are combined to form a data table and said first characteristic corresponds to a row of said data table and said second characteristic corresponds to a column of said data table.

14. The method of claim 13 wherein said first and second graphical interface controls include at least an intersection control to invoke a mathematical intersection operation.

15. The method of claim 13 wherein said first and second graphical interface controls include at least a union control to invoke a mathematical union operation.

16. The method of claim 15 wherein said first and second graphical interface controls include at least a precedence control to signify that data of a selected data set is to be used for the combined data table when values representing corresponding elements are located in both the first and second data sets.

17. The method of claim 8, further comprising the steps of:

accessing at least one data set to be imported into the data analysis system;

determining the format of the data set;

identifying each location of the data set that has a value missing; and

providing a same indicator for each missing value of each data set to be imported into the system.

18. The method of claim 17 comprising the step of providing multiple options for a user to specify a manner in which to identify a value that is missing from a location of the data set.

19. The method of claim 8, further comprising the steps of:

accessing at least one data set to be imported into the data analysis system;

determining the format of the data set;

identifying each location of the data set that has a value missing; and

providing an indicator that a value is missing at the identified locations, wherein the same indicator is provided for each missing value of each data set to be imported into the system.

20. The method of claim 19 comprising a step of providing multiple options for a user to specify a manner in which to identify a value that is missing from a location of the data set.

21. A method of creating a data set for use in a data analysis system, comprising the steps of:

accessing first and second data sets for data analysis;

enabling a user to select a basic operation for combining the data sets or to select a non-basic operation for combining the data sets,

said basic operation being operative to combine said data sets upon user selection of a first graphical interface control, said basic operation combining by concatenating said data sets in a head-to-tail manner or in a side-by-side manner, and

said non-basic operation being operative to combine said data sets based on user selection of at least two graphical interface controls from a group of graphical interface controls.

22. The method of claim 21 wherein said non-basic operation is defined by operation on a first characteristic of said data sets and by operation on a second characteristic of said data sets.

23. The method of claim 22 wherein said data sets are combined to form a data table and said first characteristic corresponds to a row of said data table and said second characteristic corresponds to a column of said data table.

24. The method of claim 23 further comprising providing a plurality of first characteristic graphical interface controls for specifying the manner of operation on said first characteristic when said data sets are combined and a plurality of second characteristic graphical interface controls for specifying the manner of operation on said second characteristic when said data sets are combined.

25. The method of claim 24 wherein said first and second graphical interface controls include at least an intersection control to invoke a mathematical intersection operation.

26. The method of claim 24 wherein said first and second graphical interface controls include at least a union control to invoke a mathematical union operation.

27. The method of claim 24 wherein said first and second graphical interface controls include at least a precedence control to signify that data of a selected data set is to be used for the combined data table when values representing corresponding elements are located in both the first and second data sets.

28. A method of creating a data set for use in a data analysis system, comprising the steps of:

accessing first and second data sets for data analysis;

enabling a user to select a basic operation for combining the data sets or to select a non-basic operation for combining the data sets,

said basic operation being operative to combine said data sets upon user selection of a first graphical interface control, and

said non-basic operation being operative to combine said data sets based on user selection of at least two graphical interface controls from a group of graphical interface controls,

said non-basic operation being defined by operation on a first characteristic of said data sets and by operation on a second characteristic of said data sets, and wherein said group of graphical interface controls comprise a plurality of first characteristic graphical interface controls for specifying the manner of operation on said first characteristic when said data sets are combined and a plurality of second characteristic graphical interface controls for specifying the manner of operation on said second characteristic when said data sets are combined.

29. The method of claim 28 wherein said data sets are combined to form a data table and said first characteristic corresponds to a row of said data table and said second characteristic corresponds to a column of said data table.

30. The method of claim 29 wherein said first and second graphical interface controls include at least an intersection control to invoke a mathematical intersection operation.

31. The method of claim 29 wherein said first and second graphical interface controls include at least a union control to invoke a mathematical union operation.

32. The method of claim 29 wherein said first and second graphical interface controls include at least a precedence control to signify that data of a selected data set is to be used for the combined data table when values representing corresponding elements are located in both the first and second data sets.