Computer-implemented system and method for tagged and rectangular data processing

• US 20030200501A1
• Filed: 04/19/2002
• Published: 10/23/2003
• Est. Priority Date: 04/19/2002
• Status: Active Grant

• Current Assignee: SAS Institute Incorporated
• Original Assignee: SAS Institute Incorporated
• Inventors: Cox, Thomas Warren, Friebel, Anthony L.
• Granted Patent: US 7,921,359 B2
• Time in Patent Office: Days
• US Class Current: 715/500
• CPC Class Codes: G06F 16/258 Data format conversion from or to a database

Abstract

A computer-implemented system and method for tagged data and rectangular data conversions. The system and method receive tagged input data that is in a non-rectangular format and uses a hierarchical arrangement of tags to indicate data relationships. Data directives are used to parse the tagged input data. The data directives indicate which tags in the hierarchical arrangement of tags are to form which columns in the rectangular formatted data output. The rectangular formatted data output is generated based upon the parsed tagged input data. The data directives may also be used to convert rectangular data into a tagged data format.

Claims

1. A computer-implemented data structure conversion method, comprising the steps of:

receiving tagged input data that is in a non-rectangular format and that uses a hierarchical arrangement of tags to indicate data relationships;

using data directives to parse the tagged input data,

said data directives indicating which tags in the hierarchical arrangement tags are to form which columns in a rectangular formatted data output; and

generating rectangular data output based upon the parsed tagged input data.

2. The method of claim 1 wherein the tagged input data is in an eXtensible Markup Language (XML) format.

3. The method of claim 2 wherein the tagged input data includes pcdata fields within the tags of the tagged input data, wherein the tagged input data includes attributes within the tags of the tagged input data.

4. The method of claim 1 wherein the data directives are expressed in a non-rectangular format and uses a hierarchical arrangement of tags to indicate relationships among conversion operations and attributes of the data directives.

5. The method of claim 4 wherein said relationships provide information about how the conversion operations and the attributes of the data directives are to be used in parsing the tagged input data.

6. The method of claim 1 wherein the data directives include row repetition boundaries.

7. The method of claim 6 wherein at least a portion of the tags within the tagged input data contain an arbitrary number of first data items, wherein the row repetition boundaries are defined such that a new observation is created in the rectangular formatted data output when a new first data item is encountered during the parsing of the tagged input data.

8. The method of claim 1 wherein the data directives include definitions for the rectangular formatted data output.

9. The method of claim 8 wherein the definitions include output column length.

10. The method of claim 8 wherein the definitions include output column data type.

11. The method of claim 8 wherein the definitions include output column type.

12. The method of claim 8 wherein the definitions include acceptable values for a column in the rectangular formatted data output.

13. The method of claim 8 wherein the definitions include an indication that a value for a data item within the tagged input data is to be retained and used in the rectangular data output until a new value for the data item is encountered in the tagged input data.

14. The method of claim 8 wherein the definitions include a selection criterion for filtering what data items contained in the tagged input data comprise content in the rectangular formatted data output.

15. The method of claim 8 wherein the definitions include a generation of a column'"'"'s content based upon at least two data items contained in the tagged input data.

16. The method of claim 8 wherein the definitions include a calculation of an output column'"'"'s content based upon at least two data items contained in the tagged input data.

17. The method of claim 8 wherein the definitions include adding a sequential number to enumerate rows created for the rectangular formatted data output.

18. The method of claim 8 wherein the definitions include specifying a default value to be used when a value for a data item is missing.

19. The method of claim 8 wherein the definitions include string function means.

20. The method of claim 8 wherein the definitions include axes function means.

21. The method of claim 8 wherein the definitions include node-related function means.

22. The method of claim 1 wherein at least a portion of the data directives are expressed in an Xpath locator format to locate data items within the tagged input data.

23. The method of claim 22 further comprising the step of:

using an XML parser to parse the tagged input data based upon the data directives.

24. The method of claim 1 wherein an XML parser is used to parse the tagged input data based upon the data directives.

25. The method of claim 1 wherein the rectangular data output is for use as a table within a database.

26. The method of claim 1 wherein the rectangular data output is for use as a table within a relational database management system.

27. The method of claim 1 wherein a graphical user interface provides a visual analysis of the tags'"'"' hierarchical arrangement in the tagged input data so that a user may construct at least one data directive for parsing the tagged input data.

28. The method of claim 27 wherein an XML parser is used in providing the visual analysis of the tags'"'"' hierarchical arrangement.

29. The method of claim 1 further comprising the steps of:

using second data directives to parse a rectangular formatted data,

said second data directives indicating which columns in the rectangular formatted data are to occupy which positions in a tagged hierarchical arrangement and with what tags;

generating tagged data based upon the parsed rectangular formatted data.

30. A computer-implemented data structure conversion method, comprising the steps of:

receiving rectangular formatted input data that contains columns and rows of data;

using data directives to parse the rectangular formatted input data,

said data directives indicating which columns in the rectangular formatted input data are to occupy which positions in a tagged hierarchical arrangement and with what tags; and

generating tagged data based upon the parsed rectangular formatted data,

said generated tagged data having, in accordance with the data directive, tags arranged in a hierarchy to indicate data relationships.

31. A computer-human interface for processing tagged input data, said tagged input data being in a non-rectangular format and using a hierarchical arrangement of tags to indicate data relationships, said graphical user interface comprising:

a first interface region for displaying the tagged input data to a user;

a second interface region that allows the user to specify a desired rectangular format, said rectangular format including information about columns to be populated with data from the tagged input data; and

an interface mechanism that allows the user to specify which data within the tagged input data shown in the first interface region is to be a source of data for at least one of the columns specified in the second interface region;

wherein data directives are to be generated for processing the tagged input data based upon information specified by the user through the interface mechanism, said data directives indicating which tags in the hierarchical arrangement of tags are associated with which of the columns.

32. The interface of claim 31 wherein the first interface region displays the tagged input data to the user by providing a visual analysis of the tags'"'"' hierarchical arrangement in the tagged input data.

33. The interface of claim 32 wherein the hierarchical arrangement is shown in a full view.

34. The interface of claim 32 wherein the hierarchical arrangement is shown in a condensed view.

35. The interface of claim 31 wherein the first interface region displays at least a portion of the tagged input data.

36. The interface of claim 35 further comprising a hierarchical view interface region that displays a hierarchical arrangement of the tagged input data.

37. The interface of claim 36 wherein the hierarchical arrangement is shown in a full view.

38. The interface of claim 36 wherein the hierarchical arrangement is shown in a condensed view.

39. The interface of claim 31 wherein the tagged input data is in an extensible Markup Language (XML) format.

40. The interface of claim 39 wherein an XML parser parses the tagged input data in order to create a hierarchical view interface region that displays a hierarchical arrangement of the tagged input data.

41. The interface of claim 31 further comprising a mapping specification interface region to display the generated data directives.

42. The interface of claim 41 wherein the generated data directives are in a tagged hierarchical format.

43. The interface of claim 41 wherein the generated data directives are in an extensible Markup Language (XML) format.

44. The interface of claim 31 further comprising a code generation interface region to display at least one instruction that utilizes data generated based upon the data directives and the tagged input data.

45. The interface of claim 44 wherein the code generation interface region comprises structured query language (SQL) instructions to create at least one table based upon the generated data directives.

46. The interface of claim 45 wherein the code generation interface region comprises structured query language (SQL) instructions to populate at least one table based upon the generated data directives.

47. The interface of claim 44 wherein the code generation interface region comprises instructions related to operation of a statistical analysis program, said statistical analysis program analyzing the data generated based upon the data directives and the tagged input data.

48. The interface of claim 31 wherein the interface mechanism comprises properties specification means for specifying at least one property characteristic associated with at least one of the columns.

49. The interface of claim 31 wherein the interface mechanism comprises format specification means for specifying at least one format characteristic associated with at least one of the columns.

50. The interface of claim 31 wherein the interface mechanism comprises processing specification means for specifying how data from the tagged input data is to be processed in order to populate the columns.

51. The interface of claim 31 wherein the interface mechanism comprises ordinal specification means for specifying how a column is to be populated with increment values.

52. The interface of claim 31 wherein the interface mechanism comprises validation specification means for specifying how validation is to be performed with respect to population of the columns with the tagged input data.

53. A computer-human interface for processing rectangular formatted input data, said rectangular formatted input data containing columns and rows of data, said graphical user interface comprising:

a first interface region for displaying the rectangular formatted input data to a user;

a second interface region that allows the user to specify a desired tagged data format structure, said tagged data format structure being in a non-rectangular format and using a hierarchical arrangement of data item tags to indicate data relationships; and

an interface mechanism to allow the user to specify which data within the rectangular formatted input data shown in the first interface region is to populate which data item tags specified in the second interface region;

wherein data directives are generated for parsing the rectangular formatted input data based upon information specified by the user through the interface mechanism, said data directives indicating which columns in the rectangular formatted input data are to be associated with which tags in the tagged hierarchical arrangement.