Understanding data in data sets

• US 9,092,411 B2
• Filed: 08/18/2009
• Issued: 07/28/2015
• Est. Priority Date: 08/18/2009
• Status: Active Grant

• Current Assignee: MIOsoft Corporation
• Original Assignee: MIOsoft Corporation
• Inventors: Barabas, Albert B., Van Gulik, Mark D. A., Raymond, Terry
• Primary Examiner(s): Alam, Hosain
• Assistant Examiner(s): HARPER, ELIYAH STONE
• Application Number:
  US12/542,969
• Publication Number:
  US 20110047186A1
• Time in Patent Office: 2,170 Days
• Field of Search: 707/609, 707/821, 707/999.102, 707/999.104, 707/758
• US Class Current: 1/1
• CPC Class Codes: G06F 16/16 File or folder operations, e.g. details of user interfaces specifically adapted to file systems
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06F 16/289 Object oriented databases
• CPC Class Codes: G06F 40/177 of tables; using ruled lines
• CPC Class Codes: G06F 40/18 of spreadsheets form-filling G06F40/174

Abstract

Among other things, there are two or more data sets. Each of the data sets contains data that can be interpreted as records each having data values for data fields. Each of the data sets contains at least some data that is related to data in at least one of the other data sets. The data in different data sets is organized or expressed possibly differently. Each of the data sets is susceptible to a definition of a key for the records of the data set. The data sets are characterized by repetitions of at least one of (a) records, (b) portions of keys, or (c) instances of values for data fields. Information about at least one of the repetitions is provided to a user.

Claims

1. A computer-implemented method comprising:

receiving two or more data sets,

each of the data sets containing data that can be interpreted as records, the records of each of the data sets each having data values for data fields of the records, the records of each of the data sets having at least one data field that is different from all data fields of the records of another one of the data sets, each data field being identified by a data field identifier,

at least one of the data fields of the records of each of the data sets being related to at least one of the data fields of the records of at least one of the other data sets,

the data of different ones of the data sets being organized or expressed possibly differently,

determining a key for each of the data sets based on one or more data field identifiers of data fields of the data set, the keys for different data sets being different, the data sets being characterized by repetitions of at least one of (a) records, (b) portions of keys, or (c) instances of values for data fields, the keys of the two or more data sets containing information about at least one of the repetitions, and

providing the information about at least one of the repetitions based on the keys of the data sets.

2. The method of claim 1 in which at least one of the data sets comprises a file having a file format.

3. The method of claim 1 in which at least two of the data sets comprise files having different file formats.

4. The method of claim 1 also including receiving from a user information about the manner in which data of at least one of the data sets may be interpreted as records each having data values for data fields.

5. The method of claim 1 in which the key for the records of one of the data sets has a defined hierarchical relationship with the key for the records of another of the data sets.

6. The method of claim 1 in which the repetitions of records comprise duplicate records in one of the data sets.

7. The method of claim 1 in which the repetitions of portions of keys comprise values of a key of one of the data sets corresponding to two different values of a portion of a key of another one of the data sets.

8. The method of claim 1 in which the repetitions of instances of values for data fields comprise two or more instances of values being included in a given field.

9. The method of claim 1 also including enabling a user to perform at least one of marking, unmarking, filtering, unfiltering, and frequency analysis for values of records of at least one of the data sets.

10. The method of claim 1 in which the providing of the information to the user comprises displaying the information.

11. The method of claim 10 in which the displaying comprises displaying records of the data sets, the data field identifiers of the fields of the records, and indications of the repetitions in data sets.

12. The method of claim 10 in which the displaying of information about the repetitions includes displaying repeated instances of values for data fields.

13. The method of claim 10 in which the displaying of information about the repetitions includes noting the existence of repeated records in a data set.

14. The method of claim 10 in which the displaying of information about the repetitions includes noting the existence of repetitions of a portion of a key.

15. The method of claim 1 comprising enabling the user to create an integrated file of records that includes the data of the data sets and information about the repetitions.

16. The method of claim 15 in which the integrated file contains records governed by a key.

17. The method of claim 16 in which the key of the integrated file comprises a hierarchical concatenation of data fields of the data sets.

18. The method of claim 15 in which repetitive data values are included in a given data field of a record of the integrated file.

19. The method of claim 15 also including displaying records of the integrated file to the user.

20. The method of claim 15 also including displaying views of the data of the integrated file that correspond to the data of the data sets from which the integrated file was created.

21. The method of claim 15 also enabling a user to perform at least one of marking, unmarking, filtering, unfiltering, and frequency analysis for values of records of at least one of the data sets.

22. The method of claim 15 also including displaying views of the data of the integrated file that correspond to the data of the data sets from which the integrated file was created, and enabling a user to perform at least one of marking, unmarking, filtering, unfiltering, and frequency analysis for values of records of the integrated file, the marking, unmarking, filtering, and frequency analysis being applied automatically to the other views of the data.

23. A computer-implemented method comprising:

receiving a data set containing data that can be interpreted as records, the records of the data set each having data values for data fields of the records, the data set being characterized by any arbitrary number of repetitions of instances of values for at least one of the data fields, each data field being identified by a data field identifier,

determining a key for the data set based on one or more data field identifiers of data fields of the data set, the key being different from any other key determined for any other data set and containing information about at least one of the repetitions, and

providing the information about at least one of the repetitions based on the key.

24. The method of claim 23 in which the data set comprises a file having a file format.

25. The method of claim 23 also including receiving from a user information about the manner in which data of the data set may be interpreted as records each having data values for data fields.

26. The method of claim 23 also including parsing the data of the data set based on the data fields.

27. The method of claim 23 also including enabling a user to perform marking, unmarking, filtering, unfiltering, and frequency analysis for values of records of the data set.

28. The method of claim 23 in which the providing of the information about at least one of the repetitions comprises displaying the information.

29. The method of claim 28 in which the displaying comprises displaying records of the data set, identifications of the fields of the records, and indications of the repetitions in data set.

30. The method of claim 28 in which the displaying of information about the repetitions includes displaying repeated instances of values for data fields.

31. A non-transitory medium bearing

an integrated file of data records acquired from a transitory source or a non-transitory source and applied to particular hardware to realize the functionality of the data records, the integrated file being formed by integrating at least two data sets,

each of the data sets containing data that can be interpreted as records, the records of each of the data sets each having data values for data fields, the records of each of the data sets having at least one data field that is different from all data fields of the records of another one of the data sets, each data field being identified by a data field identifier,

at least one of the data fields of the records of each of the data sets being related to at least one of the data fields of the records of at least one of the other data sets,

the data of different ones of the data sets being organized or expressed possibly differently,

each of the data sets being susceptible to a key determined based on one or more data field identifiers of data fields of the data set, the keys of different data sets being different the data sets being characterized by repetitions of at least one of (a) records, (b) portions of keys, or (c) instances of values for data fields, the keys containing information about at least one of the repetitions,

a file key for the integrated file of data records, the file key being formed based on the different keys of the two or more data sets.

32. A computer-implemented method comprising

receiving two or more data files,

each of the data files containing data that can be interpreted as records, the records of each of the data files each having data values for data fields of the records, the records of each of the data files having at least one data field that is different from all data fields of the records of another one of the data sets, each data field being identified by a data field identifier,

at least one of the data fields of the records of each of the data files being related to at least one of the data fields of the records of at least one of the other data files,

the data of at least two of the data files being expressed according to two different file formats,

determining a key for each of the data files based on one or more data field identifiers of data fields of the data files, the keys of different data files being different, the data files being characterized by repetitions of at least one of (a) records, (b) portions of keys, or (c) instances of values for data fields, the keys for the records of the two or more data files containing information about at least one of the repetitions

displaying to the user records of the data files, identifications of the fields of the records, and indications of the repetitions in data files including repeated instances of values for data fields, and

enabling the user to create an integrated file of records that includes the data of the data files and information about the repetitions.