Automatic file structure and field data type detection

• US 9,268,801 B2
• Filed: 03/11/2013
• Issued: 02/23/2016
• Est. Priority Date: 03/11/2013
• Status: Active Grant

• Current Assignee: Business Objects Incorporated (SAP SE)
• Original Assignee: Business Objects Incorporated (SAP SE)
• Inventors: Mazoue, Didier
• Primary Examiner(s): TRAN, ANHTAI V
• Application Number:
  US13/792,690
• Publication Number:
  US 20140258341A1
• Time in Patent Office: 1,079 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 16/211 Schema design and management

Abstract

Methods, systems, and computer-readable storage media for determining data structure and field types of a data source that is to be processed by an application. Actions include receiving the data source, providing base data associated with the data source, determining a number of fields of the data source and, for each field, a field type based on the data source and the base data, generating data structure data, the data structure data comprising the number of fields and field types, and providing the data structure data to the application.

Claims

1. A computer-implemented method for determining data structure and field types of a data source that is to be processed by an application, the method being executed using one or more processors and comprising:

receiving the data source;

providing base data associated with the data source;

determining, by the one or more processors, a number of fields of the data source and, for each field, a field type based on the data source and the base data, wherein determining the number of fields of the data source comprises;

selecting a field separator from a plurality of field separators;

for each sample row in a set of sample rows, determining an estimated number of fields based on the field separator; and

iterating over all separators in the plurality of field separators and determining that, for each field separator, the estimated number of fields for each sample row is unequal across two or more sample rows in the set of sample rows, and in response, setting the number of fields equal to one;

generating, by the one or more processors, data structure data, the data structure data comprising the number of fields and field types; and

providing, by the one or more processors, the data structure data to the application.

2. The method of claim 1, wherein determining the number of fields of the data source further comprises determining that the estimated number of fields for each sample row is equal across all sample rows in the set of sample rows, the number of fields being equal to the estimated number of fields.

3. The method of claim 1, wherein determining the number of fields of the data source further comprises:

determining that the estimated number of fields for each sample row is unequal across two or more sample rows in the set of sample rows, and in response;

selecting a second field separator from the plurality of field separators;

for each sample row in the set of sample rows, determining an estimated number of fields based on the second field separator; and

determining that the estimated number of fields for each sample row is equal across all sample rows in the set of sample rows, the number of fields being equal to the estimated number of fields.

4. The method of claim 1, wherein the set of sample rows comprises first n rows of data within the data source, where n is an integer that is greater than or equal to two.

5. The method of claim 1, wherein determining, for each field, a field type comprises determining that data provided in the field corresponds to a date and time format of a plurality of date and time formats, and in response setting the field type to date and time.

6. The method of claim 1, wherein determining, for each field, a field type comprises determining that data provided in the field corresponds to a numeric format of a plurality of numeric formats, and in response setting the field type to numeric.

7. The method of claim 6, wherein determining that data provided in the field corresponds to a numeric format of a plurality of numeric formats is performed in response to determining that the data provided in the field does not correspond to any date and time formats of a plurality of date and time formats.

8. The method of claim 1, further comprising determining that a field includes a string field type.

9. The method of claim 8, wherein determining that a field includes a string field type is performed in response to determining that the data provided in the field does not correspond to any date and time formats of a plurality of date and time formats and to determining that the data provided in the field does not correspond to any numeric formats of a plurality of numeric formats.

10. The method of claim 1, wherein the base data comprises a type of the data source.

11. The method of claim 10, wherein the data source comprises a computer-readable file and the base data comprises a suffix of the computer-readable file.

12. The method of claim 1, wherein the base data comprises regional settings.

13. The method of claim 12, where the regional settings comprise one or more of a regional language, numeric formats, and date and time formats.

14. A non-transitory computer-readable storage medium coupled to one or more processors and having instructions stored thereon which, when executed by the one or more processors, cause the one or more processors to perform operations for determining data structure and field types of a data source that is to be processed by an application, the operations comprising:

receiving the data source;

providing base data associated with the data source;

determining a number of fields of the data source and, for each field, a field type based on the data source and the base data, wherein determining the number of fields of the data source comprises;

selecting a field separator from a plurality of field separators;

for each sample row in a set of sample rows, determining an estimated number of fields based on the field separator; and

iterating over all separators in the plurality of field separators and determining that, for each field separator, the estimated number of fields for each sample row is unequal across two or more sample rows in the set of sample rows, and in response, setting the number of fields equal to one;

generating data structure data, the data structure data comprising the number of fields and field types; and

providing the data structure data to the application.

15. A system, comprising:

a computing device including at least one processor; and

a non-transitory computer-readable storage device coupled to the at least one processor and having instructions stored thereon which, when executed by the at least one processor, cause the at least one processor to perform operations for determining data structure and field types of a data source that is to be processed by an application, the operations comprising;

receiving the data source;

providing base data associated with the data source;

determining a number of fields of the data source and, for each field, a field type based on the data source and the base data, wherein determining the number of fields of the data source comprises;

selecting a field separator from a plurality of field separators;

for each sample row in a set of sample rows, determining an estimated number of fields based on the field separator; and

iterating over all separators in the plurality of field separators and determining that, for each field separator, the estimated number of fields for each sample row is unequal across two or more sample rows in the set of sample rows, and in response, setting the number of fields equal to one;

generating data structure data, the data structure data comprising the number of fields and field types; and

providing the data structure data to the application.