METHOD AND SYSTEM FOR MAINTAINING DATA IN A DATA STORAGE SYSTEM

• US 20140114993A1
• Filed: 10/22/2012
• Published: 04/24/2014
• Est. Priority Date: 10/22/2012
• Status: Abandoned Application

• Current Assignee: Excalibur IP, LLC (Acacia Research Corporation)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Watfa, Allie K., Liu, Bo, Luo, Wuheng
• Application Number:
  US13/657,143
• Publication Number:
  US 20140114993A1
• Time in Patent Office: Days
• US Class Current: 707/756
• CPC Class Codes: G06F 16/86 Mapping to a database

Abstract

Method, system, and programs for generating, storing, and maintaining data in a data storage system. A data record in a first format is received, and converted into one or more converted data records in a second format. Each of the one or more converted data records comprises a markup attribute, a content attribute, and an identifier attribute used to locate the data record in the first format. And the one or more converted data records are stored in the data storage system.

Claims

1. A method, implemented on at least one machine having at least one processor, storage, and a communication platform connected to a network for maintaining data in a data storage system, comprising the steps of:

receiving a data file including one or more elements;

converting each element of the data file to one or more records, wherein each record has one or more types of data;

assigning each record to a row of a table in the data storage system, wherein;

the table has a plurality of rows and a plurality of columns comprising at least a markup column, a content column, and a uniform resource identifier (URI) column, and

all data assigned to a same column belong to a same type; and

maintaining data in the table.

2. The method of claim 1, wherein:

each record has at least three types of data tag, value, and position; and

the step of assigning each record further comprises;

assigning tag of the record to the markup column,

assigning value of the record to the content column, and

assigning position of the record to the URI column.

3. The method of claim 1, wherein the table further comprises a virtual column identification (ID) used to query data by identifying a collection of records.

4. The method of claim 1, wherein the step of maintaining data in the table further comprises querying data, with a single query, from heterogeneous records in the table satisfying same criteria.

5. The method of claim 1, wherein the step of maintaining data in the table further comprises:

adding one record to the table by inserting one row to the table without changing quantity of columns in the table; and

removing one record from the table by deleting one row from the table without changing quantity of columns in the table.

6. The method of claim 1, wherein the step of maintaining data in the table further comprises retrieving, with a single query, one or more records in the table satisfying same criteria based on their positions in the URI column.

7. The method of claim 1, further comprising the steps of:

establishing the table in the data storage system with a plurality of rows and a plurality of columns, wherein quantity of the columns is fixed; and

storing the table in a storage unit.

8. The method of claim 1, wherein:

the data storage system comprises a distributed data warehouse based on Hadoop or Hive; and

the data file has an Extensible Markup Language (XML) format before being converted.

9. A system for maintaining data in as data storage system, comprising:

a receiver configured to receive a data file including one or more elements;

a converting unit coupled to the receiver and configured to convert each element of the data file to one or more records, wherein each record has one or more types of data;

a mapping unit coupled to the converting unit and configured to assign each record to a row of a table in the data storage system, wherein;

the table has a plurality of rows and a plurality of columns comprising at least a markup column, a content column, and a URI column, and

all data assigned to a same column belong to a same type; and

a processor configured to maintain data in the table.

10. The system of claim 9, wherein:

each record has at least three types of data;

tag, value, and position; and

for each record, the mapping unit is further configured to;

assign tag of the record to the markup column,

assign value of the record to the content column, and

assign position of the record to the URI column.

11. The system of claim 9, wherein the table further comprises a virtual column ID used to query data by identifying a collection of records.

12. The system of claim 9, wherein the processor further comprises a querying unit configured to query data, with a single query, from heterogeneous records in the table satisfying same criteria.

13. The system of claim 9, wherein the processor further comprises a modifying unit configured to:

add one record to the table by inserting one row to the table without changing quantity of columns in the table; and

remove one record from the table by deleting one row from the table without changing quantity of columns in the table.

14. The system of claim 9, wherein the processor further comprises a retrieving unit configured to retrieve, with a single query, one or more records in the table satisfying same criteria based on their positions in the URI column.

15. The system of claim 9, further comprising:

an establishing unit configured to establish the table in the data storage system with a plurality of rows and a plurality of columns, wherein quantity of the columns is fixed; and

a storage unit coupled to the establishing unit, the mapping unit, and the processor, and configured to store the table.

16. The system of claim 9, wherein:

the data storage system comprises a distributed data warehouse based on Hadoop or Hive; and

the data file has an XML format before being converted.

17. A machine-readable tangible and non-transitory medium having information for maintaining data in a data storage system, wherein the information, when read by the machine, causes the machine to perform the following steps:

receiving a data file including one or more elements;

converting each element of the data file to one or more records, wherein each record has one or more types of data;

assigning each record to a row of a table in the data storage system, wherein;

the table has a plurality of rows and a plurality of columns comprising at least a markup column, a content column, and a URI column, and

all data assigned to a same column belong to a same type; and

maintaining data in the table.

18. The medium of claim 17, wherein:

each record has at least three types of data;

tag, value, and position; and

the step of assigning each record further comprises;

assigning tag of the record to the markup column,

assigning value of the record to the content column, and

assigning position of the record to the URI column.

19. The medium of claim 17, wherein the table further comprises a virtual column ID used to query data by identifying a collection of records.

20. A method, implemented on at least one machine having at least one processor, storage, and a communication platform connected to a network for storing data in a data storage system, comprising the steps of:

receiving a data record in a first format;

converting the data record in the first format into one or more converted data records in a second format, wherein each of the one or more converted data records comprises a markup attribute, a content attribute, and an identifier attribute used to locate the data record in the first format; and

storing the one or more converted data records in the data storage system.

21. A method, implemented on at least one machine having at least one processor, storage, and a communication platform connected to a network for generating data, comprising the steps of:

receiving a piece of information comprising one or more parts;

identifying the one or more parts of the piece of information; and

generating one or more data records, each for a part of the piece of information, wherein each of the one or more data records comprises a markup attribute, a content attribute, and an identifier attribute used to locate the corresponding part in the piece of information.