DATA MANAGEMENT APPARATUS, METHOD AND PROGRAM

• US 20100094803A1
• Filed: 11/30/2007
• Published: 04/15/2010
• Est. Priority Date: 12/08/2006
• Status: Active Grant

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Yamakawa, Satoshi, Kajiki, Yoshihiro
• Granted Patent: US 8,224,772 B2
• Time in Patent Office: Days
• US Class Current: 707/609
• CPC Class Codes: G06F 16/907 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Abstract

Disclosed is a data management apparatus comprising a message transfer/analysis unit that receives an object data access request from a client and returns a response; an object metadata storage unit that stores, as metadata, a set of ancillary information on data for an object and a data area corresponding to the ancillary information; an object metadata management unit that performs reading, updating, and registration processing for the metadata; a data conversion unit that sends the data access request to a data access unit, extracts object data; and a data access unit that sends a data access command, specified by the data conversion unit, to a file server or a database server and sends response data, received from the file server or the database server, to the data conversion unit.

Claims

1. A data management apparatus comprising:

a unit that maps data having a plurality of data formats into objects having a common format for consolidated data access; and

a unit that assigns ancillary information which is used to represent or describe contents of data to an object and saves the ancillary information by relating thereto data part information which indicates to which data area of the object the ancillary information is related.

2. The data management apparatus according to claim 1, further comprising:

a unit that searches for an object, based on the ancillary information; and

a unit that reads a part of object data, based on the ancillary information and the data part information related to the ancillary information.

3. The data management apparatus according to claim 1,

wherein a plurality of pieces of the ancillary information are given to one object.

4. A data management apparatus that manages access to data of an object, comprising:

an object metadata storage unit that stores metadata of the object; and

a unit that stores and manages ancillary information and data part information, in association with an object ID which identifies an object, as the metadata of the object, in the object metadata storage unit separately from a storage unit in which the object data is stored, the ancillary information used to represent or describe contents of data of the object, the data part information indicating to which data area of the object the ancillary information is related.

5. The data management apparatus according to claim 4, wherein the data management apparatus receives a read request from a client, searches the object metadata storage unit, using an object ID and ancillary information specified by the read request, and selectively reads a data part of object data, corresponding to the ancillary information, from the storage in which the object data is stored.

6. The data management apparatus according to claim 4, wherein the object metadata storage unit stores, in association with the object ID, a data format of the object data and storage location address information on the object data.

7. The data management apparatus according to claim 6, further comprising:

a unit that receives a read request from a client, searches the object metadata storage unit, using an object ID and ancillary information specified by the read request, and generates an object data read command corresponding to a data format corresponding to the object ID and data part information related to the ancillary information; and

a unit that sends the object data read command to the storage unit in which the object data is stored, and selectively reading a data part of the object data, corresponding to the ancillary information, from the storage unit in which the object data is stored.

8. The data management apparatus according to claim 7, wherein the data format of the object data includes a file format and a database format, and

the unit that generates an object data read command generates a file read instruction or a database data read instruction according to the data format of the object.

9. The data management apparatus according to claim 6, wherein when an object generation or update request is received from the client, the data management apparatus generates an object generation or update command according to the data format of the object, and sends a file creation or update instruction to a file server that is a storage destination of the object data or sends a table creation or update instruction to a database server that is a storage destination of the object data.

10. The data management apparatus according to claim 6, wherein when an ancillary information addition request is received from the client, the data management apparatus adds the ancillary information and data part information corresponding to the added ancillary information to the object metadata storage unit.

11. The data management apparatus according to claim 6, further comprising

a unit that, when an object list request is received from the client, extracts object IDs, which match ancillary information specified by the request, and displays a list of data formats, ancillary information, and data part information.

12. A data management apparatus that manages access to data of an object, comprising:

a first storage area in which ancillary information and data part information, in association with an object ID which identifies an object, are stored and managed as metadata of the object, the ancillary information used to represent or describe contents of data of the object, the data part information indicating to which data area of the object the ancillary information is related;

a second storage area in which a data format of the object and information on a storage location address where the data of the object is stored, in association with the object ID, are stored and managed;

a unit that receives a read request from a client and, based on an object ID specified by the read request, and that generates an object data read command corresponding to the data format information and the data part information by referencing the first storage area and the second storage area; and

a unit that selectively reads a data part of object data, corresponding to the ancillary information, from a file server or a database server in which the data of the object is stored.

13. A data management apparatus comprising:

a message transfer/analysis unit that receives an object data access request, sent from a client, and returns a response;

an object metadata storage unit that stores a set of ancillary information on data on an object and a data area corresponding to the ancillary information as metadata on the object;

an object metadata management unit that processes metadata on objects and, in response to a request from the message transfer/analysis unit, manages an execution of reading, update, and registration of metadata stored in the object metadata storage unit;

a data access unit that connects to a file server and a database server; and

a data conversion unit that generates a data access command, corresponding to the file server or the database server, in response to a request from the message transfer/analysis unit, sends the data access command to the data access unit, and extracts object data from response data sent from the data access unit for sending the extracted object data to the message transfer/analysis unit, wherein

the data access unit sends the data access command, specified by the data conversion unit, to the file server and the database server and sends response data, received from the file server or the database server, to the data conversion unit.

14. The data management apparatus according to claim 13, wherein the object metadata storage unit includes:

object ID information which uniquely identifies object data;

data format information which indicates a data format of an object;

data storage location address information which indicates address information on an object data storage location;

ancillary information which represents contents of data for determining contents of object data; and

data part information which indicates a position of data corresponding to the ancillary information.

15. The data management apparatus according to claim 13, wherein

(a1)) when an object creation request, for which a data format and ancillary information representing whole object data to be created are specified, is received from the client,

the message transfer/analysis unit extracts the data format information and the ancillary information from the object creation request, and

sends the data format information, the ancillary information, and the object creation request to the object metadata management unit,

(a2) the object metadata management unit generates an object ID based on the ancillary information, determines a data storage location address at which the object is to be stored, establishes a relation among the object ID information, data format information, ancillary information, and data storage location address information and stores the related information in the object metadata storage unit, and sends the object ID and the data storage location address information to the message transfer/analysis unit,

(a3) the message transfer/analysis unit sends an object creation request, for which the object ID, data storage location address information, and data format information are specified, to the data conversion unit,

(a4) the data conversion unit creates an object creation command corresponding to the specified data format and sends the object creation command, corresponding to the data format, to the data access unit,

(a5) the data access unit, based on the object creation command, sends a file creation command to the file server if the object is a file,

sends a table creation command to the database server if the object is a database, and

if a response is returned from the file server or the database server indicating that object creation processing is executed successfully,

notifies to the data conversion unit that the file or table creation request is executed successfully,

(a6) the data conversion unit notifies to the message transfer/analysis unit that the object creation request is executed successfully when the notification is received from the data access unit notifying that the file or table creation request is executed successfully, and

(a7) the message transfer/analysis unit receives the notification, which notifies that the object creation request is executed successfully, from the data conversion unit and notifies the object ID, as well as the notification that the object creation request is executed successfully, to the client.

16. The data management apparatus according to claim 13, wherein

(b1) when an object data update request, for which the object ID, data format, object data, and data update position information are specified, is sent from the client, which receives the notification notifying that the object creation request is executed successfully, to the message transfer/analysis unit, the message transfer/analysis unit extracts the object ID and the object data from the object data update request received from the client and sends an extraction request, which requests to extract data storage location address information and data format information using the object ID as the key, to the object metadata management unit,

(b2) the object metadata management unit extracts data storage location address information and data format information, included in information related to the object ID, from the object metadata storage unit based on the extraction request and sends the extracted information to the message transfer/analysis unit,

(b3) the message transfer/analysis unit sends the data storage location address information and the data format information, sent from the object metadata management unit, and the object data and the data update position information sent from the client, as well as the object data update request, to the data conversion unit,

(b4) the data conversion unit converts the request from the message transfer/analysis unit to an object update command based on the given data format for processing by the data conversion unit and sends the update command, for which the object data is specified, to the data access unit, and

the data access unit sends the update command and object data to the file server or the database server based on the object update request from the data conversion unit and

(b5) in processing of response data indicating that the update request is executed successfully, received from the file server or the database server, the response data is passed through the data access unit, the data conversion unit, and the message transfer/analysis unit, and a notification, which notifies that the object data update request is executed successfully, is sent from the message transfer/analysis unit to the client.

17. A system comprising the data management apparatus according to claim 13, the client, the file server, and the database server.

18. The system according to claim 17, wherein

(c1) the client sends an ancillary information addition request, for which

object ID to which ancillary information is to be added,

data format information,

ancillary information, and

data part information corresponding to the ancillary information are specified, to the message transfer/analysis unit of the data management apparatus,

(c2) the message transfer/analysis unit extracts the object ID, data format information, ancillary information, and data part information corresponding to the ancillary information from the request and sends the data part information and the ancillary information addition request to the object metadata management unit,

(c3) the object metadata management unit extracts metadata on the corresponding object from the object metadata storage unit using the object ID, included in the data part information, compares the data format information included in the data part information with data format information registered in the object metadata storage unit to confirm that the data formats match,

if the data format information specified by the client matches the data format information registered in the object metadata storage unit, if the data format is a file and is a stream-type data format, and if data area ancillary information corresponding to the ancillary information is specified in a form of an offset time and an elapsed time from the offset, notifies information indicating that an offset byte and a byte range corresponding to the offset time and the elapsed time from the offset is required, as well as object storage location address information and the data format, to the message transfer/analysis unit,

(c4) the message transfer/analysis unit sends the object storage location address information, offset time, and elapsed time from the offset, notified from the object metadata management unit, as well as an offset byte and byte range acquisition request, to the data conversion unit,

(c5) the data conversion unit generates a data read command, which will be sent to the file server, from the object storage location address information sent from the message transfer/analysis unit and acquires file data from the file server via the data access unit,

expands stream-type data from the acquired file data and file format, analyzes an offset and a range thereof on a storage basis, which correspond to the offset time and the elapsed time from the offset, and

sends the analyzed information to the message transfer/analysis unit,

(c6) the message transfer/analysis unit sends the acquired offset and range to the object metadata management unit,

(c7) the object metadata storage unit adds the ancillary information, which is specified to be added, and the data part information, which has the offset and the range on a storage basis as data and which corresponds to the ancillary information, to the object metadata on the object ID to which the ancillary information is specified to be added, saves the object metadata in the object metadata storage unit, and notifies the message transfer/analysis unit that the addition request is executed successfully, and

(c8) the message transfer/analysis unit receives a response from the object metadata management unit and notifies a response result to the client.

19. The system according to claim 18, wherein

(d1) if the data format information specified by the client matches data format information registered in the object metadata storage unit and if the data format is a file but is not a stream-type data format, the object metadata management unit adds the ancillary information, which is specified to be added, and the data part information corresponding to the ancillary information to the object metadata on the object ID to which the ancillary information is specified to be added, saves the object metadata in the object metadata storage unit, and notifies the message transfer/analysis unit that the addition request is executed successfully, and

(d2) the message transfer/analysis unit receives a response from the object metadata management unit and notifies a response result to the client.

20. The system according to claim 19 wherein

(e1)) the client sends ancillary information related to object data, as well as an object list acquisition request, to the message transfer/analysis unit of the data management apparatus,

(e2) the message transfer/analysis unit extracts the ancillary information, included in the object list acquisition request received from the client, and sends the ancillary information and the object list acquisition request to the object metadata management unit,

(e3) the object metadata management unit extracts object IDs having ancillary information, which matches the specified ancillary information, from the object metadata storage unit and sends the extracted object IDs, as well as data format information, object storage location address information, ancillary information related to the objects, and data part information related to the ancillary information, to the message transfer/analysis unit, and

(e4) the message transfer/analysis unit creates a list of data format information, ancillary information, and data part information related to the ancillary information for each object ID based on the information sent from the object metadata management unit, and returns the created list to the client.

21. The system according to claim 20 wherein

(f1) the client selects an object, from which data is to be acquired, based on the information on the list concerning the object IDs and sends a data read request, for which an object ID, object storage location address information, data format information, ancillary information, and data part information related to the ancillary information are specified, to the message transfer/analysis unit of the data management apparatus,

(f2) the message transfer/analysis unit transfers an object data read request, for which the object storage location address information, data format information, and data part information related to the ancillary information are specified, to the data conversion unit,

(f3) the data conversion unit generates a data read command targeted for un in the file server or the database server where objects are stored, based on the data format information, data storage location address information, and data part information related to the ancillary information transferred from the message transfer/analysis unit, and sends the generated data read command to the data access unit,

a part of object data, which is returned from the file server or database server and is related to the ancillary information, is sent to the client via the data access unit, data conversion unit, and message transfer/analysis unit, and

the data read command is

a file read request based on a file path name, which is the data storage location address information, and an offset byte and a byte range, which are the data part information related to the ancillary information, if the data format indicates a file, or

a database read request based on a table name, which is the data storage location address information, and an SQL (Structured Query Language) query, which is the data part information related to the ancillary information, if the data format indicates a database.

22. A program causing a computer to execute the processing comprising:

mapping data having a plurality of data formats into objects having a common format for consolidated data access; and

assigning ancillary information which is used to represent or describe contents of data to an object and saving the ancillary information by relating thereto data part information which indicates to which data area of the object the ancillary information is related.

23. The program according to claim 22, the program further causing the computer to execute the processing comprising:

searching for an object based on the ancillary information; and

reading a part of object data based on the ancillary information and the data part information related to the ancillary information.

24. A data management method comprising:

mapping data having a plurality of data formats into objects having a common format for consolidated data access; and

assigning ancillary information which is used to represent or describe contents of data to an object and saves the ancillary information by relating thereto data part information which indicates to which data area of the object the ancillary information is related.

25. The data management method according to claim 24, further comprising:

searching for an object based on the ancillary information; and

reading a part of object data based on the ancillary information and the data part information related to the ancillary information.

26. A data management method that manages access to data of an object, the method comprising:

storing and managing ancillary information and data part information, in association with an object ID which identifies an object, as metadata in a first storage area and storing and managing a data format of the object and information on a storage location address, where the data of the object is stored, in association with the object ID in a second storage area, the ancillary information used to represent or describe contents of data of the object, the data part information indicating to which data area of the object the ancillary information is related;

receiving a read request from a client and, based on an object ID specified by the read request, generating an object data read command corresponding to the data format information and the data part information by referencing the first storage area and the second storage area using an object ID specified by the read request; and

selectively reading a data part of object data, corresponding to the ancillary information, from a storage unit where in which the data of the object is stored.

27. A program causing a computer to execute processing of managing access to data of an object, the processing comprising:

storing and managing ancillary information and data part information, in association with an object ID which identifies an object, as metadata in a first storage area and storing and managing a data format of the object and information on a storage location address, where the data of the object is stored, in association with the object ID in a second storage area, the ancillary information used to represent or describe contents of data of the object, the data part information indicating to which data area of the object the ancillary information is related;

receiving a read request from a client and, based on an object ID specified by the read request, generating an object data read command corresponding to the data format information and the data part information by referencing the first storage area and the second storage area using an object ID specified by the read request; and

selectively reading a data part of object data, corresponding to the ancillary information, from a storage unit in which the data of the object is stored.