Framework for extremely large complex objects (XLCOs)

• US 9,110,907 B2
• Filed: 09/29/2010
• Issued: 08/18/2015
• Est. Priority Date: 09/29/2010
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Shazly, Hassan A.
• Primary Examiner(s): Hershley, Mark E
• Application Number:
  US12/894,013
• Publication Number:
  US 20120078930A1
• Time in Patent Office: 1,784 Days
• Field of Search: 707/756
• US Class Current: 1/1
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/2219 Large Object storage; Management thereof

Abstract

Provided are techniques for receiving multiple data streams having different data formats. The multiple data streams are converted into an Extremely Large Complex Object (XLCO) file by forming multiple XLCO segments corresponding to the multiple data streams, wherein different data formats are stored in different XLCO segments, and wherein different XLCO segments are different sizes. The XLCO segments of the XLCO are stored in different physical file systems having different operating systems.

Claims

1. A method, comprising:

converting, using a computer including a processor, data streams having different data formats into Extremely Large Complex Object (XLCO) segments that form a file;

storing the file by;

storing a first XLCO segment of the XLCO segments in a first physical file system accessed with a first operating system with the first XLCO segment identified by a first value in an XLCO segment structure; and

storing a second XLCO segment of the XLCO segments in a second physical file system accessed with a second operating system with the second XLCO segment identified by a second value in the XLCO segment structure;

in response to receiving an XLCO request to access the first XLCO segment that specifies the first value, mapping the XLCO request to an operating system request for access by the first operating system of the first physical file system; and

in response to receiving an XLCO request to access the second XLCO segment that specifies the second value, mapping the XLCO request to an operating system request for access by the second operating system of the second physical file system.

2. The method of claim 1, further comprising:

updating XLCO metadata, wherein the XLCO metadata includes a header section and a segment section for each of the XLCO segments.

3. The method of claim 1, further comprising:

using XLCO metadata to retrieve one or more of the XLCO segments.

4. The method of claim 1, further comprising:

processing different XLCO segments in parallel using different processors.

5. The method of claim 1, further comprising:

processing different XLCO segments individually using different processors.

6. The method of claim 1, further comprising:

performing one of appending an XLCO segment to an XLCO file, inserting an XLCO segment into the XLCO file, deleting an XLCO segment from the XLCO file, physically linking XLCO segments in the XLCO file, and logically linking XLCO segments in the XLCO file.

7. The method of claim 1, further comprising:

performing one of reading bytes from an XLCO segment, writing bytes to an XLCO segment, appending bytes to an XLCO segment, and deleting bytes from an XLCO segment.

8. The method of claim 1, further comprising:

using an N-dimensional XLCO segment structure having rows and columns, wherein an XLCO segment is identified by a row and column value.

9. A system, comprising:

a processor; and

storage coupled to the processor, wherein the storage has stored thereon a computer program, and wherein the processor executes the computer program to perform operations, the operations comprising;

converting data streams having different data formats into Extremely Large Complex Object (XLCO) segments that form a file;

storing the file by;

storing a first XLCO segment of the XLCO segments in a first physical file system accessed with a first operating system with the first XLCO segment identified by a first value in an XLCO segment structure; and

storing a second XLCO segment of the XLCO segments in a second physical file system accessed with a second operating system with the second XLCO segment identified by a second value in the XLCO segment structure;

in response to receiving an XLCO request to access the first XLCO segment that specifies the first value, mapping the XLCO request to an operating system request for access by the first operating system of the first physical file system; and

in response to receiving an XLCO request to access the second XLCO segment that specifies the second value, mapping the XLCO request to an operating system request for access by the second operating system of the second physical file system.

10. The system of claim 9, wherein the operations further comprise:

updating XLCO metadata, wherein the XLCO metadata includes a header section and a segment section for each of the XLCO segments.

11. The system of claim 9, wherein the operations further comprise:

using XLCO metadata to retrieve one or more of the XLCO segments.

12. The system of claim 9, wherein the operations further comprise:

processing different XLCO segments in parallel using different processors.

13. The system of claim 9, wherein the operations further comprise:

processing different XLCO segments individually using different processors.

14. The system of claim 9, wherein the operations further comprise:

performing one of appending an XLCO segment to an XLCO file, inserting an XLCO segment into the XLCO file, deleting an XLCO segment from the XLCO file, physically linking XLCO segments in the XLCO file, and logically linking XLCO segments in the XLCO file.

15. The system of claim 9, wherein the operations further comprise:

performing one of reading bytes from an XLCO segment, writing bytes to an XLCO segment, appending bytes to an XLCO segment, and deleting bytes from an XLCO segment.

16. The system of claim 9, wherein the operations further comprise:

using an N-dimensional XLCO segment structure having rows and columns, wherein an XLCO segment is identified by a row and column value.

17. A computer program product comprising a non-transitory computer readable storage medium including a computer readable program, wherein the computer readable program when executed by a processor on a computer causes the computer to perform:

converting data streams having different data formats into Extremely Large Complex Object (XLCO) segments that form a file;

storing the file by;

storing a first XLCO segment of the XLCO segments in a first physical file system accessed with a first operating system with the first XLCO segment identified by a first value in an XLCO segment structure; and

storing a second XLCO segment of the XLCO segments in a second physical file system accessed with a second operating system with the second XLCO segment identified by a second value in the XLCO segment structure;

in response to receiving an XLCO request to access the first XLCO segment that specifies the first value, mapping the XLCO request to an operating system request for access by the first operating system of the first physical file system; and

in response to receiving an XLCO request to access the second XLCO segment that specifies the second value, mapping the XLCO request to an operating system request for access by the second operating system of the second physical file system.

18. The computer program product of claim 17, wherein the computer readable program when executed by the processor on the computer causes the computer to perform:

updating XLCO metadata, wherein the XLCO metadata includes a header section and a segment section for each of the XLCO segments.

19. The computer program product of claim 17, wherein the computer readable program when executed by the processor on the computer causes the computer to perform:

using XLCO metadata to retrieve one or more of the XLCO segments.

20. The computer program product of claim 17, wherein the computer readable program when executed by the processor on the computer causes the computer to perform:

processing different XLCO segments in parallel using different processors.

21. The computer program product of claim 17, wherein the computer readable program when executed by the processor on the computer causes the computer to perform:

processing different XLCO segments individually using different processors.

22. The computer program product of claim 17, wherein the computer readable program when executed by the processor on the computer causes the computer to perform:

performing one of appending an XLCO segment to an XLCO file, inserting an XLCO segment into the XLCO file, deleting an XLCO segment from the XLCO file, physically linking XLCO segments in the XLCO file, and logically linking XLCO segments in the XLCO file.

23. The computer program product of claim 17, wherein the computer readable program when executed by the processor on the computer causes the computer to perform:

performing one of reading bytes from an XLCO segment, writing bytes to an XLCO segment, appending bytes to an XLCO segment, and deleting bytes from an XLCO segment.

24. The computer program product of claim 17, wherein the computer readable program when executed by the processor on the computer causes the computer to perform:

using an N-dimensional XLCO segment structure having rows and columns, wherein an XLCO segment is identified by a row and column value.