FRAMEWORK FOR EXTREMELY LARGE COMPLEX OBJECTS (XLCOS)

• US 20120078930A1
• Filed: 09/29/2010
• Published: 03/29/2012
• Est. Priority Date: 09/29/2010
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: SHAZLY, Hassan A.
• Granted Patent: US 9,110,907 B2
• Time in Patent Office: Days
• US Class Current: 707/756
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/2219 Large Object storage; Management thereof

Abstract

Provided are techniques for receiving multiple data streams having different data formats. The multiple data streams are converted into an Extremely Large Complex Object (XLCO) file by forming multiple XLCO segments corresponding to the multiple data streams, wherein different data formats are stored in different XLCO segments, and wherein different XLCO segments are different sizes. The XLCO segments of the XLCO are stored in different physical file systems having different operating systems.

Claims

1. A method, comprising:

receiving, using a computer including a processor, multiple data streams having different data formats;

converting the multiple data streams into an Extremely Large Complex Object (XLCO) file by forming multiple XLCO segments corresponding to the multiple data streams, wherein different data formats are stored in different XLCO segments, and wherein different XLCO segments are different sizes; and

storing the XLCO segments of the XLCO file in different physical file systems having different operating systems.

2. The method of claim 1, further comprising:

updating one copy of XLCO metadata for the XLCO file, wherein the XLCO metadata includes a header section for the XLCO file and a segment section for each XLCO segment of the XLCO file.

3. The method of claim 2, further comprising:

using the XLCO metadata to retrieve one or more of the XLCO segments of the XLCO file.

4. The method of claim 1, further comprising:

processing the different XLCO segments in parallel using different processors.

5. The method of claim 1, further comprising:

processing the different XLCO segments individually using different processors.

6. The method of claim 1, further comprising:

performing one of appending an XLCO segment to the XLCO file, inserting an XLCO segment into the XLCO file, deleting an XLCO segment from the XLCO file, physically linking XLCO segments in the XLCO file, and logically linking XLCO segments in the XLCO file.

7. The method of claim 1, further comprising:

performing one of reading bytes from an XLCO segment, writing bytes to an XLCO segment, appending bytes to an XLCO segment, and deleting bytes from an XLCO segment.

8. The method of claim 1, further comprising:

using an N-dimensional XLCO segment structure having rows and columns, wherein an XLCO segment is identified by a row and column value.

9. A system, comprising:

a processor; and

hardware logic coupled to the processor and performing operations, the operations comprising;

receiving multiple data streams having different data formats;

converting the multiple data streams into an Extremely Large Complex Object (XLCO) file by forming multiple XLCO segments corresponding to the multiple data streams, wherein different data formats are stored in different XLCO segments, and wherein different XLCO segments are different sizes; and

storing the XLCO segments of the XLCO file in different physical file systems having different operating systems.

10. The system of claim 9, wherein the operations further comprise:

updating one copy of XLCO metadata for the XLCO file, wherein the XLCO metadata includes a header section for the XLCO file and a segment section for each XLCO segment of the XLCO file.

11. The system of claim 10, wherein the operations further comprise:

using the XLCO metadata to retrieve one or more of the XLCO segments of the XLCO file.

12. The system of claim 9, wherein the operations further comprise:

processing the different XLCO segments in parallel using different processors.

13. The system of claim 9, wherein the operations further comprise:

processing the different XLCO segments individually using different processors.

14. The system of claim 9, wherein the operations further comprise:

performing one of appending an XLCO segment to the XLCO file, inserting an XLCO segment into the XLCO file, deleting an XLCO segment from the XLCO file, physically linking XLCO segments in the XLCO file, and logically linking XLCO segments in the XLCO file.

15. The system of claim 9, wherein the operations further comprise:

performing one of reading bytes from an XLCO segment, writing bytes to an XLCO segment, appending bytes to an XLCO segment, and deleting bytes from an XLCO segment.

16. The system of claim 9, wherein the operations further comprise:

using an N-dimensional XLCO segment structure having rows and columns, wherein an XLCO segment is identified by a row and column value.

17. A computer program product comprising a computer readable storage medium including a computer readable program, wherein the computer readable program when executed by a processor on a computer causes the computer to perform:

receiving multiple data streams having different data formats;

converting the multiple data streams into an Extremely Large Complex Object (XLCO) file by forming multiple XLCO segments corresponding to the multiple data streams, wherein different data formats are stored in different XLCO segments, and wherein different XLCO segments are different sizes; and

storing the XLCO segments of the XLCO file in different physical file systems having different operating systems.

18. The computer program product of claim 17, wherein the computer readable program when executed by the processor on the computer causes the computer to perform:

updating one copy of XLCO metadata for the XLCO file, wherein the XLCO metadata includes a header section for the XLCO file and a segment section for each XLCO segment of the XLCO file.

19. The computer program product of claim 18, wherein the computer readable program when executed by the processor on the computer causes the computer to perform:

using the XLCO metadata to retrieve one or more of the XLCO segments of the XLCO file.

20. The computer program product of claim 17, wherein the computer readable program when executed by the processor on the computer causes the computer to perform:

processing the different XLCO segments in parallel using different processors.

21. The computer program product of claim 17, wherein the computer readable program when executed by the processor on the computer causes the computer to perform:

processing the different XLCO segments individually using different processors.

22. The computer program product of claim 17, wherein the computer readable program when executed by the processor on the computer causes the computer to perform:

performing one of appending an XLCO segment to the XLCO file, inserting an XLCO segment into the XLCO file, deleting an XLCO segment from the XLCO file, physically linking XLCO segments in the XLCO file, and logically linking XLCO segments in the XLCO file.

23. The computer program product of claim 17, wherein the computer readable program when executed by the processor on the computer causes the computer to perform:

performing one of reading bytes from an XLCO segment, writing bytes to an XLCO segment, appending bytes to an XLCO segment, and deleting bytes from an XLCO segment.

24. The computer program product of claim 17, wherein the computer readable program when executed by the processor on the computer causes the computer to perform:

using an N-dimensional XLCO segment structure having rows and columns, wherein an XLCO segment is identified by a row and column value.