SUPPORTING BIG DATA IN ENTERPRISE CONTENT MANAGEMENT SYSTEMS

• US 20140229514A1
• Filed: 02/13/2013
• Published: 08/14/2014
• Est. Priority Date: 02/13/2013
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: SHAZLY, Hassan A.
• Granted Patent: US 9,348,855 B2
• Time in Patent Office: Days
• US Class Current: 707/812
• CPC Class Codes: G06F 16/2219 Large Object storage; Management thereof
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

Provided are a method, computer program product, and system for storing a data stream. A distributed library server is configured with a plurality of data servers. A data type is mapped to at least one of the plurality of data servers. A portion of a data stream containing data of the data type is selected. An object part including the portion of the data stream is generated, and the object part is stored at one of the data servers mapped to the data.

Claims

1. A computer implemented method for storing a data stream, comprising:

configuring a distributed library server with a plurality of data servers, by at least one computing processor;

mapping a data type to at least one of the plurality of data servers, by the at least one computing processor;

selecting a portion of the data stream containing data of the data type, by the at least one computing processor;

generating an object part including the portion of the data stream, by the at least one computing processor; and

storing the object part at one of the data servers mapped to the data type of the data, by the at least one computing processor.

2. The method of claim 1, further comprising:

storing the data stream as a collection of the object parts.

3. The method of claim 1, wherein selecting further comprises:

receiving a control signal; and

upon receiving the control signal, identifying the portion.

4. The method of claim 1, wherein selecting further comprises:

determining a data format of the portion of the data stream; and

upon a change of the data format, identifying the portion.

5. The method of claim 1, further comprising:

indexing the object part with text information contained in the data stream.

6. The method of claim 1, further comprising:

processing the data stream with an executable process.

7. The method of claim 6 further comprising:

executing the executable process at a node determined by the distributed library server.

8. A system for storing a data stream, comprising:

a processor; and

a storage device connected to the processor, wherein the storage device has stored thereon a program, wherein the processor is configured to execute instructions of the program to perform operations, and wherein the operations comprise;

configuring a distributed library server with a plurality of data servers;

mapping a data type to at least one of the plurality of data servers;

selecting a portion of the data stream containing data of the data type;

generating an object part including the portion of the data stream; and

storing the object part at one of the data servers mapped to the data type of the data.

9. The system of claim 8, wherein the operations further comprise:

storing the data stream as a collection of the object parts.

10. The system of claim 8, wherein the operation of selecting includes:

receiving a control signal; and

upon receiving the control signal, identifying the portion.

11. The system of claim 8, wherein the operation of selecting includes:

determining a data format of the portion of the data stream; and

upon a change of the data format, identifying the portion.

12. The system of claim 8, wherein the operations further comprise:

indexing the object part with text information contained in the data stream.

13. The system of claim 8, wherein the operations further comprise:

processing the data stream with an executable process.

14. The system of claim 13, wherein the operations further comprise:

executing the executable process at a node determined by the distributed library server.

15. A computer program product for storing a data stream, the computer program product comprising:

a computer readable storage medium having computer readable program code embodied therewith, the computer readable program code, executable by at least one processor of a computer is configured to perform;

configuring a distributed library server with a plurality of data servers;

mapping a data type to at least one of the plurality of data servers;

selecting a portion of the data stream containing data of the data type;

generating an object part including the portion of the data stream; and

storing the object part at one of the data servers mapped to the data type of the data.

16. The computer program product of claim 15, wherein the computer readable program code, executable by the at least one processor of the computer, is configured to perform:

storing the data stream as a collection of the object parts.

17. The computer program product of claim 15, wherein the computer readable program code performing the selecting is configured to include:

receiving a control signal; and

upon receiving the control signal, identifying the portion.

18. The computer program product of claim 15, wherein the computer readable program code performing the selecting is configured to include:

determining a data format of the portion of the data stream; and

upon a change of the data format, identifying the portion.

19. The computer program product of claim 15, wherein the computer readable program code, executable by the at least one processor of the computer, is configured to perform:

indexing the object part with text information contained in the data stream.

20. The computer program product of claim 15, wherein the computer readable program code, executable by the at least one processor of the computer, is configured to perform:

processing the data stream with an executable process.

21. The computer program product of claim 20, wherein the computer readable program code, executable by the at least one processor of the computer, is configured to perform:

executing the executable process at a node determined by the distributed library server.