Distributed storage device, storage node, data providing method, and medium

• US 9,870,402 B2
• Filed: 09/27/2013
• Issued: 01/16/2018
• Est. Priority Date: 09/28/2012
• Status: Expired due to Fees

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Kobayashi, Dai, Yamakawa, Satoshi
• Primary Examiner(s): Kuddus, Daniel
• Application Number:
  US14/427,182
• Publication Number:
  US 20150347516A1
• Time in Patent Office: 1,572 Days
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/24568 Data stream processing; Continuous queries
• CPC Class Codes: G06F 16/24575 using context
• CPC Class Codes: G06F 16/248 Presentation of query results
• CPC Class Codes: G06F 16/27 Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
• CPC Class Codes: G06F 16/9535 Search customisation based on user profiles and personalisation

Abstract

A distributed storage device according to the present invention includes: a plurality of storage nodes, the plurality of storage nodes includes: a data storage unit that accumulates stream data output from a device; a first time frame generation unit that generates a time frame synchronized with another storage node and associates a data element included in stream data accumulated in the data storage unit with one of time frames; a data selection unit that selects a data element associated with a predetermined time frame from the stream data accumulated in the data storage unit, based on an access request from a client terminal; and a data transmission unit that transmits a data element selected by the data selection unit to the client terminal.

Claims

1. A distributed storage device comprising:

a plurality of storage nodes,

the plurality of storage nodes comprising at least one hardware processor configured to implement;

a data storage unit configured to accumulate stream data output from a device;

a first time frame generation unit configured to generate a first time frame synchronized with another storage node and associate a data element included in stream data accumulated in the data storage unit with one of the first time frames;

a data selection unit configured to select a data element associated with a predetermined first time frame from the stream data accumulated in the data storage unit, based on an access request from a client terminal, and select a data element associated with a time frame synchronized with a time frame associated with a data element selected by another storage node with respect to an access request from the client terminal, as a data element with respect to a subsequent access request from the client terminal;

a data transmission unit configured to transmit a data element selected by the data selection unit to the client terminal; and

an index server comprising at least one hardware processor configured to implement;

an index storage unit configured to accumulate an index with respect to stream data accumulated in a data storage unit of the plurality of storage nodes;

a second time frame generation unit configured to generate a second time frame synchronized with the plurality of storage nodes and associate an index accumulated in the index storage unit with one of the first time frames and one of the second time frames; and

an index retrieval unit configured to select an index associated with a predetermined first time frame from indexes accumulated in the index storage unit, based on the access request transferred from one of storage nodes of the plurality of storage nodes, and transmit a selected index to the one of storage nodes,

wherein the at least one hardware processor of the plurality of storage nodes is further configured to implement;

a data update unit configured to transmit stream data accumulated in the data storage unit to the index server; and

a data retrieval unit configured to transfer an access request from a client terminal to the index retrieval unit, and

wherein a time frame difference between the first time frame and the second time frame which are associated is designed to be constant.

2. The distributed storage device according to claim 1, wherein

the data selection unit is further configured to select a data element associated with a time frame associated with the selected data element, as a data element with respect to a subsequent access request from the client terminal.

3. The distributed storage device according to claim 1, wherein

the data selection unit is further configured to select a data element associated with a time frame by a predetermined number before a time frame on receiving an access request from the client terminal.

4. The distributed storage device according to claim 1, wherein

the data selection unit is further configured to select a latest data element included in the predetermined time frame.

5. The distributed storage device according to claim 1, wherein

the access request includes an identifier that identifies the client terminal.

6. The distributed storage device according to claim 1, comprising:

at least one hardware processor configured to implement an inter-node synchronization unit configured to generate a request for generating a time frame, wherein

the first time frame generation unit is further configured to generate a time frame depending on a request generated by the inter-node synchronization unit.

7. The distributed storage device according to claim 1, wherein

the data selection unit is further configured to select a data element associated with a time frame on an own storage node corresponding to a time frame on the index server associated with an index selected by the index retrieval unit with respect to an access request from a client terminal transferred to the index server, as a data element with respect to a subsequent access request from the client terminal.

8. The distributed storage device according to claim 1, wherein

the data selection unit is further configured to select a data element associated with a time frame on an own storage node corresponding to a time frame on the index server associated with an index selected by the index retrieval unit with respect to an access request from a client terminal transferred to the index server by another storage node, as a data element with respect to a subsequent access request from the client terminal.

9. A storage node that is one of a plurality of storage nodes included in a distributed storage device, comprising at least one hardware processor configured to implement:

a data storage unit configured to accumulate stream data output from a device;

an inter-node synchronization unit configured to generate a request for generating a time frame;

a first time frame generation unit configured to generate a first time frame synchronized with another storage node and associates a data element included in stream data accumulated in the data storage unit with one of the first time frames, and generate a time frame depending on a request generated by the inter-node synchronization unit;

a data selection unit configured to select a data element associated with a predetermined first time frame from the stream data accumulated in the data storage unit, based on an access request from a client terminal; and

a data transmission unit configured to transmit a data element selected by the data selection unit to the client terminal; and

an index server comprising at least one hardware processor configured to implement;

an index storage unit configured to accumulate an index with respect to stream data accumulated in a data storage unit of the plurality of storage nodes;

a second time frame generation unit configured to generate a second time frame synchronized with the plurality of storage nodes and associate an index accumulated in the index storage unit with one of the first time frames and one of the second time frames; and

an index retrieval unit configured to select an index associated with a predetermined first time frame from indexes accumulated in the index storage unit, based on the access request transferred from one of storage nodes of the plurality of storage nodes, and transmit a selected index to the one of storage nodes,

wherein the at least one hardware processor of the storage node is further configured to implement;

a data update unit configured to transmit stream data accumulated in the data storage unit to the index server; and

a data retrieval unit configured to transfer an access request from a client terminal to the index retrieval unit, and

wherein a time frame difference between the first time frame and the second time frame which are associated is designed to be constant.

10. The storage node according to claim 9, wherein

the data selection unit is further configured to select a data element associated with a time frame associated with the selected data element, as a data element with respect to a subsequent access request from the client terminal.

11. The storage node according to claim 9, wherein

the data selection unit is further configured to select a data element associated with a time frame synchronized with a time frame associated with a data element selected by another storage node with respect to an access request from the client terminal, as a data element with respect to a subsequent access request from the client terminal.

12. The storage node according to claim 9, wherein

the data selection unit is further configured to select a data element associated with a time frame on the storage node corresponding to a time frame on the index server associated with an index selected by the index retrieval unit with respect to an access request from a client terminal transferred to the index server, as a data element with respect to a subsequent access request from the client terminal.

13. The storage node according to claim 9, wherein

the data selection unit is further configured to select a data element associated with a time frame on the storage node corresponding to a time frame on the index server associated with an index selected by the index retrieval unit with respect to an access request from a client terminal transferred to the index server by another storage node, as a data element with respect to a subsequent access request from the client terminal.

14. A data providing method comprising:

accumulating stream data output from a device in a data storage unit by a storage node that is one of a plurality of storage nodes included in a distributed storage device;

generating a first time frame synchronized with another storage node and associating a data element included in stream data accumulated in the data storage unit with one of the first time frames;

selecting a data element associated with a predetermined first time frame from the stream data accumulated in the data storage unit, based on an access request from a client terminal;

transmitting a selected data element to the client terminal;

transmitting stream data accumulated in the data storage unit to an index server by the storage node;

transferring an access request from the client terminal to the index server;

accumulating an index with respect to stream data accumulated in a storage unit of the plurality of storage nodes in an index storage unit by the index server;

generating a second time frame synchronized with the plurality of storage nodes and associating an index accumulated in the index storage unit with one of the first time frames and one of the second time frames; and

selecting an index associated with a predetermined first time frame from indexes accumulated in the index storage unit, based on an access request from the client terminal transferred from one of storage nodes of the plurality of storage nodes, and transmitting a selected index to the one of storage nodes,

wherein a time frame difference between the first time frame and the second time frame which are associated is designed to be constant, and

wherein the storage node selects a data element associated with a time frame associated with the selected data element, as a data element with respect to a subsequent access request from the client terminal.

15. The data providing method according to claim 14, wherein

the storage node selects a data element associated with a time frame synchronized with a time frame associated with a data element selected by another storage node with respect to an access request from the client terminal, as a data element with respect to a subsequent access request from the client terminal.

16. The data providing method according to claim 14, wherein

the storage node selects a data element associated with a time frame on an own storage node corresponding to a time frame on the index server associated with an index selected by the index server with respect to an access request from the client terminal transferred to the index server, as a data element with respect to a subsequent access request from the client terminal.

17. The data providing method according to claim 14, wherein

the storage node selects a data element associated with a time frame on the storage node corresponding to a time frame on the index server associated with an index selected by the index server with respect to an access request from the client terminal transferred to the index server by another storage node, as a data element with respect to a subsequent access request from the client terminal.

18. A computer readable non-transitory medium embodying a program, the program causing a storage node included in a distributed storage device to perform a method, the method comprising:

accumulating stream data output from a device in a data storage unit;

generating a first time frame synchronized with another storage node and associating a data element included in stream data accumulated in the data storage unit with one of the first time frames;

selecting a data element associated with a predetermined first time frame from the stream data accumulated in the data storage unit, based on an access request from a client terminal;

transmitting a selected data element to the client terminal;

transmitting stream data accumulated in the data storage unit to an index server by the storage node;

transferring an access request from the client terminal to the index server;

accumulating an index with respect to stream data accumulated in the storage unit in an index storage unit by the index server;

generating a second time frame synchronized with the plurality of storage nodes and associating an index accumulated in the index storage unit with one of the first time frames and one of the second time frames;

selecting an index associated with a predetermined first time frame from indexes accumulated in the index storage unit, based on an access request from the client terminal transferred from one of storage nodes of the plurality of storage nodes, and transmitting a selected index to the one of storage nodes; and

selecting a data element associated with a time frame associated with the selected data element, as a data element with respect to a subsequent access request from the client terminal,

wherein a time frame difference between the first time frame and the second time frame which are associated is designed to be constant.

19. The computer readable non-transitory medium according to claim 18, the method further comprising:

selecting a data element associated with a time frame synchronized with a time frame associated with a data element selected by another storage node with respect to an access request from the client terminal, as a data element with respect to a subsequent access request from the client terminal.

20. A distributed storage device comprising:

a plurality of storage nodes, the plurality of storage nodes comprising;

a data storage configured to accumulate stream data output from a device;

a first time frame generator configured to generate a first time frame synchronized with another storage node and associate a data element included in stream data accumulated in the data storage with one of the first time frames;

a data selector configured to select a data element associated with a predetermined first time frame from the stream data accumulated in the data storage, based on an access request from a client terminal, and select a data element associated with a time frame by a predetermined number before a time frame on receiving an access request from the client terminal; and

a data transmitter configured to transmit a data element selected by the data selector to the client terminal; and

an index server comprising;

an index storage configured to accumulate an index with respect to stream data accumulated in a data storage of the plurality of storage nodes;

a second time frame generator configured to generate a second time frame synchronized with the plurality of storage nodes and associate an index accumulated in the index storage with one of the first time frames and one of the second time frames; and

an index retriever configured to select an index associated with a predetermined first time frame from indexes accumulated in the index storage, based on the access request transferred from one of storage nodes of the plurality of storage nodes, and transmit a selected index to the one of storage nodes,

wherein the plurality of storage nodes further comprises;

a data updater configured to transmit stream data accumulated in the data storage to the index server; and

a data retriever configured to transfer an access request from a client terminal to the index retriever, and

wherein a time frame difference between the first time frame and the second time frame which are associated is designed to be constant.