Method and apparatus for collecting and storing statistics data from network elements using scalable architecture

• US 9,430,502 B1
• Filed: 09/10/2010
• Issued: 08/30/2016
• Est. Priority Date: 09/10/2010
• Status: Active Grant

• Current Assignee: Tellabs Operations Incorporated (Infinera Corp.)
• Original Assignee: Tellabs Operations Incorporated (Infinera Corp.)
• Inventors: Purushothaman, Mahesh, Upadhyay, Ajaykumar L., Palanisamy, Anand K., Dey, Debashis
• Primary Examiner(s): Hershley, Mark E
• Application Number:
  US12/879,223
• Time in Patent Office: 2,181 Days
• Field of Search: 707/688
• US Class Current: 1/1
• CPC Class Codes: G06F 11/3006 where the computing system is distributed, e.g. networked systems, clusters, multiprocessor systems multiprogramming arrangements G06F9/46; allocation of resources G06F9/50
• CPC Class Codes: G06F 11/3065 Monitoring arrangements determined by the means or processing involved in reporting the monitored data error or fault reporting or logging G06F11/0766
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06F 16/2365 Ensuring data consistency and integrity
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: H04L 41/0896 Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities flow or congestion control using dynamic resource allocation, e.g. in-call renegotiation, H04L47/76
• CPC Class Codes: H04L 43/0805 by checking availability
• CPC Class Codes: H04L 43/0852 Delays

Abstract

A method and apparatus for collecting statistics data over a communications network using scalable stateless processes are disclosed. A process capable of obtaining statistics data, in one embodiment, detects a delay associated with the availability of statistics data in accordance with predefined lag duration, and subsequently, adds one or more stateless processes to a pool of stateless processes to increase capacity of collecting statistics data. Upon enabling a stateless process to collect statistics data from network element (“NE”) in response to a statistics collecting tasks, the statistics data is stored in a database.

Claims

1. A method for collecting statistics data over a communications network, comprising:

detecting a delay associated with availability of statistics data collected from a plurality of network elements (“

NEs”

) in accordance with a predefined time period;

determining insufficient processing capacity of stateless processes to complete collecting statistics data from all nodes within predefined time duration in response to the delay;

adding one or more additional stateless processes to a pool of stateless processes for scaling up computing power in response to the delay to increase capacity of collecting statistics data;

enabling one of the pool of stateless processes to collect statistics data from an NE in response to one of a list of statistics collecting tasks, wherein enabling one of the pool of stateless processes includes fetching the statistics data in response to a data collection manager (“

DCM”

) task generated by a scheduler; and

storing the statistics data in a database.

2. The method of claim 1, further comprising,

fetching the statistics data from the database;

and

generating a status report associated with the NE in accordance with the statistics data.

3. The method of claim 1, further comprising generating a comma-separated values (“

CSV”

) file based on information stored in the database.

4. The method of claim 1, wherein detecting a delay associated with availability of statistics data further includes:

identifying a predefined waiting period needed for fetching the statistics data from an NE for a predefined time interval; and

determining duration of the delay is greater than the predefined waiting period.

5. The method of claim 1, wherein adding one or more additional stateless processes to a plurality of stateless processes includes coupling an additional computer, having one or more stateless data collection manager (“

DCM”

) containers capable of processing one or more DCM tasks, to a group of computers containing a plurality of DCM containers.

6. The method of claim 1, wherein enabling one of the plurality of stateless process to collect statistics data from an NE includes obtaining the statistics data from a node over a network employing file transfer protocol (“

FTP).

7. The method of claim 1, wherein fetching the statistics data in response to a data collection manager (“

DCM”

) task includes generating the DCM task in response to input from orchestrator.

8. The method of claim 7, further comprising activating the scheduler to generate information relating to time of each DCM task to be executed.

9. The method of claim 8, further comprising enabling the orchestrator to generate information relating to tasks listed in the DCM task.

10. The method of claim 1, wherein storing the statistics data in a database includes storing the statistics data in a customized partitioned Oracle®

database.

11. A communications network, comprising:

a scheduler and orchestrator module in a network server coupled to the communications network and configured to generate a list of data collecting tasks for obtaining statistics data from various network nodes, wherein the scheduler and orchestrator module is configured to determine insufficient processing capacity of stateless processes for collecting statistics data from all nodes within predefined time duration;

a pool of stateless processes residing in one or more network devices coupled to the scheduler and orchestrate module and configured to collect information from the various network nodes in accordance with the list of data collecting tasks, wherein a stateless process is dynamically added to or removed from the pool of stateless processes for increasing or reducing capacity of collecting statistics data in accordance with computing power needed to perform data collecting within a predefined time period; and

a database coupled to the plurality of stateless processes and capable of storing the statistics data collected from one or more network nodes.

12. The network of claim 11, further includes a scalable module coupled to the scheduler and orchestrator and operable to add additional stateless processes to the plurality of stateless processes when processing capacity of the plurality of the stateless processes is insufficient to collect statistics data from the network nodes within a predefined time interval.

13. The network of claim 12, wherein each network node is a network elements (“

NEs”

) including a computer, a server, a mobile device, or a cluster of systems coupled to the communications network.

14. The network of claim 13, wherein a list of data collecting tasks is a list of data collection manager (“

DCM”

).

15. The network of claim 11, wherein the database is an Oracle®

database which is partitioned in multiple levels to minimize performance interference between a write operation and a read operation of statistics data.

16. A method for collecting statistics over a communications network, comprising:

partitioning a relational database into a daily level and an interval level;

detecting a delay associated with availability of statistics data collected from a plurality of network elements (“

NEs”

) in accordance with a predefined expected period;

determining insufficient processing capacity of stateless processes to complete collecting statistics data from all nodes within predefined time duration in response to the delay;

adding one or more stateless processes to a pool of existing stateless processes for scaling up computing power in response to the delay to increase capacity of processing power;

collecting statistics data from NEs of the communications network in response to a list of tasks;

wherein collecting statistics data includes fetching statistics data in response to a data collection manager (“

DCM”

) task generated by a scheduler; and

storing the statistics data in the relational database in accordance with the daily level and the interval level.

17. The method of claim 16, further comprising,

searching and fetching the statistics data stored in the relational database;

and

generating a status report in accordance with the statistics data.

18. The method of claim 16, wherein partitioning a relational database into a daily level and an interval level further includes:

allocating a portion of Oracle®

relational database in accordance with a calendar dates; and

allocating another portion of Oracle®

relational database in accordance with a collection time interval.

19. The method of claim 16, wherein detecting a delay associated with availability of statistics data in accordance with a predefined expected duration further includes:

identifying duration of the predefined expected period which is time needed to fetch the statistics data; and

determining duration of the delay is greater than the duration of the predefined expected period.

20. The method of claim 16, wherein adding one or more stateless processes to a pool of existing stateless processes includes coupling an additional computer, having one or more stateless data collection manager (“

DCM”

) containers capable of processing one or more DCM tasks, to a group of computers containing a plurality of DCM containers.