Elastic scaling of data volume

• US 8,849,779 B2
• Filed: 10/31/2013
• Issued: 09/30/2014
• Est. Priority Date: 08/12/2011
• Status: Active Grant

• Current Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Original Assignee: Splunk Inc. (Cisco Systems, Inc.)
• Inventors: Das, Robin Kumar, Ago, Ledio, Shanaghy, Declan Gerard, Gupta, Gaurav
• Primary Examiner(s): To, Baoquoc N
• Application Number:
  US14/068,445
• Publication Number:
  US 20140059036A1
• Time in Patent Office: 334 Days
• Field of Search: 707/696
• US Class Current: 707/696
• CPC Class Codes: G06F 16/13 File access structures, e.g. distributed indices arrangements of input from, or output to, record carriers G06F3/06
• CPC Class Codes: G06F 16/134 Distributed indices
• CPC Class Codes: G06F 16/162 Delete operations erasing in storage systems G06F3/0652
• CPC Class Codes: G06F 16/21 Design, administration or maintenance of databases
• CPC Class Codes: G06F 16/2228 Indexing structures
• CPC Class Codes: G06F 16/2272 Management thereof
• CPC Class Codes: G06F 16/901 Indexing; Data structures therefor; Storage structures for retrieval from the web G06F16/951
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques
• CPC Class Codes: G06F 16/9537 Spatial or temporal dependent retrieval, e.g. spatiotemporal queries
• CPC Class Codes: G06F 16/955 using information identifiers, e.g. uniform resource locators [URL]
• CPC Class Codes: G06F 16/986 Document structures and storage, e.g. HTML extensions
• CPC Class Codes: G06F 21/6218 to a system of files or objects, e.g. local or distributed file system or database
• CPC Class Codes: G06F 9/4887 involving deadlines, e.g. rate based, periodic
• CPC Class Codes: G06F 9/5088 involving task migration
• CPC Class Codes: G06Q 20/145 Payments according to the detected use or quantity
• CPC Class Codes: H04L 12/1435 volume-based
• CPC Class Codes: H04L 63/10 for controlling access to devices or network resources
• CPC Class Codes: H04L 63/105 Multiple levels of security
• CPC Class Codes: H04L 67/1097 for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

Embodiments are directed towards a system and method for a cloud-based front end that may abstract and enable access to the underlying cloud-hosted elements and objects that may be part of a multi-tenant application, such as a search application. Search objects may be employed to access indexed objects. An amount of indexed data accessible to a user may be based on an index storage limit selected by the user, such that data that exceeds the index storage limit may continue to be indexed. Also, one or more projects can be elastically scaled for a user to provide resources that may meet the specific needs of each project.

Claims

1. A method comprising:

evaluating a request to adjust at least one resource for a project, the at least one resource includes at least any combination of;

CPU quota, memory quota, or network bandwidth quota;

determining an impact of the request upon system resources, the determining including determining necessary resources required to fulfill the request to adjust the at least one resource for the project;

in response to a determination that the request to adjust the at least one resource for the project does not impact system resources, determining an impact of the request to adjust the at least one resource for the project upon local resources;

in response to a determination that the request to adjust the at least one resource for the project impacts system resources, placing the request to adjust the at least one resource for the project in a request first until the request to adjust the at least one resource for the project can be fulfilled;

wherein the method is performed by one or more computing devices.

2. The method of claim 1, further comprising:

in response to a determination that the request to adjust the at least one resource for the project impacts local resources, migrating a virtual machine allocated to the project to another physical machine.

3. The method of claim 1, further comprising:

in response to a determination that the request to adjust the at least one resource for the project impacts local resources, allocating more resources for the project by a virtual machine allocated to the project.

4. The method of claim 1, further comprising:

in response to a determination that the request to adjust the at least one resource for the project impacts local resources, scheduling a future allocation of increased resources for the project by a virtual machine allocated to the project.

5. The method of claim 1, wherein the project is associated with an index store where data associated with the project are stored, the index store accessible by at least one indexer server, the at least one indexer server associated with at least one virtual machine.

6. The method of claim 1, further comprising:

causing a user interface to display at least one user interface control to adjust at least one parameter related to a project;

in response to input from the at least one user interface control, creating a request to adjust at least one resource.

7. The method of claim 1, wherein local resources are resources allocated to a virtual machine, and wherein system resources are resources allocated to a physical machine where a virtual machine resides.

8. The method of claim 1, wherein the project comprises:

indexing raw data.

9. The method of claim 1, wherein the project comprises:

storing indexed data.

10. The method of claim 1, wherein the project comprises:

searching indexed data.

11. The method of claim 1, wherein the project comprises:

dividing raw data into time stamped searchable events, storing the time-stamped searchable events, and searching the time-stamped searchable events.

12. An apparatus comprising:

a request receiver subsystem, implemented at least partially in hardware, that receives a request to adjust at least one resource for a project, the at least one resource includes at least any combination of;

CPU quota, memory quota, or network bandwidth quota;

a system resource determination subsystem, implemented at least partially in hardware, that determines an impact of the request upon system resources, the system resource determination subsystem further determines necessary resources required to fulfill the request to adjust the at least one resource for the project;

a local resource determination subsystem, implemented at least partially in hardware, that, in response to a determination that the request to adjust the at least one resource for the project does not impact system resources, determines an impact of the request to adjust the at least one resource for the project upon local resources;

a subsystem, implemented at least partially in hardware, that, in response to a determination that the request to adjust the at least one resource for the project impacts system resources, places the request to adjust the at least one resource for the project in a request list until the request to adjust the at least one resource for the project can be fulfilled.

13. The apparatus of claim 12, further comprising:

a subsystem, implemented at least partially in hardware, that, in response to a determination that the request to adjust the at least one resource for the project impacts local resources, migrates a virtual machine allocated to the project to another physical machine.

14. The apparatus of claim 12, wherein in response to a determination that the request to adjust the at least one resource for the project impacts local resources, a virtual machine allocated to the project allocates more resources for the project.

15. The apparatus of claim 12, wherein in response to a determination that the request to adjust the at least one resource for the project impacts local resources, a virtual machine allocated to the project schedules a future allocation of increased resources for the project.

16. The apparatus of claim 12, wherein the project is associated with an index store where data associated with the project are stored, the index store accessible by at least one indexer server, the at least one indexer server associated with at least one virtual machine.

17. The apparatus of claim 12, further comprising:

a subsystem, implemented at least partially in hardware, that causes a user interface to display at least one user interface control to adjust at least one parameter related to a project;

a subsystem, implemented at least partially in hardware, that, in response to input from the at least one user interface control, creates a request to adjust at least one resource.

18. The apparatus of claim 12, wherein local resources are resources allocated to a virtual machine, and wherein system resources are resources allocated to a physical machine where a virtual machine resides.

19. The apparatus of claim 12, wherein the project comprises:

a subsystem, implemented at least partially in hardware, that divides raw data into time stamped searchable events;

a subsystem, implemented at least partially in hardware, that stores the time-stamped searchable events; and

a subsystem, implemented at least partially in hardware, that searches the time-stamped searchable events.

20. A non-transitory computer readable medium, storing software instructions, which when executed by one or more processors cause performance of steps of:

evaluating a request to adjust at least one resource for a project, the at least one resource includes at least any combination of;

CPU quota, memory quota, or network bandwidth quota;

determining an impact of the request upon system resources, the determining including determining necessary resources required to fulfill the request to adjust the at least one resource for the project;

in response to a determination that the request to adjust the at least one resource for the project does not impact system resources, determining an impact of the request to adjust the at least one resource for the project upon local resources;

in response to a determination that the request to adjust the at least one resource for the project impacts system resources, placing the request to adjust the at least one resource for the project in a request list until the request to adjust the at least one resource for the project can be fulfilled.

21. The non-transitory computer readable medium of claim 20, further comprising:

in response to a determination that the request to adjust the at least one resource for the project impacts local resources, migrating a virtual machine allocated to the project to another physical machine.

22. The non-transitory computer readable medium of claim 20, further comprising:

in response to a determination that the request to adjust the at least one resource for the project impacts local resources, allocating more resources for the project by a virtual machine allocated to the project.

23. The non-transitory computer readable medium of claim 20, further comprising:

in response to a determination that the request to adjust the at least one resource for the project impacts local resources, scheduling a future allocation of increased resources for the project by a virtual machine allocated to the project.

24. The non-transitory computer readable medium of claim 20, wherein the project is associated with an index store where data associated with the project are stored, the index store including at least one indexer server, the at least one indexer server associated with at least one virtual machine.

25. The non-transitory computer readable medium of claim 20, further comprising:

causing a user interface to display at least one user interface control to adjust at least one parameter related to a project;

in response to input from the at least one user interface control, creating a request to adjust at least one resource.

26. The non-transitory computer readable medium of claim 20, wherein local resources are resources allocated to a virtual machine, and wherein system resources are resources allocated to a physical machine where a virtual machine resides.

27. The non-transitory computer readable medium of claim 20, wherein the project comprises:

indexing raw data.

28. The non-transitory computer readable medium of claim 20, wherein the project comprises:

storing indexed data.

29. The non-transitory computer readable medium of claim 20, wherein the project comprises:

searching indexed data.

30. The non-transitory computer readable medium of claim 20, wherein the project comprises:

dividing raw data into time stamped searchable events, storing the time-stamped searchable events, and searching the time-stamped searchable events.