Cloud based test environment

• US 10,672,286 B2
• Filed: 10/06/2010
• Issued: 06/02/2020
• Est. Priority Date: 03/14/2010
• Status: Active Grant

• Current Assignee: Kryterion, Inc. (Drake International Incorporated)
• Original Assignee: Kryterion, Inc. (Drake International Incorporated)
• Inventors: Saxon, Steve, Neubeck, Joel, Collins, Michael, Weinmann, Dan, Dorman, William
• Primary Examiner(s): Yip, Jack
• Application Number:
  US12/899,085
• Publication Number:
  US 20110244440A1
• Time in Patent Office: 3,527 Days
• Field of Search: 434362
• CPC Class Codes: G06F 21/316 by observing the pattern of computer usage, e.g. typical user behaviour
• CPC Class Codes: G06F 9/50 Allocation of resources, e.g. of the central processing unit [CPU]
• CPC Class Codes: G06F 9/5027 the resource being a machine, e.g. CPUs, Servers, Terminals
• CPC Class Codes: G06F 9/505 considering the load
• CPC Class Codes: G06F 9/5072 Grid computing
• CPC Class Codes: G06Q 10/06 Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
• CPC Class Codes: G06Q 10/0631 Resource planning, allocation, distributing or scheduling for enterprises or organisations
• CPC Class Codes: G06Q 50/20 Education
• CPC Class Codes: G09B 5/00 Electrically-operated educational appliances working with questions and answers G09B7/00; simulators G09B9/00; advertising or displaying in general G09F
• CPC Class Codes: G09B 7/00 Electrically-operated teaching apparatus or devices working with questions and answers mechanically operated G09B3/00; computing arrangements G06F
• CPC Class Codes: G09B 7/02 of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Abstract

Systems and methods described herein provide for administering and proctoring of a secure online examination utilizing a ‘cloud’ computing environment. The cloud computing environment provides scalable, shared computing resources such as processing, memory, communications, storage and other hardware and software resources associated with the online examination. The computing resources provided by the cloud computing environment can be provisioned dynamically to match the requirements of the particular computing tasks carried out at the client and/or server level during the administration of the examination.

Claims

1. A method for dynamic examination administration over a network, the method comprising:

receiving over the network at an application server a first request for computing resources of a cloud computer system having a plurality of resource servers, the first request received from a first computing device having a first set of computing resources for executing a routine for an examination;

identifying an amount of supplemental computing resources to provision in order to administer the examination at the first computing device the routine for the examination;

identifying one or more of the resource servers having at least the identified amount of the supplemental computing resources available by analyzing a plurality of performance metrics of each of the resource servers, the performance metrics including a respective amount of available processing power and memory associated with each resource server;

provisioning the identified resource servers to provide the identified amount of supplemental computing resources for use by the first computing device, wherein provisioning the computing resources includes;

creating a virtual instance of the provisioned computing resources in a form of a virtual server, and

sending unique identification information of the created virtual instance to the first computing device, the unique identification information providing the first computing device with access to the provisioned computing resources on the created virtual instance;

monitoring examination data received from the first computing device wherein monitoring the examination data includes identifying when the received examination data is indicative of aberrant behavior by a user of the first computing device in violation of a testing protocol associated with the examination;

dynamically adjusting the amount of supplemental computing resources when the received examination data is identified as being indicative of the aberrant behavior, wherein the amount of the supplemental computing resources is adjusted based on an amount of additional processing power and memory required to process the received examination data that is indicative of the aberrant behavior; and

provisioning the adjusted amount of supplemental computing resources, wherein provisioning the adjusted amount includes shifting computing resources associated with a second computing device.

2. The method of claim 1, wherein the received examination data includes identification data of the user of the first computing device, and wherein monitoring the received examination data includes comparing the identification data to stored identification data of an authorized test taker to identify that the user of the first computing device is the authorized test taker, and further comprising transmitting authorization data to the first computing device to allow the first computing device to execute the routine for the examination based on identifying that the user of the first computing device is the authorized test taker.

3. The method of claim 2, wherein the identification data of the user of the first computing device is generated by comparing a typing sample to a predetermined phrase previously typed and stored for the authorized test taker.

4. The method of claim 3, wherein comparing the typing sample to the predetermined phrase is based on at least one of:

speed of typing associated with the typing sample,

pauses during typing of the typing sample, and

intensity of typing of the typing sample.

5. The method of claim 1, wherein the received examination data includes observation data captured during execution of the routine for the examination on the first computing device, and wherein identifying that the received examination data is indicative of aberrant behavior includes detecting that the observation data includes one or more indicators of the aberrant behavior, and transmitting resulting data that includes the observation data indicating that the aberrant behavior has been detected at the first computing device.

6. The method of claim 5, wherein identifying that the received examination data is indicative of the aberrant behavior includes:

transmitting the observation data to the second computing device; and

receiving an indication from the second computing device validating that the received examination data is indicative of the aberrant behavior.

7. The method of claim 6, wherein the observation data includes visual data of an environment surrounding the first computing device during execution of the routine for the examination, and wherein transmitting the observation data includes transmitting the visual data for display on the second computing device.

8. The method of claim 5, wherein identifying that the received examination data is indicative of the aberrant behavior includes executing an application on the provisioned computing resources to analyze the observation data.

9. The method of claim 5, further comprising transmitting intervention data to the second computing device instructing the second computing device to intervene in the routine for the examination executing on the first computing device to resolve the aberrant behavior.

10. The method of claim 9, wherein the intervention data includes a modification request that an environment surrounding the first computing device be modified to resolve the aberrant behavior.

11. The method of claim 9, wherein the intervention data includes a termination request that the routine for the examination at the first computing device is no longer allowed to continue being executed.

12. The method of claim 1, wherein the testing protocol includes securing the first computing device during execution of the routine for the examination, wherein the received examination data includes exam registration data from the first computing device for an associated test taker, and wherein monitoring the received examination data includes:

analyzing the exam registration data to identify the testing protocol associated with the examination; and

transmitting the testing protocol to the first computing device, the testing protocol including security limitations applied to the first computing device during execution of the routine for the examination.

13. The method of claim 12, wherein the testing protocol further includes questions associated with the examination, the questions to be provided during execution of the routine for the examination, and further comprising:

receiving answers to the questions from the first computing device; and

analyzing the answers using the provisioned computing resources to determine a competence of the test taker.

14. The method of claim 12, wherein applying the security limitation to the first computing device includes dynamically disabling a functionality of the first computing device during execution of the routine for the examination.

15. The method of claim 14, further comprising dynamically unlocking the disabled functionality during execution of the routine for the examination.

16. The method of claim 1, wherein identifying that the received examination data is indicative of the aberrant behavior further includes generating a termination request at the virtual server, the termination request specifying that execution of the routine for the examination at the first computing device to be automatically terminated.

17. The method of claim 1, wherein provisioning the adjusted amount of supplemental computing resources comprises real-time shifting of current computing resources among the plurality of resource servers associated with other examinations.

18. The method of claim 1, wherein the computing resources provisioned in the virtual instance are specific to the first amount of computing resources available at the first computing device, and wherein a different amount of supplemental computing resources is provisioned in a different virtual instance for a different computing device having a different amount of computing resources for executing the same routine for the examination.

19. The method of claim 1, wherein the provisioned computing resources supplement a local application of the first computing device.

20. The method of claim 1, wherein the provisioned computing resources replace a local application of the first computing device.

21. The method of claim 1, further comprising analyzing the performance metrics, wherein provisioning of a new request is based on the analyzed performance metrics.

22. The method of claim 1, wherein performance of the resource servers change over time, and further comprising adjusting the provisioned resources in real-time based on the change.

23. The method of claim 1, wherein identifying the supplemental amount of the computing resources comprises analyzing the first request to identify one or more types of the computing resources and a respective amount of the identified types of computing resources.

24. The method of claim 23, wherein provisioning the identified resource servers includes providing the identified resource servers with configuration data based on the analysis of the first request.

25. The method of claim 24, further comprising creating the virtual instance of the provisioned computing resources based on the configuration data.

26. A non-transitory computer-readable storage medium having embodied thereon a program, the program being executable by a processor to perform a method for administration of an examination over a network, the method comprising:

receiving over the network at an application server a first request for computing resources of a cloud computer system having a plurality of resource servers, the first request received from a first computing device having a first set of computing resources for executing a routine for an examination;

identifying an amount of supplemental computing resources to provision in order to administer the examination at the first computing device the routine for the examination;

identifying one or more of the resource servers having at least the identified amount of the supplemental computing resources available by analyzing a plurality of performance metrics of each of the resource servers, the performance metrics including a respective amount of available processing power and memory associated with each resource server;

provisioning the identified resource servers to provide the identified amount of supplemental computing resources for use by the first computing device, wherein provisioning the computing resources includes;

creating a virtual instance of the provisioned computing resources in a form of a virtual server, and

sending unique identification information of the created virtual instance to the first computing device, the unique identification information providing the first computing device with access to the provisioned computing resources on the created virtual instance;

monitoring examination data received from the first computing device wherein monitoring the examination data includes identifying when the received examination data is indicative of aberrant behavior by a user of the first computing device in violation of a testing protocol associated with the examination;

dynamically adjusting the amount of supplemental computing resources when the received examination data is identified as being indicative of the aberrant behavior, wherein the amount of the supplemental computing resources is adjusted based on an amount of additional processing power and memory required to process the received examination data that is indicative of the aberrant behavior; and

provisioning the adjusted amount of supplemental computing resources, wherein provisioning the adjusted amount includes shifting computing resources associated with a second computing device.