Techniques and systems for detecting anomalous operational data

• US 9,785,495 B1
• Filed: 12/14/2015
• Issued: 10/10/2017
• Est. Priority Date: 12/14/2015
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Lazier, Colin Laird, Argenti, Marco
• Primary Examiner(s): Ehne, Charles
• Application Number:
  US14/968,701
• Time in Patent Office: 666 Days
• CPC Class Codes: G06F 11/0709 in a distributed system consisting of a plurality of standalone computer nodes, e.g. clusters, client-server systems
• CPC Class Codes: G06F 11/0754 by exceeding limits
• CPC Class Codes: G06F 11/10 Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's

Abstract

A system stores data, such as sensor data or other operational data, on a plurality of storage volumes in a sequence so as to allow for interpolations or other approximations of the data using a subset of the storage volumes in response to a request for information regarding that data. For example, a plurality of devices connect to the system to provide operational data, which is then stored in a specified sequence on a specified set of volumes. In response to a request for operational information regarding some or all of the devices, the system reads at least one of the volumes, and approximates the values of the data over a specified period of time. In some embodiments, the data may be buffered prior to storage, and a jitter analyzer determines whether the incoming data is anomalous relative to a baseline, which may be determined using related data sets.

Claims

1. A computer-implemented method, comprising:

under the control of one or more computer systems configured with executable instructions,

receiving, from a plurality of remote devices connected to the one or more computer systems, sensor data related to operation of the plurality of remote devices;

processing the received sensor data by at least;

buffering the received sensor data for a period of time;

determining, based at least in part on baseline values associated with the received sensor data, an anomaly score that indicates a level of abnormality for the buffered sensor data for the period of time; and

if the anomaly score indicates that the level of abnormality is higher than a determined threshold;

apportioning the buffered data to a volume of a plurality of volumes such that more than one volume of the plurality of volumes individually contains a portion of the buffered data that indicates the level of abnormality; and

alerting an entity associated with the plurality of remote devices that the level of abnormality is higher than the determined threshold.

2. The computer-implemented method of claim 1, wherein the buffered data is apportioned to the plurality of volumes in a sequence, such that at least one volume of the plurality of volumes includes at least two subsets of the received sensor data, each subset of the sensor data being received by the one or more computer systems at a different time.

3. The computer-implemented method of claim 2, wherein the determined threshold is derived from a baseline that is dynamically determined via operation of one or more of the plurality of remote devices.

4. The computer-implemented method of claim 1, wherein the plurality of devices are connected to the one or more computer systems via the Internet.

5. A system, comprising:

at least one computing device configured to implement one or more services, wherein the one or more services are configured to;

receive data related to operation of a plurality of remote devices;

process the received data by at least;

buffering the received data for a period of time;

determining a level of abnormality for the buffered data for the period of time; and

if the level of abnormality is outside of a determined range;

store at least a portion of the received data and anomaly information in response to the level of abnormality, the stored portion of data associated with the level of abnormality and determined based on the level of abnormality; and

alert an entity associated with the plurality of remote devices that the level of abnormality is outside of the determined range.

6. The system of claim 5, wherein the one or more services are further configured to determine the level of abnormality by generating an anomaly score, the anomaly score being normalized against a baseline value associated with the received data.

7. The system of claim 5, wherein the data includes operational data periodically captured by the plurality of devices.

8. The system of claim 5, wherein the plurality of devices connects to the at least one computing device over a network using a protocol that is selected from the group consisting of:

Z-wave, Zigbee, Thread, Bluetooth, Insteon, 6LoWPAN, InternetProtocol, Highway Addressable Remote Transducer Protocol (HART), supervisory control and data acquisition (SCADA), fieldbus, and sneakernet.

9. The system of claim 5, wherein the one or more services are further configured to process the received data by storing the buffered data on a plurality of volumes.

10. The system of claim 9, wherein the processed data is stored on volumes associated with identity shards.

11. The system of claim 5, wherein the one or more services are further configured to receive the data in response to a periodic request by the device to store the data via the one or more services.

12. The system of claim 5, wherein the one or more services are further configured to respond to requests for the data by at least providing the stored anomaly information.

13. A non-transitory computer-readable storage medium having stored thereon executable instructions that, when executed by one or more processors of a computer system, cause the computer system to at least:

buffer incoming operational data for a period of time;

calculate an anomaly score for the buffered operational data that enumerates a level of abnormality for the incoming operational data over the period of time; and

if the anomaly score deviates from a determined range, store at least a subset of the buffered operational data on a sequence of volumes such that at least two volumes in the sequence individually capture information related to an anomaly associated with the anomaly score, the stored subset of the buffered operational data being determined based on the value of the anomaly score.

14. The non-transitory computer-readable storage medium of claim 13, wherein the instructions further comprise instructions that, when executed by the one or more processors, cause the computer system to store the incoming operational data on the sequence of volumes after the period of time.

15. The non-transitory computer-readable storage medium of claim 13, wherein the instructions further comprise instructions that, when executed by the one or more processors, cause the computer system to cause calculation of the anomaly score of the operational data in response to receiving requests from the devices to store the operational data.

16. The non-transitory computer-readable storage medium of claim 13, wherein the instructions further comprise instructions that, when executed by the one or more processors, cause the computer system to generate the anomaly score by using interpolated data interpolated from the incoming operational data.

17. The non-transitory computer-readable storage medium of claim 13, wherein the instructions further comprise instructions that, when executed by the one or more processors, cause the computer system to provide the anomaly score to an entity associated with the operational data if the anomaly score is above the predetermined level.

18. The non-transitory computer-readable storage medium of claim 16, wherein the instructions further comprise instructions that, when executed by the one or more processors, cause the computer system to refine the anomaly score for the operational data by using at least a different set of operational data from a different period of time.

19. The non-transitory computer-readable storage medium of claim 13, wherein the operational data is received from a plurality of devices connected to the computer system over a network that uses a protocol selected from the group consisting of:

Z-wave, Zigbee, Thread, Bluetooth, Insteon, 6LoWPAN, InternetProtocol, Highway Addressable Remote Transducer Protocol (HART), supervisory control and data acquisition (SCADA), fieldbus, and sneakernet.

20. The non-transitory computer-readable storage medium of claim 19, wherein the operational data includes sensor data associated with the plurality of devices.