Edge intelligence platform, and internet of things sensor streams system

• US 10,007,513 B2
• Filed: 08/29/2016
• Issued: 06/26/2018
• Est. Priority Date: 08/27/2015
• Status: Active Grant

• Current Assignee: Tyco Fire & Security GmbH (Johnson Controls International plc)
• Original Assignee: Foghorn Systems, Inc. (Johnson Controls International plc)
• Inventors: Malladi, Sastry K M, Ravi, Thirumalai Muppur, Reddy, Mohan Komalla, Raghavendra, Kamesh
• Primary Examiner(s): Wu, Daxin
• Application Number:
  US15/250,720
• Publication Number:
  US 20170060574A1
• Time in Patent Office: 666 Days
• Field of Search: None
• CPC Class Codes: G06F 8/70 Software maintenance or management
• CPC Class Codes: G06F 9/542 Event management; Broadcasting; Multicasting; Notifications
• CPC Class Codes: H04L 67/12 specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks
• CPC Class Codes: H04W 4/38 for collecting sensor information
• CPC Class Codes: H04W 4/70 Services for machine-to-machine communication [M2M] or machine type communication [MTC]
• CPC Class Codes: H04W 4/80 Services using short range communication, e.g. near-field communication [NFC], radio-frequency identification [RFID] or low energy communication

Abstract

A method for enabling intelligence at the edge. Features include: triggering by sensor data in a software layer hosted on either a gateway device or an embedded system. Software layer is connected to a local-area network. A repository of services, applications, and data processing engines is made accessible by the software layer. Matching the sensor data with semantic descriptions of occurrence of specific conditions through an expression language made available by the software layer. Automatic discovery of pattern events by continuously executing expressions. Intelligently composing services and applications across the gateway device and embedded systems across the network managed by the software layer for chaining applications and analytics expressions. Optimizing the layout of the applications and analytics based on resource availability. Monitoring the health of the software layer. Storing of raw sensor data or results of expressions in a local time-series database or cloud storage. Services and components can be containerized to ensure smooth running in any gateway environment.

Claims

1. A method comprising:

receiving a sensor data stream from a first physical sensor over a first network connection type at a data ingestion agent of an edge platform system;

from the data ingestion agent delivering the ingested stream data to a data bus of the edge platform system;

enriching the ingested stream data in real time through a data enrichment component of the edge platform system by performing an enrichment action on the ingested stream data, wherein the enrichment action comprises one or more of decoding the ingested stream data, supplementing the ingested stream data with metadata, and normalizing the ingested stream data;

making the enriched ingested stream data available on the data bus;

processing the enriched ingested stream data at an analytics engine of the edge platform system that is subscribing to data from the data bus, wherein the processing comprises applying to the enriched ingested stream data in real time a first function specified by selected analytic expressions of an expression language and generating in real time a stream of data comprising intelligence information, wherein the first function comprises at least one of a transform, pattern detection, dynamic calibration, signal processing, math expression, data compaction, data analytic, data aggregation, rule application, alert, and service invocation;

publishing the intelligence information on the data bus, wherein the publishing comprises storing the intelligence information on the edge platform system in a rolling time-series database indexed by time for use by applications executable on the edge platform system; and

determining based on the intelligence information whether to take selected action in the local network on a hardware device being monitored by the first physical sensor and whether to transmit at least a portion of the intelligence information to the remote network for additional processing, wherein the selected action comprises sending an alert to an operator on making a corrective action on a hardware device being monitored by the first physical sensor.

2. The method of claim 1 comprising transferring at least a portion of the intelligence information to the remote network over a second network connection type and storing in a cloud storage connected to the remote network.

3. The method of claim 2 wherein the second network connection type is different from the first network connection type, and the second network connection type has a lower bandwidth connection than the first network connection type to the sensor data stream data.

4. The method of claim 2 wherein the receiving over a first network connection type comprises receiving according to at least one of a MQTT, OPC Unified Architecture, or Modbus protocol.

5. The method of claim 4 wherein the transferring over a second network connection type comprises transferring according to at least one of an TCP/IP, Hypertext Transfer Protocol (HTTP), Web Sockets, WebHDFS, Apache Kafk, or Internet protocol.

6. The method of claim 1 wherein the data ingestion agent accesses the sensor stream data over a first network connection type via a push protocol.

7. The method of claim 1 wherein the data ingestion agent accesses the sensor stream data over a first network connection type via a pull protocol.

8. The method of claim 1 comprising:

providing a software development kit that allows a user to develop applications for the edge platform system; and

by way of the software development kit, developing a first containerized application that can access and process the intelligence information available from the data bus.

9. The method of claim 8 comprising:

by way of the software development kit, developing a second containerized application that can access and process the intelligence information stored in the time-series database.

10. The method of claim 9 comprising:

providing an application store of the edge platform system, wherein the first and second containerized applications are available for other users to access through the application store.

11. The method of claim 1 wherein the processing comprises:

using the expression language, creating a first virtual sensor having a first input and a first output, wherein the first input is coupled to an output of the first physical sensor,

the first virtual sensor receives a first sensor data stream from the first physical sensor at the first input, and at the first output the first virtual sensor outputs a second sensor data stream that is the first sensor data stream operated on according to the first function.

12. The method of claim 1 comprising:

using the expression language, creating a first virtual sensor having a first input, a second input, and a first output, wherein the first input is coupled to an output of the first physical sensor, the second input is coupled to an output of a second physical sensor,

the first virtual sensor receives a first sensor data stream from the first physical sensor and second sensor data stream from the second physical sensor, and at the first output the first virtual sensor outputs a third sensor data stream that is the first and second sensor data streams operated on according to the first function.

18. The method of claim 12 wherein the determining whether to transmit at least a portion of the intelligence information to the remote network for additional processing comprises determining whether to aggregate the intelligence information with other information before transmitting to the remote network.

19. The method of claim 12 wherein the enriching the ingested stream data in real time through a data enrichment component of the edge platform system comprises performing an enrichment action on the ingested stream data, wherein the enrichment action comprises one or more of decoding the ingested stream data, supplementing the ingested stream data with metadata, and normalizing the ingested stream data.

13. A method for improving the processing of sensor data in a distributed network including a local network with a plurality of network devices and a plurality of physical sensors and a remote network comprising:

receiving a first data stream from a first physical sensor in the local network over a first network connection type at a data ingestion agent of an edge platform system connected to the local network between the first physical sensor and the remote network;

from the data ingestion agent delivering the ingested first data stream to a data bus of the edge platform system;

enriching the ingested first data stream in real time through a data enrichment component of the edge platform system and making the enriched ingested first data stream available on the data bus;

processing the enriched ingested first data stream in real time at an analytics engine of the edge platform system that is subscribing to data from the data bus without first transmitting the first data stream to the remote network, wherein the processing comprises executing analytic expressions provided in an expression language to generate intelligence information from the enriched ingested first data stream;

making the intelligence information available on the data bus; and

determining based on the intelligence information whether to transmit at least a portion of the intelligence information to the remote network for additional processing and whether to take selected action in the local network with respect to an operation in the local network monitored by the first physical sensor without awaiting additional processing of the intelligence data at the remote network, wherein the selected action comprises at least one of generating an alert and altering the operation.

14. The method of claim 13 comprising:

storing the intelligence information on the edge platform system in a rolling time-series database indexed by time for use by applications executable on the edge platform.

15. The method of claim 14 comprising:

providing a software development kit that allows a user to develop applications executable on the edge platform system; and

by way of the software development kit, developing a first containerized application that can access and process the intelligence information available from the data bus.

16. The method of claim 15 comprising:

by way of the software development kit, developing a second containerized application that can access and process the intelligence information stored in the time-series database.

17. The method of claim 16 comprising:

providing an application store of the edge platform system, wherein the first and second containerized applications are available for users to access through the application store.

20. The method of claim 13 wherein the processing the enriched ingested stream data comprises applying to the enriched ingested stream data in real time a first function specified by selected analytic expressions of an expression language and generating in real time a stream of data comprising intelligence information, wherein the first function comprises at least one of a transform, pattern detection, dynamic calibration, signal processing, math expression, data compaction, data analytic, data aggregation, rule application, alert, and service invocation.

21. The method of claim 20 wherein the processing the enriched ingested first data stream comprises:

using the expression language, creating a first virtual sensor having a first input and a first output, wherein the first input is coupled to an output of the first physical sensor,

the first virtual sensor receives a first sensor data stream from the first physical sensor at the first input, and at the first output the first virtual sensor outputs a second sensor data stream that is the first sensor data stream operated on according to the first function.

22. A method for improving the processing of sensor data in a distributed network including a local network with a plurality of network devices and a plurality of physical sensors each generating a data stream on a topic, and a remote network comprising:

receiving a first data stream from a first physical sensor in the local network over a first network connection type at a data ingestion agent of an edge platform system connected to the local network between the first physical sensor and the remote network;

from the data ingestion agent delivering the ingested first data stream to a data bus of the edge platform system;

enriching the ingested first data stream in real time through a data enrichment component of the edge platform system and making the enriched ingested first data stream and information identifying the topic of the first data stream available on the data bus;

storing on the edge platform system a plurality of sets of analytic expressions provided in an expression language, wherein each set of analytic expressions defines a function that may be carried out on the enriched ingested first data stream from the data bus;

processing the enriched ingested stream data in real time at an analytics engine of the edge platform system that is subscribing to data from the data bus without first transmitting the stream data to the remote network, wherein the processing comprises automatically selecting based on the topic of the first data stream a first set of analytic expressions defining a first function from the plurality of sets of analytic expressions and executing the selected set of analytic expressions to generate intelligence information from the enriched ingested first data stream;

making the intelligence information available on the data bus; and

determining based on the intelligence information whether to transmit at least a portion of the intelligence information to the remote network for additional processing and whether to take selected action in the local network with respect to an operation in the local network monitored by the first physical sensor and involving a network device without awaiting additional processing of the intelligence data at the remote network, wherein the selected action comprises at least one of generating an alert and altering the operation.

23. The method of claim 22 comprising:

storing the intelligence information on the edge platform system in a rolling time-series database indexed by time for use by applications executable on the edge platform.

24. The method of claim 23 comprising:

providing a software development kit that allows a user to develop applications executable on the edge platform system; and

by way of the software development kit, developing a first containerized application that can access and process the intelligence information available from the data bus.

25. The method of claim 24 comprising:

by way of the software development kit, developing a second containerized application that can access and process the intelligence information stored in the time-series database.

26. The method of claim 25 comprising:

providing an application store of the edge platform system, wherein the first and second containerized applications are available for users to access through the application store.

27. The method of claim 22 wherein the determining whether to transmit at least a portion of the intelligence information to the remote network for additional processing comprises determining whether to aggregate the intelligence information with other information before transmitting to the remote network.

28. The method of claim 22 wherein the enriching the ingested stream data in real time through a data enrichment component of the edge platform system comprises performing an enrichment action on the ingested stream data, wherein the enrichment action comprises one or more of decoding the ingested stream data, supplementing the ingested stream data with metadata, and normalizing the ingested stream data.

29. The method of claim 22 wherein the first function comprises at least one of a transform, pattern detection, dynamic calibration, signal processing, math expression, data compaction, data analytic, data aggregation, rule application, alert, and service invocation.

30. The method of claim 29 wherein the processing the enriched ingested first data stream comprises:

using the expression language, creating a first virtual sensor having a first input and a first output, wherein the first input is coupled to an output of the first physical sensor,

the first virtual sensor receives a first sensor data stream from the first physical sensor at the first input, and at the first output the first virtual sensor outputs a second sensor data stream that is the first sensor data stream operated on according to the first function.

31. A method comprising:

receiving a first stream data from a first physical sensor over a first network connection type at a data ingestion agent of an edge platform system;

from the data ingestion agent, delivering ingested first stream data to a data bus of the edge platform system;

enriching the ingested data in real time through a data enrichment component of the edge platform system and making the ingested data available on the data bus, wherein the enrichment action comprises one or more of decoding the ingested stream data, supplementing the ingested data with metadata, and normalizing the ingested stream data;

providing an analytics engine of the edge platform system, coupled to the data bus, wherein the analytics engine processes code provided in an expression language;

using the expression language, creating a first virtual sensor having a first input and a first output, wherein the first input is coupled to an output of the first physical sensor and the first output is a first function of the first input, and the first function is specified in the expression language, the first input of the first virtual sensor receives the first stream data from the first physical sensor, and the first output of the first virtual sensor outputs second stream data that comprises the first stream data operated on according to the first function, and the first function comprises at least one of a transform, pattern detection, dynamic calibration, signal processing, math expression, data compaction, data analytic, data aggregation, rule application, alert, and service invocation, and without having passed through and enriched by the data enrichment component, the second stream data is made available on the data bus; and

processing the ingested stream data from the first stream data of the first physical sensor and the second stream data of the first virtual sensor at the analytics engine that is subscribing to data from the data bus wherein the processing comprises executing analytic expressions provided in the expression language to generate intelligence information from the ingested stream data and second stream data; and

publishing the intelligence information from the ingested stream data of the first physical sensor and second stream data of the first virtual sensor on the data bus.

32. The method of claim 31 comprising:

building via machine learning at least one model for intelligent pattern matching of a data stream; and

using the model for intelligent pattern matching to segment the first data stream, wherein the segmented first data stream is used for the first input of the first virtual sensor.

33. The method of claim 31 comprising:

storing second stream data of the first virtual sensor in a time-series database;

analyzing the second stream data stored in the time-series database to match one or more patterns in the data; and

identifying a first pattern match in second stream data stored in the time-series database;

based on the first pattern match being identified in the second stream data stored in the time-series database, altering operation of a first hardware device that is being monitored by the first physical sensor.