Fine grained network management to edge device features

• US 10,594,549 B2
• Filed: 05/14/2019
• Issued: 03/17/2020
• Est. Priority Date: 05/18/2018
• Status: Active Grant

• Current Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Original Assignee: Nant Holdings IP, LLC (NantWorks, LLC)
• Inventors: Soon-Shiong, Patrick, Wiacek, John
• Primary Examiner(s): Nguyen, Phuoc H
• Application Number:
  US16/411,441
• Publication Number:
  US 20190356539A1
• Time in Patent Office: 308 Days
• Field of Search: None
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06N 5/022 Knowledge engineering; Knowledge acquisition
• CPC Class Codes: H04L 41/0213 Standardised network management protocols, e.g. simple network management protocol [SNMP]
• CPC Class Codes: H04L 41/069 using logs of notifications; Post-processing of notifications
• CPC Class Codes: H04L 41/0803 Configuration setting
• CPC Class Codes: H04L 41/082 the condition being updates or upgrades of network functionality
• CPC Class Codes: H04L 41/22 comprising specially adapted graphical user interfaces [GUI]
• CPC Class Codes: H04L 43/16 Threshold monitoring
• CPC Class Codes: H04Q 2209/40 using a wireless architecture
• CPC Class Codes: H04Q 2209/70 Arrangements in the main station, i.e. central controller
• CPC Class Codes: H04Q 2209/80 Arrangements in the sub-station, i.e. sensing device
• CPC Class Codes: H04Q 2209/823 where the data is sent when the measured values exceed a threshold, e.g. sending an alarm
• CPC Class Codes: H04Q 9/00 Arrangements in telecontrol or telemetry systems for selectively calling a substation from a main station, in which substation desired apparatus is selected for applying a control signal thereto or for obtaining measured values therefrom
• CPC Class Codes: H04Q 9/02 Automatically-operated arrangements

Abstract

Network management systems and methods are provided. A system is provided that includes an event database that provides a mapping of multimodal sensor data to events of devices coupled to a network, such as operational or behavioral events. A network management engine obtains a set of multimodal sensor data relating to a device, which may include sensor measurement or output data relating to each of multiple device operation or behavior parameters. The engine determines, utilizing the mapping, a match of the set of multimodal sensor data to a specific event associated with the device. Based at least in part on the determined match, the engine causes generation or updating of event data associated with the specific event, the data being accessible by network management software of the network management system.

Claims

1. A network management system, comprising:

one or more processors;

an event database providing a mapping of multimodal sensor data to events of one or more devices coupled to a network, wherein the event database comprises a Management Information Base (MIB), and wherein the MIB is ontologically organized relative to operational or behavioral events and utilizes a defined hierarchical vocabulary relating to the organization structure of the MIB; and

a memory storing software instructions that, when executed by the one or more processors, cause the processor to;

obtain a set of multimodal sensor data relating to a device of the one or more devices;

determine, utilizing the mapping, a match of the set of multimodal sensor data to a first event associated with the device; and

based at least in part on the determined match, cause generation or updating of first event data associated with the first event, the first event data being accessible by network management software of the network management system.

2. The system of claim 1, wherein the events of the one or more devices comprise operational events and the first event is related to at least one of chemical, mechanical and electrical operation of the device.

3. The system of claim 1, wherein an event of the events of the one or more devices is associated with a biological, physiological, electrical, magnetic, electromagnetic, environmental, structural, or weather-related occurrence, condition or phenomenon.

4. The system of claim 1, the events of the one or more devices comprise behavioral events and the first event is related to behavior of an entity interacting with the device.

5. The system of claim 1, wherein the first event data reflects occurrence, continued occurrence or repeated occurrence of the first event.

6. The system of claim 1, wherein the multimodal sensor data comprises multiple outputs or measurements, each relating to a different sensed parameter.

7. The system of claim 1, wherein the multimodal sensor data is stored in the event database.

8. The system of claim 1, wherein the multimodal sensor data is collected, and computationally modified, or filtered, at the device of the one or more devices.

9. The system of claim 1, wherein the multimodal sensor data is collected, and computationally processed or filtered, at the device of the one or more devices prior to wireless transmission over the Internet and prior to storage in the event database.

10. The system of claim 1, wherein the match is determined based on the multimodal sensor data falling within a range or being above a threshold.

11. The system of claim 1, wherein:

the match is determined based on the multimodal sensor data falling within a range or being above a threshold; and

the range or the threshold requires that, for each of multiple single outputs or single measurements of the multimodal sensor data, the single output or the single measurement falls within a specified range or is above a specified threshold, the specified range or the specified threshold relating to the single output or the single measurement.

12. The system of claim 1, wherein:

the multimodal sensor data comprises multiple sensor outputs, each relating to a different sensed parameter; and

only a combination of all of the multiple sensor outputs, and not any single one of the multiple sensor outputs or any subset of the multiple sensor outputs, matches to the first event.

13. The system of claim 1, wherein the one or more processors are further caused to:

cause monitoring for, and detection of, the first event data utilizing the network management software.

14. The system of claim 1, wherein the one or more processors are further caused to:

upon detection of the first event data by the network management software, cause generation of an alert.

15. The system of claim 1, wherein the one or more processors are further caused to:

upon detection of the first event data by the network management software, based at least in part on the on the first event, cause modification of function of the device to better optimize function of the device.

16. The system of claim 1, wherein the event database specifies, in terms of multimodal sensor data, definitions or thresholds relating to events.

17. The system of claim 1, wherein an event of the events of the one or more devices relates to operation or behavior of a device or of a person.

18. The system of claim 1, wherein an event of the events of the one or more devices is associated with a person, animal, plant, or organ.

19. The system of claim 1, wherein an event of the events of the one or more devices is associated with a feature, aspect, element, portion, or condition of a living entity.

20. The system of claim 1, wherein the one or more processors are further caused to:

utilize one or more machine learning techniques in determining the mapping.

21. The system of claim 1, wherein the one or more processors are further caused to:

utilizing event data, determine and store data regarding sensed operation or behavior;

track and store data regarding outcomes resulting from sensed operation or behavior; and

utilize one or more machine learning techniques, determine patterns associating sensed operation or behavior with outcomes.

22. The system of claim 1, wherein the one or more processors are further caused to:

utilize one or more machine learning techniques in;

determining or updating the mapping, comprising utilizing the multimodal sensor data associated with the one or more devices as machine learning training data; and

updating the event database based on the determined or updated mapping.

23. The system of claim 1, wherein the one or more processors are further caused to:

store a log over time of varying multimodal sensor data associated with the device of the one or more devices.

24. The system of claim 1, wherein the one or more processors are further caused to:

store a log over time of varying multimodal sensor data associated with the device of the one or more devices using one or more blockchains.

25. The system of claim 1, wherein the device comprises a battery or power cell and wherein the multimodal sensor data comprises data relating to each of multiple parameters of the battery or power cell.

26. The system of claim 1, wherein the device comprises a component of a battery or power cell and wherein the multimodal sensor data comprises data relating to the component.

27. The system of claim 1, wherein the device comprises a component of a vehicle.

28. The system of claim 1, wherein the device comprises a medical device.

29. The system of claim 1, wherein the first event is associated with a patient or a consumer.

30. The system of claim 1, wherein the one or more processors are further caused to:

generate a user interface dashboard displaying an indication or alert of the first event; and

allowing a user, through the dashboard, to take action based on the first event.

31. The system of claim 1, wherein the network management software utilizes Open Database Connectivity (ODBC) standards.

32. The system of claim 1, wherein the network management software utilizes simple network management protocol (SNMP).

33. The system of claim 1, wherein the network management software utilizes OpenNMS.

34. A network management method, comprising:

maintaining an event database providing a mapping of multimodal sensor data to events of one or more devices coupled to a network, wherein the event database comprises a Management Information Base (MIB), and wherein the MIB is ontologically organized relative to operational or behavioral events and utilizes a defined hierarchical vocabulary relating to the organization structure of the MIB;

obtaining a set of multimodal sensor data relating to a device of the one or more devices;

determining, utilizing the mapping, a match of the set of multimodal sensor data to a first event associated with the device; and

based at least in part on the determined match, causing generation or updating of first event data associated with the first event, the first event data being accessible by network management software of the network management system;

detecting the first event data; and

based at least in part on the detection the first event data, causing a modification of operation of the device.

35. The method of claim 34, wherein causing the modification of the operation of the device is to better optimize operation of the device based at least in part on the first event data.

36. The method of claim 34, wherein the first event is related to at least one of chemical, mechanical or electrical operation of the device.

37. A non-transitory computer readable medium or media containing instructions for executing a method comprising:

maintaining an event database providing a mapping of multimodal sensor data to events of one or more devices coupled to a network, wherein the event database comprises a Management Information Base (MIB), and wherein the MIB is ontologically organized relative to operational or behavioral events and utilizes a defined hierarchical vocabulary relating to the organization structure of the MIB;

obtaining a set of multimodal sensor data relating to a device of the one or more devices;

determining, utilizing the mapping, a match of the set of multimodal sensor data to a first event associated with the device;

based at least in part on the determined match, causing generation or updating of first event data associated with the first event, the first event data being accessible by network management software of the network management system;

detecting the first event data; and

based at least in part on the detection the first event data, causing a modification of operation of the device.

38. The computer readable medium or media of claim 37, wherein the first event is related to at least one of chemical, mechanical or electrical operation of the device.