Method and system of identifying environment features for use in analyzing asset operation

• US 10,579,961 B2
• Filed: 01/26/2017
• Issued: 03/03/2020
• Est. Priority Date: 01/26/2017
• Status: Active Grant

• Current Assignee: Uptake Technologies, Inc.
• Original Assignee: Uptake Technologies, Inc.
• Inventors: Bruns, Tyler Eilert, Keating, Brian Robert, McElhinney, Adam
• Primary Examiner(s): Obaid, Fateh M
• Application Number:
  US15/416,996
• Publication Number:
  US 20180211204A1
• Time in Patent Office: 1,132 Days
• Field of Search: None
• CPC Class Codes: G06F 18/28 Determining representative reference patterns, e.g. by averaging or distorting; Generating dictionaries
• CPC Class Codes: G06Q 10/087 Inventory or stock management, e.g. order filling, procurement or balancing against orders

Abstract

Based on an analysis of asset attribute data associated with a plurality of assets, a platform may detect a locality that is a possible instance of a given type of environment, such as a mine or construction site. In response, the platform may obtain image data associated with the detected locality and input that image data into a model that outputs likelihood data indicating a likelihood that any portion of the detected locality comprises a given feature of the given type of environment (e.g., a boundary, navigation route, hazard, etc.), where this model is defined based on training data. Based on the likelihood data, the platform may then generate output data indicating a location of any portion of the detected locality that is likely to comprise the given feature. In turn, the platform may use the output data to simulate asset operation in the detected locality.

Claims

1. A method carried out by a computing system that is communicatively coupled to one or more asset-related data sources, the method comprising:

receiving asset attribute data associated with a plurality of assets, wherein the asset attribute data includes sensor data captured by sensors affixed to assets in the plurality of assets;

based on an analysis of the received asset attribute data, detecting a grouping of related assets in a locality and thereby determining that the locality is a possible instance of a given type of real-world environment in which assets operate;

in response to the determining, obtaining image data associated with the detected locality;

inputting the obtained image data into a model that outputs likelihood data indicating a likelihood that any portion of the detected locality comprises a given feature of the given type of real-world environment, wherein the model was previously defined by applying a machine learning technique to training data that includes respective image data for each of a plurality of known instances of the given type of real-world environment;

based on the likelihood data, generating output data indicating a location of any portion of the detected locality that is likely to comprise the given feature; and

using the output data to simulate asset operation in the detected locality.

2. The method of claim 1, wherein the given type of real-world environment is one of a mining site, a construction site, a manufacturing plant, or a distribution center.

3. The method of claim 1, wherein the given feature is a one of a boundary, a navigation route, a hazard, an asset, or a structure.

4. The method of claim 1, wherein the training data further includes, for the respective image data for each of the plurality of known instances of the given type of real-world environment, label data indicating which one or more portions of the image data comprise the given feature.

5. The method of claim 4, wherein the respective image data for each of the plurality of known instances of the given type of real-world environment comprises an array of pixels, and wherein the label data comprises a plurality of labels that each correspond to a respective subset of the array of pixels.

6. The method of claim 5, wherein the model embodies a relationship between characteristics of pixels and a likelihood that pixels represent the given feature.

7. The method of claim 1, wherein the training data further includes asset attribute data associated with assets located in the known instances of the given type of real-world environment.

8. The method of claim 1, wherein the received data captured by sensors affixed to assets in the plurality of assets comprises location data captured by sensors affixed to assets in the plurality of assets.

9. The method of claim 1, wherein the obtained image data for the detected locality comprises an array of pixels, and wherein the likelihood data comprises a set of likelihood values that each correspond to a subset of the array of pixels.

10. The method of claim 1, further comprising:

transmitting the output data to a computing device to facilitate causing the computing device to display a visualization of the output data.

11. The method of claim 1, wherein the image data for the detected locality comprises first image data, the likelihood data for the detected locality comprises first likelihood data, and the output data for the detected locality comprises first output data, the method further comprising:

obtaining second image data associated with the detected locality, wherein the second image data is captured at a different time than first image data;

inputting the obtained second image data into a model that outputs second likelihood data indicating a likelihood that any portion of the detected locality comprises the given feature of the given type of real-world environment, wherein the model that outputs second likelihood data is defined based on respective training data that includes respective image data for each of the plurality of known instances of the given type of real-world environment;

based on the second likelihood data, generating the second output data indicating a location of any portion of the detected locality that is likely to comprise the given feature;

comparing the second output data with the first output data; and

based on the comparison, identifying a change in at least one instance of the given feature in the detected locality.

12. A computing system comprising:

a network interface configured to facilitate communications over a communication network with one or more data sources;

at least one processor;

a non-transitory computer-readable medium; and

program instructions stored on the non-transitory computer-readable medium that are executable by the at least one processor to cause the computing system to;

receive, via the network interface, asset attribute data associated with a plurality of assets, wherein the asset attribute data includes sensor data captured by sensors affixed to assets in the plurality of assets;

based on an analysis of the received asset attribute data, detect a grouping of related assets in a locality and thereby determine that the locality is a possible instance of a given type of real-world environment;

in response to the determining, obtain, via the network interface, image data associated with the detected locality;

input the obtained image data into a model that outputs likelihood data indicating a likelihood that any portion of the detected locality comprises a given feature of the given type of real-world environment, wherein the model was previously defined by applying a machine learning technique to training data that includes respective image data for each of a plurality of known instances of the given type of real-world environment;

based on the likelihood data, generate output data indicating a location of any portion of the detected locality that is likely to comprise the given feature; and

use the output data to simulate asset operation in the detected locality.

13. The computing system claim 12, wherein the given type of real-world environment is one of a mining site, a construction site, a manufacturing plant, or a distribution center.

14. The computing system of claim 12, wherein the given feature is a one of a boundary, a navigation route, a hazard, an asset, or a structure.

15. The computing system of claim 12, wherein the training data further includes, for the respective image data for each of the plurality of known instances of the given type of real-world environment, label data indicating which one or more portions of the image data comprise the given feature.

16. The computing system of claim 12, wherein the model embodies a relationship between characteristics of pixels and a likelihood that pixels represent the given feature.

17. The computing system of claim 12, wherein the training data further includes asset attribute data associated with assets located in the known instances of the given type of real-world environment.

18. The computing system of claim 12, wherein the obtained image data for the detected locality comprises an array of pixels, and wherein the likelihood data comprises a set of likelihood values that each correspond to a subset of the array of pixels.

19. The computing system of claim 12, further comprising:

transmitting the output data to a computing device to facilitate causing the computing device to display a visualization of the output data.

20. A non-transitory computer-readable medium having program instructions stored thereon that are executable to cause a computing device to:

receive asset attribute data associated with a plurality of assets, wherein the asset attribute data includes sensor data captured by sensors affixed to assets in the plurality of assets;

based on an analysis of the received asset attribute data, detect a grouping of related assets in a locality and thereby determine that the locality is a possible instance of a given type of real-world environment;

in response to the determining, obtain image data associated with the detected locality;

input the obtained image data into a model that outputs likelihood data indicating a likelihood that any portion of the detected locality comprises a given feature of the given type of real-world environment, wherein the model was previously defined by applying a machine learning technique to training data that includes respective image data for each of a plurality of known instances of the given type of real-world environment;

based on the likelihood data, generate output data indicating a location of any portion of the detected locality that is likely to comprise the given feature; and

use the output data to simulate asset operation in the detected locality.