System or process to detect, discriminate, aggregate, track, and rank safety related information in a collaborative workspace

• US 10,325,485 B1
• Filed: 09/11/2018
• Issued: 06/18/2019
• Est. Priority Date: 09/11/2018
• Status: Active Grant

• Current Assignee: Rockwell Automation Technologies Incorporated (Rockwell Automation, Inc.)
• Original Assignee: Rockwell Automation Technologies Incorporated (Rockwell Automation, Inc.)
• Inventors: Schuster, George Karl
• Primary Examiner(s): Barakat, Mohamed
• Application Number:
  US16/127,629
• Time in Patent Office: 280 Days
• CPC Class Codes: F16P 3/142 using image capturing devices
• CPC Class Codes: F16P 3/147 using electro-magnetic technology, e.g. tags or radar
• CPC Class Codes: G01S 17/88 Lidar systems specially adapted for specific applications
• CPC Class Codes: G01S 7/497 Means for monitoring or calibrating
• CPC Class Codes: G05B 19/4061 Avoiding collision or forbidden zones
• CPC Class Codes: G05B 2219/49143 Obstacle, collision avoiding control, move so that no collision occurs
• CPC Class Codes: G05B 2219/49157 Limitation, collision, interference, forbidden zones, avoid obstacles
• CPC Class Codes: G05B 2219/50193 Safety in general
• CPC Class Codes: G08B 21/02 Alarms for ensuring the safety of persons
• CPC Class Codes: G08B 21/187 Machine fault alarms
• CPC Class Codes: G08B 31/00 Predictive alarm systems characterised by extrapolation or other computation using updated historic data

Abstract

An industrial safety system implements predictive method of detecting possible hazardous interactions between humans and machines or other hazardous entities. The system receives sensor data indicative of the locations of people and hazardous entities within the plant environment. Sensor data corresponding to each detected entity is aggregated, ranked, and weighed based on determined reliabilities of the respective data sources. The resulting ranked and weighed location information is tracked and analyzed by the system to determine current and predicted trajectories of the detected objects, and the system analyzes these trajectories to determine a probability of a future hazardous interaction between a human and a hazardous entity. In response to determining that a likelihood of a future hazardous interaction between a person and a machine exists, the system delivers a control output that places the machine in a safe state or a notification output warning of the hazard.

Claims

1. An industrial safety system, comprising:

a memory that stores executable components;

a processor, operatively coupled to the memory, that executes the executable components, the executable components comprising;

an input device interface component configured to receive sensor data from industrial sensor devices, wherein the sensor data represents monitored characteristics of an industrial area;

an object detection component configured to identify, based on analysis of the sensor data, entities that are present within the industrial area, and to classify the entities according to two or more classifications, wherein the two or more classifications comprise at least a human entity and a hazardous entity;

a weighing component configured to, for each subset of the sensor data corresponding to an entity of the entities, apply a weight value to the subset of the sensor data indicative of a determined reliability of the subset of the sensor data to yield weighed sensor data;

a vector generation component configured to, for each entity of the entities,

aggregate subsets of the weighed sensor data corresponding to the entity to determine a measured trajectory of the entity within the industrial area,

predict one or more future trajectories of the entity based on analysis of the measured trajectory and one or more constraints of the entity defined by characteristic constraint data, and

combine the measured trajectory and the one or more future trajectories to yield vector data for the entity;

a hazard analysis component configured to, based on analysis of first vector data corresponding to a first entity of the entities and second vector data corresponding to a second entity of the entities, predict a future hazardous interaction between the first entity and the second entity; and

an industrial device interface component configured to, in response to a determination that the future hazardous interaction satisfies a defined criterion, generate a control output signal directed to a device in the industrial area, wherein the control output signal is configured to at least one of notify at least one of the first entity or the second entity of the hazardous interaction or to modify a controlled behavior of at least one of the first entity or the second entity to mitigate the hazardous interaction.

2. The industrial safety system of claim 1, wherein the sensor devices comprise at least one of a two-dimensional imaging sensor, a three-dimensional camera, a laser scanner, a telemetry device, a photo-electric sensor, or a proximity switch.

3. The industrial safety system of claim 1, wherein the weighing component is configured to determine the weight value based on at least one of a measured atmospheric condition in proximity to an industrial sensor device, of the industrial sensor devices, that generated the subset of the sensor data;

a determination of whether a measured value represented by the subset of the sensor data is near an extreme end of a measurable range of the industrial sensor device;

a type of the industrial sensing device;

a measured level of reflectivity of the entity, or a determination of whether the industrial sensor device is in a fault state.

4. The industrial safety system of claim 1, wherein the hazardous entity is at least one of an industrial machine, a portion of an industrial machine, a vehicle, a manufactured product, a toxic gas leak, a radiation leak, an explosion, a fire, or a hazardous noise level.

5. The industrial safety system of claim 1, wherein the one or more constraints of the entity comprise at least one of a speed limit of the entity, an acceleration limit of the entity, a deceleration limit of the entity, a weight of the entity, an operating mode of the entity, a diffusion rate of the entity, an inertia of the entity, or a coefficient of friction of the entity.

6. The industrial safety system of claim 1, wherein

the hazard analysis component is further configured to assign a classification to the future hazardous interaction as a function of one or more characteristics of the future hazardous interaction,

the one or more characteristics comprise at least one of a severity of the future hazardous interaction, a time of closest approach of the future hazardous interaction, or a likelihood that the future hazardous interaction will occur, and

the industrial device interface component is further configured to generate the control output signal and select the device for receipt of the control output signal based on the classification of the future hazardous interaction.

7. The industrial safety system of claim 1, wherein the industrial device interface component is configured to generate the control output signal to cause the device to at least one of disconnect power from the hazardous entity, place the hazardous entity in a slow operating mode, alter a trajectory of the hazardous entity, or issue an audible warning or a visual warning.

8. The industrial safety system of claim 1, further comprising a visualization component configured to render graphical representations of the vector data on a client device.

9. The industrial safety system of claim 8, wherein the visualization component is further configured to

generate a ranking of multiple future hazardous interactions, including the future hazardous interactions, as a function of one or more characteristics of the future hazardous interactions, wherein the one or more characteristics comprise at least one of severities of the future hazardous interactions, times of closest approach of the future hazardous interactions, or likelihoods that the future hazardous interactions will occur, and

color-code the graphical representations of the vector data based on the ranking.

10. A method for mitigating predicted industrial hazards, comprising:

receiving, by a system comprising a processor, sensor data from industrial sensor devices that monitor for presence of entities within an industrial area;

identifying, by the system based on an analysis of the sensor data, entities that are present within the industrial area;

classifying, by the system, the entities according to two or more classifications, wherein the two or more classifications comprise at least a human entity and a hazardous entity;

applying, by the system for each subset of the sensor data corresponding to an entity of the entities, a weight value to the subset of the sensor data representing a determined accuracy of the subset of the sensor data to yield weighed sensor data;

for each entity of the entities;

determining, by the system, a measured trajectory of the entity within the industrial area based on an aggregation of subsets of the weighted sensor data corresponding to the entity;

predicting, by the system, one or more future trajectories of the entity based on analysis of the measured trajectory and one or more constraints of the entity defined by characteristic constraint data; and

combining, by the system, the measured trajectory and the one or more future trajectories to yield vector data for the entity;

predicting, by the system, a future hazardous interaction between a first entity of the entities and a second entity of the entities based on analysis of first vector data corresponding to the first entity and second vector data corresponding to the second entity; and

in response to determining that the future hazardous interaction satisfies a defined criterion, generating, by the system, a control output signal directed to a device in the industrial area,

wherein the control output signal is configured to at least one of notify at least one of the first entity or the second entity of the hazardous interaction or to initiate a change of a controlled behavior of at least one of the first entity or the second entity to mitigate the hazardous interaction.

11. The method of claim 10, wherein the receiving comprises receiving the sensor data from at least one of a two-dimensional imaging sensor, a three-dimensional camera, a laser scanner, a telemetry device, a photo-electric sensor, or a proximity switch.

12. The method of claim 10, wherein the applying the weight value comprises determining the weight value based on at least one of a measured atmospheric condition in proximity to an industrial sensor device, of the industrial sensor devices, that generated the subset of the sensor data;

a determination of whether a measured value represented by the subset of the sensor data is near an extreme end of a measurable range of the industrial sensor device;

or a determination of whether the industrial sensor device is in a fault state.

13. The method of claim 10, wherein the hazardous entity is at least one of, an industrial machine, a portion of an industrial machine, a vehicle, a manufactured product, a toxic gas leak, a radiation leak, an explosion, a fire, or a hazardous noise level.

14. The method of claim 10, wherein the predicting the one or more future trajectories of the entity comprises predicting the one or more future trajectories based on, as the one or more constraints, at least one of a speed limit of the entity, an acceleration limit of the entity, a deceleration limit of the entity, a weight of the entity, an operating mode of the entity, a diffusion rate of the entity, an inertia of the entity, or a coefficient of friction of the entity.

15. The method of claim 10, further comprising:

assigning, by the system, a classification to the future hazardous interaction based on one or more characteristics of the future hazardous interaction, wherein the one or more characteristics comprise at least one of a severity of the future hazardous interaction, a time of closest approach of the future hazardous interaction, or a likelihood that the future hazardous interaction will occur,

wherein the generating the control output signal comprises generating the control output signal based on the classification of the future hazardous interaction.

16. The method of claim 10, wherein the generating the control output signal comprises generating the control output signal to cause the device to at least one of disconnect power from the hazardous entity, place the hazardous entity in a slow operating mode, alter a trajectory of the hazardous entity, or issue an audible warning or a visual warning.

17. The method of claim 10, further comprising rendering, by the system, graphical representations of the vector data for the entities on a client device.

18. A non-transitory computer-readable medium having stored thereon instructions that, in response to execution, cause a system comprising a processor to perform operations, the operations comprising:

receiving, from industrial sensor devices, sensor data representing monitored characteristics of an industrial area;

identifying, based on an analysis of the sensor data, entities within the industrial area;

classifying the entities according to at least two classifications, wherein the at least two classifications comprise at least a human entity and a hazardous entity;

for each subset of the sensor data corresponding to an entity of the entities, assigning a weight value to the subset of the sensor data to yield weighed sensor data, the weight value representing a determined reliability of the subset of the sensor data;

for each entity of the entities;

aggregating subsets of the weight sensor data corresponding to the entity to yield a measured trajectory of the entity within the industrial area;

predicting one or more future trajectories of the entity based on analysis of the measured trajectory and one or more constraints of the entity defined by characteristic constraint data; and

combining the measured trajectory and the one or more future trajectories to yield vector data for the entity;

predicting a future hazardous interaction between a first entity of the entities and a second entity of the entities based on analysis of first vector data corresponding to the first entity and second vector data corresponding to the second entity; and

in response to determining that the future hazardous interaction satisfies a defined criterion, generating a control output signal directed to a device within the industrial area,

wherein the control output signal is configured to at least one of notify at least one of the first entity or the second entity of the hazardous interaction or to alter a controlled behavior of the at least one of the first entity or the second entity to mitigate the hazardous interaction.

19. The non-transitory computer-readable medium of claim 18, wherein the assigning the weight value comprises determining the weight value based on at least one of a measured atmospheric condition in proximity to an industrial sensor device, of the industrial sensor devices, that generated the subset of the sensor data;

a determination of whether a measured value represented by the subset of the sensor data is near an extreme end of a measurable range of the industrial sensor device;

or a determination of whether the industrial sensor device is in a fault state.

20. The non-transitory computer-readable medium of claim 18, wherein the operations further comprise:

assigning a classification to the future hazardous interaction based on one or more characteristics of the future hazardous interaction, wherein the one or more characteristics comprise at least one of a severity of the future hazardous interaction, a time of closest approach of the future hazardous interaction, or a likelihood that the future hazardous interaction will occur,

wherein the generating the control output signal comprises generating the control output signal based on the classification of the future hazardous interaction.