RECOGNITION-BASED INDUSTRIAL AUTOMATION CONTROL WITH CONFIDENCE-BASED DECISION SUPPORT

• US 20140244036A1
• Filed: 02/27/2013
• Published: 08/28/2014
• Est. Priority Date: 02/27/2013
• Status: Active Grant

• Current Assignee: Rockwell Automation Technologies Incorporated (Rockwell Automation, Inc.)
• Original Assignee: Rockwell Automation Technologies Incorporated (Rockwell Automation, Inc.)
• Inventors: Scott, Steven J., Nguyen, Thong T., Vasko, David A., Gasperi, Michael, Brandt, David D.
• Granted Patent: US 9,498,885 B2
• Time in Patent Office: Days
• US Class Current: 700/253
• CPC Class Codes: B25J 9/1694 characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G05B 19/409 characterised by using manual data input [MDI] or by using control panel, e.g. controlling functions with the panel; characterised by control panel details or by setting parameters G05B19/408, G05B19/4093 take precedence

Abstract

The present disclosure generally relates to a method for performing industrial automation control in an industrial automation system. As such, the method may include detecting, via a sensor system, positions and/or motions of a human. The method may then include determining a possible automation command corresponding to the detected positions and/or motions. After determining the possible automation command, the method may implement a control and/or notification action based upon the detected positions and/or motions.

Claims

1. A method for performing industrial automation control in an industrial automation system, comprising:

detecting, via a sensor system, positions and/or motions of a human;

determining a possible automation command corresponding to the detected positions and/or motions; and

implementing a control and/or notification action based upon the detected positions and/or motions.

2. The method of claim 1, wherein the detected positions and/or motions comprise at least one gesture associated with the determined command.

3. The method of claim 2, wherein the at least one gesture is one of a plurality of possible gestures, each gesture being associated with a determined automation command.

4. The method of claim 1, comprising:

determining a confidence value associated with the detected positions, the detected motions, and/or the determined automation command; and

implementing the control and/or notification action based upon the detected positions and/or motions and the confidence value.

5. The method of claim 4, wherein an operation of an industrial automation device in the industrial automation system is not altered if the confidence value is below a threshold.

6. The method of claim 4, wherein the confidence value is indicative of relative accuracy in the determination of at least one the detected positions and/or motions.

7. The method of claim 4, wherein the confidence value is indicative of relative accuracy of matching of the positions and/or motions to the determined automation command.

8. The method of claim 4, wherein the confidence value is indicative of relative accuracy of matching of the positions and/or motions to an expected pattern of movement.

9. The method of claim 4, wherein implementing the control comprises sending the determined automation command to at least one industrial automation command when the confidence value is greater than a threshold.

10. The method of claim 1, wherein implementing the notification action comprises sending a notification message indicative of the determined automation command.

11. A method for performing industrial automation control, comprising:

detecting, via a sensor system, positions and/or motions of an element of an industrial system;

determining a possible automation action corresponding to the detected positions and/or motions; and

implementing a control and/or notification action based upon the detected positions and/or motions.

12. The method of claim 11, wherein the element comprises a human operator.

13. The method of claim 11, wherein the element comprises a machine component.

14. The method of claim 11, wherein the element comprises a workpiece.

15. The method of claim 11, wherein the control action comprises a command for operation of a machine.

16. The method of claim 11, comprising:

determining a confidence value associated with the detected positions and/or motions and/or the determined automation action; and

implementing a control and/or notification action based upon the detected positions and/or motions and the confidence value.

17. The method of claim 16, wherein implementing the notification action comprises sending a notification method indicating that the confidence value is low when the confidence value is not greater than a threshold.

18. The method of claim 11, wherein determining the possible automation action comprises:

detecting, via the sensor system, positions and/or motions of a human operating in the industrial system;

detecting, via the sensor system, positions and/or motions of a machine operating in the industrial system;

determining a probability that the human will contact the machine based on the positions and/or motions of the human and the positions and/or the motions of the machine; and

determining that the possible automation action comprises sending a command to alter an operation that corresponds to the machine when the probability is greater than a threshold.

19. The method of claim 18, wherein the command to alter the operation comprises stopping the operation of the machine.

20. A system for performing industrial automation control, comprising:

a sensor system configured to detect positions and/or motions of an element of an industrial system;

a processing component configured to;

compare the detected positions and/or motions with a plurality of automation commands;

identify an automation command from the plurality of automation commands based at least in part on the comparison; and

send the identified automation command to an industrial automation device in the industrial system.

21. The system of claim 20, wherein the sensor system comprises digital cameras, video cameras, infrared sensors, optical sensors, radio frequency energy detectors, sound sensors, vibration sensors, magnetic sensors, heat sensors, pressure sensors, sonar sensors, range imaging technology, or any combination thereof

22. The system of claim 20, wherein the processing component is configured to:

determine whether the element is within a space of the industrial system; and

send a command to at least one industrial automation device in the industrial system to stop at least one operation.

23. The system of claim 22, wherein the space is designated using a light curtain.

24. The system of claim 20, wherein the processing component is configured to generate a confidence value that corresponds to the detected positions and/or motions and/or based at least in part the comparisons, wherein the identified automation command is sent to the industrial automation device based at least in part on the confidence value.

25. The system of claim 24, wherein the confidence value is generated based at least in part on visibility of the element, historical data related to the element, expected positions of the element, or any combination thereof

26. The system of claim 24, wherein the processing component is configured to send a second automation command to the industrial automation device to operate in a safe state when the confidence value is not greater than a threshold.

27. The system of claim 26, wherein the safe state comprises continuing the operation of the industrial automation device.