Recognition-based industrial automation control with person and object discrimination

• US 9,393,695 B2
• Filed: 02/27/2013
• Issued: 07/19/2016
• Est. Priority Date: 02/27/2013
• Status: Active Grant

• Current Assignee: Rockwell Automation Technologies Incorporated (Allen-Bradley Co., Inc.)
• Original Assignee: Rockwell Automation Technologies Incorporated (Allen-Bradley Co., Inc.)
• Inventors: Scott, Steven J., Nguyen, Thong T., Brandt, David D., Gibart, Tony, Dotson, Gary D.
• Primary Examiner(s): Lau, Hoi
• Application Number:
  US13/779,569
• Publication Number:
  US 20140244037A1
• Time in Patent Office: 1,238 Days
• Field of Search: 340/425.5, 340/540, 340/567, 340/5.83, 340/13.23, 340/539.13, 340/573.1, 345/427, 345/589, 382/103, 382/195, 382/154, 382/100, 382/115, 382/228, 382/107, 348/222.1, 348/46, 348/155, 348/143, 348/86, 356/4.01, 356/614
• US Class Current: 1/1
• CPC Class Codes: B25J 9/1694 characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 15/02 electric
• CPC Class Codes: G05B 19/409 characterised by using manual data input [MDI] or by using control panel, e.g. controlling functions with the panel; characterised by control panel details or by setting parameters G05B19/408, G05B19/4093 take precedence
• CPC Class Codes: G05B 2219/2614 HVAC, heating, ventillation, climate control
• CPC Class Codes: G05B 2219/2642 Domotique, domestic, home control, automation, smart house
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments

Abstract

The present disclosure generally relates to a method for performing industrial automation control in an industrial automation system may include detecting, via a sensor system, positions and/or motions of one or more humans and/or one or more objects in an industrial automation system and distinguishing, via a programmed computer system, between one or more humans and one or more objects based upon the detected positions and/or motions. The method may then include implementing a control and/or notification action based upon the distinction.

Claims

1. A method for performing industrial automation control, comprising:

receiving, via a sensor system, image data representing one or more humans and one or more objects in an industrial automation system;

converting the image data into two-dimensional representations of the humans and the objects, wherein each of the two-dimensional representations comprise a plurality of data points located at edges of a respective two-dimensional representation;

distinguishing between the one or more humans and the one or more objects based upon the plurality of data points for each respective two-dimensional representation and a plurality of templates, wherein each of the plurality of templates comprise a known arrangement of data points;

designating one of the one or more humans as an authorized operator based on an article carried or worn by the one of the one or more humans, wherein the authorized operator is authorized to implement one or more control actions to control one or more operations of one or more machines in the industrial automation system; and

implementing the one or more control actions based upon one or more motions of the authorized operator.

2. The method of claim 1, wherein the one of the one or more humans is distinguished from other humans who is not the authorized operator.

3. The method of claim 1, wherein the article comprises an article of clothing.

4. The method of claim 1, wherein the one of the one or more humans is distinguished from one of the objects.

5. The method of claim 4, wherein the control actions comprise adjusting an environmental system based upon whether movement of the one of the one or more humans or the one of the objects is detected.

6. The method of claim 5, wherein the environmental system comprises a lighting unit or an air conditioning unit, and wherein the lighting unit or the air conditioning unit is configured to operate in a human-suitable condition if the movement of the one of the humans is detected.

7. The method of claim 5, wherein the environmental system comprises a lighting unit or an air conditioning unit, and wherein the lighting unit or the air conditioning unit is placed in a human-unsuitable condition if the movement of the one of the objects is detected and no human is detected.

8. The method of claim 1, comprising:

detecting, via the sensor system, an undesirable object in the industrial automation system; and

implementing the one or more control actions based upon the detecting of the undesired object.

23. The method of claim 8, wherein the undesired object comprises a gun or a laptop.

9. A system for performing industrial automation control, comprising:

a sensor system configured to receive image data depicting an element in an industrial system;

a processing component configured to;

convert the image data into a two-dimensional representation of the element, wherein the two-dimensional representation comprises a plurality of data points located at edges of the two-dimensional representation;

identify one or more humans based on the plurality of data points and a plurality of templates, wherein each of the plurality of templates comprises a known arrangement of data points;

receive one or more signals from one or more emitters disposed on one of the one or more humans;

designate the one of the one or more humans as an authorized operator based on the signals, wherein the authorized operator is authorized to implement one or more control actions to control one or more operations of one or more machines in the industrial system; and

send a command to the one or more machines based on one or more actions by the authorized operator.

10. The system of claim 9, comprising determining whether the element corresponds to the one or more humans or one or more objects by comparing positions of the data points with a known pattern of positions associated with a human.

11. The system of claim 9, comprising determining whether the element corresponds to the one or more humans or one or more objects by comparing motions of the data points with a known pattern of motion associated with a human.

12. The system of claim 9, wherein the processing component is configured to send the command to an environment control device to control an airflow, maintain an air quality, or regulate a temperature in the industrial system when the one or more humans are identified.

13. The system of claim 12, wherein the environment control device is a heating, ventilation, and air conditioning (HVAC) unit.

14. The system of claim 13, wherein the command to the HVAC unit is configured to set a condition in the industrial system to a human-unsuitable condition when the one or more humans are not identified.

15. The system of claim 12, wherein the processing component is configured to send the command to the environment control device to turn on one or more lights in the industrial system when the one or more humans are identified.

16. The system of claim 9, wherein the processing component is configured to

send an automation command to stop an industrial automation device when the one or more humans are determined to be present in an area associated with the industrial system.

17. A method for performing industrial automation control, comprising:

receiving, via a sensor system, image data depicting one or more humans in an industrial automation system;

converting the image data into one or more two-dimensional representations of the one or more humans, wherein each of the one or more two-dimensional representations comprises a plurality of data points located at edges thereof;

detecting, via the sensor system, motions of the one or more humans in the industrial automation system based on the plurality of data points and a plurality of templates, wherein each of the plurality of templates comprises a known arrangement of data points;

determining whether the one or more humans are associated with an authorized status based upon the detected motions, wherein the detected motions comprise a gesture-based sign in motion; and

implementing a control or notification action based upon whether the one or more humans are associated with the authorized status, wherein the one or more humans associated with the authorized status are authorized to implement the control or notification action.

18. The method of claim 17, comprising:

detecting, via the sensor system, an article of clothing is disposed on the one or more humans; and

confirming that the one or more humans are associated with the authorized status when the article of clothing is disposed on the one or more humans.

19. The method of claim 17, wherein determining that the one or more humans are associated with the authorized status comprises:

detecting, via the sensor system, one or more facial features and/or body parameters of the one or more humans;

comparing the detected facial features and/or body parameters to one or more profiles of a human; and

confirming that the one or more humans are associated with the authorized status based on the comparison.

20. The method of claim 17, wherein determining that the one or more humans are associated with the authorized status comprises:

detecting, via the sensor system, one or more colors disposed on the one or more humans; and

confirming that the one or more humans are associated with the authorized status when the colors are associated with the authorized status.

21. The method of claim 17, wherein determining that the one or more humans are associated with the authorized status comprises receiving one or more signals from one or more emitters disposed on the one or more humans, wherein the one or more signals are associated with the authorized status.

22. The method of claim 21, wherein the one or more emitters comprise radio frequency identification (RFID) sensors.