TIME OF FLIGHT (TOF) HUMAN MACHINE INTERFACE (HMI)

• US 20120095575A1
• Filed: 10/14/2010
• Published: 04/19/2012
• Est. Priority Date: 10/14/2010
• Status: Abandoned Application

• Current Assignee: Rockwell Automation Safety AG (Rockwell Automation, Inc.)
• Original Assignee: Rockwell Automation Technologies Incorporated (Allen-Bradley Co., Inc.), Cedes Safety & Automation AG
• Inventors: Meinherz, Carl, Brockman, Craig Martin, Fooks, Elik I., Stein, Manfred Norbert, Hardegger, Martin, Chen, Wei Jie, Berner, Reto, Galera, Richard, Black, Robert M., Merz, Roger, Nair, Suresh, Eisenbrown, Steven A.
• Application Number:
  US12/904,471
• Publication Number:
  US 20120095575A1
• Time in Patent Office: Days
• US Class Current: 700/79
• CPC Class Codes: B25J 9/1674 characterised by safety, monitoring, diagnostic
• CPC Class Codes: B25J 9/1676 Avoiding collision or forbidden zones
• CPC Class Codes: B25J 9/1694 characterised by use of sensors other than normal servo-feedback from position, speed or acceleration sensors, perception control, multi-sensor controlled systems, sensor fusion
• CPC Class Codes: G05B 19/406 characterised by monitoring or safety G05B19/19 takes precedence
• CPC Class Codes: G05B 19/4061 Avoiding collision or forbidden zones
• CPC Class Codes: G05B 19/409 characterised by using manual data input [MDI] or by using control panel, e.g. controlling functions with the panel; characterised by control panel details or by setting parameters G05B19/408, G05B19/4093 take precedence
• CPC Class Codes: G05B 2219/35444 Gesture interface, controlled machine observes operator, executes commands
• CPC Class Codes: G05B 2219/36133 MMI, HMI: man machine interface, communication
• CPC Class Codes: G05B 2219/36184 Record actions of human expert, teach by showing
• CPC Class Codes: G05B 2219/36442 Automatically teaching, teach by showing
• CPC Class Codes: G05B 2219/40202 Human robot coexistence
• CPC Class Codes: G05B 2219/40414 Man robot interface, exchange of information between operator and robot
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/23 Recognition of whole body movements, e.g. for sport training

Abstract

Systems and methods are provided for controlling industrial equipment in the performance of various industrial activities based on the detected body movement of a user in an industrial automation environment. The method includes employing a time-of-flight sensor to detect movement of a body part of the user, ascertaining whether or not the movement of the body part conforms to a recognized movement of the body part, interpreting the recognized movement of the body part as a performable action, and thereafter actuating industrial machinery to perform the performable action.

Claims

1. A method for utilizing a user'"'"'s body movement in an industrial automation environment, comprising:

employing a time-of-flight sensor to detect movement of a body part of the user;

ascertaining whether the movement of the body part conforms to a recognized movement of the body part;

interpreting the recognized movement of the body part as a performable action; and

actuating industrial machinery to perform the performable action.

2. The method of claim 1, wherein the employing of the time-of-flight sensor further comprises detecting movement of the body part in three dimensions.

3. The method of claim 1, wherein the employing of the time-of-flight sensor further comprises detecting movement of fingers, hands, arms, or torso of the user.

4. The method of claim 1, wherein the employing of the time-of-flight sensor comprises detecting a velocity of movement of the body part of the user.

5. The method of claim 1, further comprising utilizing the time of flight sensor to correlate the movement of the body part to an industrial automation command used by the industrial machinery to perform the performable action, and persisting a correspondence of the movement of the body part to the industrial automation command to memory.

6. The method of claim 5, wherein the correspondence of the movement of the body part to the industrial automation command is applicable to actuate all industrial machinery included in the industrial automation environment or at least a specific industrial machine included in the industrial automation environment.

7. The method of claim 1, wherein the movement of the body part conveys commands to the industrial machinery to stop, go, or be on standby.

8. The method of claim 1, wherein the movement of the body part conveys commands to the industrial machinery to move right, left, up, down, forwards, or backwards.

9. The method of claim 1, wherein the movement of the body part conveys a command modifier to increase or decrease a magnitude associated with a previously interpreted action or an action to be interpreted.

10. The method of claim 1, wherein the employing of the time-of-flight sensor further comprises utilizing a logic component, and a memory that persists patterns of movement.

11. The method of claim 10, wherein the utilizing of the time-of-flight sensor in conjunction with the logic component and the memory, further comprises employing fuzzy logic to ascertain whether the movement of the body part conforms to a persisted pattern of movement.

12. The method of claim 1, wherein the employing of the time-of-flight sensor to detect movement of the body part of the user further comprises recognizing an accidental or inadvertent intrusion of the body part within a bounded area monitored by the time-of-flight sensor.

13. The method of claim 12, wherein the bounded area monitored by the time-of-flight sensor is demarcated by the user using a body part to trace a periphery of the bounded area, wherein the periphery traced and associated with the bounded area is persisted to a memory.

14. A system that employs body movement to control industrial machinery in an industrial automation environment, comprising:

a time-of-flight sensor that detects movement of a body part of a user positioned proximate to the time-of-flight sensor, wherein the movement of the body part includes utilization of a pre-established sign language;

an industrial controller that establishes whether the movement of the body part conforms with a recognized movement of the body part; and

an industrial machine that performs an action based at least in part on instructions received from the industrial controller.

15. The system of claim 14, wherein the action received from the industrial controller is based at least in part on a translation of the recognized movement of the body part into an instruction.

16. The system of claim 14, wherein the time-of-flight sensor detects a velocity of the movement of the body part.

17. The system of claim 16, wherein the velocity indicates:

a speed with which a control surface associated with the industrial machine is manipulated, a force with which the control surface is manipulated, or a pressure exerted on the control surface.

18. The system of claim 17, wherein the control surface associated with the industrial machine includes:

buttons, wheels, levers, or scroll bars.

19. The system of claim 14, further comprising a human machine interface component that generates a touch screen display projected onto a projection surface with which the user interacts without touching the projection surface.

20. A system that utilizes movement performed by a user to actuate actions on industrial equipment, comprising:

means for constantly monitoring the movement performed by the user;

means for detecting an appropriate movement performed by the user;

means for demarcating, on a generated or persisted map, a safety zone around the industrial equipment described by the appropriate movement performed by the user; and

means for actuating the industrial equipment to monitor the safety zone for inadvertent intrusion.