SENSOR-BASED SAFETY FEATURES FOR ROBOTIC EQUIPMENT

• US 20160224012A1
• Filed: 08/08/2014
• Published: 08/04/2016
• Est. Priority Date: 08/08/2014
• Status: Active Grant

• Current Assignee: Roboticvisiontech LLC
• Original Assignee: Robotic VISION Technologies Inc.
• Inventors: Hunt, Shawn
• Granted Patent: US 9,740,193 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B25J 9/1676 Avoiding collision or forbidden zones
• CPC Class Codes: G05B 19/4061 Avoiding collision or forbidden zones
• CPC Class Codes: G05B 2219/39001 Robot, manipulator control
• CPC Class Codes: G05B 2219/39082 Collision, real time collision avoidance
• CPC Class Codes: G05B 2219/39094 Interference checking between robot and fixture
• CPC Class Codes: G05B 2219/40339 Avoid collision

Abstract

Technologies are generally described for sensor-based safety features for robotic equipment, and the implementation thereof. One or more sensors may be positioned relative to the robotic equipment such that the sensors may capture light from at least a portion of an environment surrounding the robotic equipment. In some examples, the sensors may be integrated with the robotic equipment and/or may be configured to rotate. An analysis module coupled to the sensors may build a model image of the environment based on the light captured by the sensors. The analysis module may detect that an unintended object is approaching the robotic equipment in response to detecting a change in the model image, and based on a proximity and/or a speed of approach of the object to the robotic equipment, the analysis module may instruct the robotic equipment to reduce an operating speed and/or stop motion of the robotic equipment.

Claims

1. A method to implement sensor-based safety features for robotic equipment, the method comprising:

generating a model image of an environment surrounding the robotic equipment based on light captured by one or more sensors, wherein a time variation of the model image is stored as a plurality of frames;

detecting a change in the model image based on a comparison of one or more frames;

determining whether an unintended object is approaching the robotic equipment based on the detected change; and

in response to a determination that the unintended object is approaching the robotic equipment, instructing the robotic equipment to adjust an operating speed of the robotic equipment based on one or more of a proximity and a speed of approach of the object to the robotic equipment.

2. The method of claim 1, further comprising:

determining the one or more of the proximity and the speed of approach of the object to the robotic equipment based on the comparison of the one or more frames.

3. The method of claim 1, further comprising:

classifying pixels of the model image into one of a foreground and a background of the model image.

4. The method of claim 3, further comprising:

grouping the pixels classified as the foreground of the model image, wherein the grouped pixels classified as the foreground of the model image are indicative of motion.

5. The method of claim 4, further comprising:

assigning a threshold value for a size of the grouped pixels.

6. The method of claim 5, wherein detecting the change in the model image comprises:

determining that the size of the grouped pixels exceeds the threshold value.

7. The method of claim 1, wherein instructing the robotic equipment to adjust the operating speed of the robotic equipment comprises:

instructing the robotic equipment to reduce the operating speed of the robotic equipment to substantially zero.

8. The method of claim 1, wherein instructing the robotic equipment to adjust the operating speed of the robotic equipment comprises:

instructing the robotic equipment to bring the robotic equipment to a stop.

9. The method of claim 1, further comprising:

receiving, from an operator of the robotic equipment through a user interface displaying the model image, a definition of at least a first boundary and a second boundary within the environment surrounding the robotic equipment.

10. The method of claim 9, wherein a distance between the first boundary and the robotic equipment is less than a distance between the second boundary and the robotic equipment.

11. The method of claim 10, wherein instructing the robotic equipment comprises:

instructing the robotic equipment to stop motion in response to detecting the object at a location between the first boundary and the robotic equipment.

12. The method of claim 10, wherein instructing the robotic equipment comprises:

instructing the robotic equipment to reduce the operating speed of the robotic equipment in response to detecting the object at a location between the second boundary and the first boundary.

13. The method of claim 10, wherein instructing the robotic equipment comprises:

instructing the robotic equipment to restart motion in response to detecting the object leave the first boundary.

14. The method of claim 10, wherein instructing the robotic equipment comprises:

instructing the robotic equipment to increase the operating speed of the robotic equipment in response to detecting the object leave the second boundary.

15. The method of claim 14, wherein detecting the object leave the second boundary comprises:

storing the plurality of frames as histograms; and

determining an absence of the object in response to determining that a histogram of a current frame matches a histogram of a last known frame in which the object was absent within the second boundary.

16. An apparatus configured to implement sensor-based safety features for robotic equipment, the apparatus comprising:

one or more sensors configured to capture light from an environment surrounding the robotic equipment;

at least one analysis module communicatively coupled to the one or more sensors, the at least one analysis module configured to;

generate a model image of an environment surrounding the robotic equipment based on the light captured by the one or more sensors, wherein a time variation of the model image is stored as a plurality of frames;

detect a change in the model image based on a comparison of one or more frames;

determine whether an unintended object is approaching the robotic equipment based on the detected change; and

in response to a determination that the unintended object is approaching the robotic equipment, instruct the robotic equipment to adjust an operating speed of the robotic equipment based on one or more of a proximity and a speed of approach of the object to the robotic equipment.

17. The apparatus of claim 16, wherein the one or more sensors comprise at least one of complementary metal oxide semiconductor (CMOS) image sensors, charged coupled device (CCD) image sensors, and N-type metal oxide semiconductor (NMOS) image sensors.

18. The apparatus of claim 16, wherein the one or more sensors are positioned at one or more locations relative to the robotic equipment such that the light is captured from at least one portion of the environment surrounding the robotic equipment.

19. The apparatus of claim 16, wherein the one or more sensors are integrated with the robotic equipment.

20. The apparatus of claim 16, wherein the one or more sensors are positioned or configured to rotate such that the light is captured from substantially an entirety of the environment surrounding the robotic equipment.

21. The apparatus of claim 16, wherein the at least one analysis module is configured to generate the model image employing one or more Gaussian distributions.

22. The apparatus of claim 16, further comprising:

a user interface coupled to the at least one analysis module, the user interface configured to display the model image of the environment surrounding the robotic equipment to an operator of the robotic equipment.

23. The apparatus of claim 22, wherein the at least one analysis module is further configured to receive, from the operator of the robotic equipment, a definition of at least a first boundary and a second boundary within the environment surrounding the robotic equipment.

24. The apparatus of claim 23, wherein the at least one analysis module is further configured to determine the proximity of the object to the robotic equipment based on a location of the object within the first boundary or the second boundary.

25. A system to implement sensor-based safety features for robotic equipment, the system comprising:

at least one imaging module configured to capture light of an environment surrounding the robotic equipment through one or more sensors;

at least one modeling module configured to generate a model image of an environment surrounding the robotic equipment based on the light captured by the at least one imaging module, wherein a time variation of the model image is stored as a plurality of frames;

at least one detection module configured to detect a change in the model image based on a comparison of one or more frames and determine whether an unintended object is approaching the robotic equipment based on the detected change; and

at least one controller configured to instruct the robotic equipment to adjust an operating speed of the robotic equipment based on one or more of a proximity and a speed of approach of the object to the robotic equipment in response to a determination that the unintended object is approaching the robotic equipment.

26. The system of claim 25, wherein the at least one controller is further configured to:

control operational aspects of the at least one imaging module, the at least one modeling module, and the at least one detection module.

27. The system of claim 25, wherein the at least one modeling module is further configured to:

classify pixels of the model image into one of a foreground and a background of the model image;

group the pixels classified as the foreground of the model image; and

assign a threshold value for a size of the grouped pixels.

28. The system of claim 27, wherein the at least one detection module is further configured to:

determine that the size of the grouped pixels exceeds the threshold value in order to detect the change in the model image.