Sensor-based safety features for robotic equipment

• US 9,740,193 B2
• Filed: 08/08/2014
• Issued: 08/22/2017
• Est. Priority Date: 08/08/2014
• Status: Active Grant

• Current Assignee: Roboticvisiontech LLC
• Original Assignee: Roboticvisiontech LLC
• Inventors: Hunt, Shawn
• Primary Examiner(s): Jen, Ian
• Application Number:
  US14/432,732
• Publication Number:
  US 20160224012A1
• Time in Patent Office: 1,110 Days
• CPC Class Codes: B25J 9/1676 Avoiding collision or forbidden zones
• CPC Class Codes: G05B 19/4061 Avoiding collision or forbidden zones
• CPC Class Codes: G05B 2219/39001 Robot, manipulator control
• CPC Class Codes: G05B 2219/39082 Collision, real time collision avoidance
• CPC Class Codes: G05B 2219/39094 Interference checking between robot and fixture
• CPC Class Codes: G05B 2219/40339 Avoid collision

Abstract

Technologies are generally described for sensor-based safety features for robotic equipment, and the implementation thereof. One or more sensors may be positioned relative to the robotic equipment such that the sensors may capture light from at least a portion of an environment surrounding the robotic equipment. In some examples, the sensors may be integrated with the robotic equipment and/or may be configured to rotate. An analysis module coupled to the sensors may build a model image of the environment based on the light captured by the sensors. The analysis module may detect that an unintended object is approaching the robotic equipment in response to detecting a change in the model image, and based on a proximity and/or a speed of approach of the object to the robotic equipment, the analysis module may instruct the robotic equipment to reduce an operating speed and/or stop motion of the robotic equipment.

Claims

1. A method to implement sensor-based safety features for robotic equipment, the method comprising:

generating a model image of an environment that surrounds the robotic equipment based on light captured by one or more sensors, wherein a time variation of the model image is stored as a plurality of frames;

receiving, through a user interface that displays the model image, a definition of at least a first boundary and a second boundary within the environment that surrounds the robotic equipment,

wherein a distance between the first boundary and the robotic equipment is less than a distance between the second boundary and the robotic equipment;

detecting a change in the model image based on a comparison of one or more of the plurality of frames;

determining whether an unintended object is approaching the robotic equipment based on the detected change;

in response to a determination that the unintended object is approaching the robotic equipment, instructing the robotic equipment to adjust an operating speed of the robotic equipment based on one or more of a proximity and a speed of approach of the unintended object to the robotic equipment; and

in response to detection that the unintended object is leaving the second boundary within the environment that surrounds the robotic equipment, instructing the robotic equipment to increase the operating speed of the robotic equipment,

wherein the detection that the unintended object is leaving the second boundary comprises storing the plurality of frames as histograms and determining an absence of the unintended object in response to a determination that a histogram of a current frame matches a histogram of a last known frame in which the unintended object was absent within the second boundary.

2. The method of claim 1, further comprising:

determining the one or more of the proximity and the speed of approach of the unintended object to the robotic equipment based on the comparison of the one or more of the plurality of frames.

3. The method of claim 1, wherein instructing the robotic equipment to adjust the operating speed of the robotic equipment comprises:

instructing the robotic equipment to reduce the operating speed of the robotic equipment to substantially zero.

4. The method of claim 1, wherein instructing the robotic equipment to adjust the operating speed of the robotic equipment comprises:

instructing the robotic equipment to stop motion in response to detection that the unintended object is at a location between the first boundary and the robotic equipment.

5. The method of claim 1, wherein instructing the robotic equipment to adjust the operating speed of the robotic equipment comprises:

instructing the robotic equipment to reduce the operating speed of the robotic equipment in response to detection that the unintended object is at a location between the second boundary and the first boundary.

6. The method of claim 1, further comprising instructing the robotic equipment to restart motion in response to detection that the unintended object is leaving the first boundary.

7. An apparatus configured to implement sensor-based safety features for robotic equipment, the apparatus comprising:

one or more sensors configured to capture light from an environment that surrounds the robotic equipment; and

at least one analysis module communicatively coupled to the one or more sensors, wherein the at least one analysis module is configured to;

generate a model image of the environment that surrounds the robotic equipment based on the light captured by the one or more sensors, wherein a time variation of the model image is stored as a plurality of frames;

receive, through a user interface that displays the model image, a definition of at least a first boundary and a second boundary within the environment that surrounds the robotic equipment,

wherein a distance between the first boundary and the robotic equipment is less than a distance between the second boundary and the robotic equipment;

detect a change in the model image based on a comparison of one or more of the plurality of frames;

determine whether an unintended object is approaching the robotic equipment based on the detected change;

in response to a determination that the unintended object is approaching the robotic equipment, instruct the robotic equipment to adjust an operating speed of the robotic equipment based on one or more of a proximity and a speed of approach of the unintended object to the robotic equipment; and

in response to detection that the unintended object is leaving the second boundary within the environment that surrounds the robotic equipment, instruct the robotic equipment to increase the operating speed of the robotic equipment,

wherein to detect that the unintended object is leaving the second boundary, the at least one analysis module is configured to store the plurality of frames as histograms and to determine an absence of the unintended object in response to a determination that a histogram of a current frame matches a histogram of a last known frame in which the unintended object was absent within the second boundary.

8. The apparatus of claim 7, wherein the one or more sensors comprise at least one of complementary metal oxide semiconductor (CMOS) image sensors, charged coupled device (CCD) image sensors, and N-type metal oxide semiconductor (NMOS) image sensors.

9. The apparatus of claim 7, wherein the one or more sensors are positioned at one or more locations relative to the robotic equipment such that the light is captured from at least one portion of the environment that surrounds the robotic equipment.

10. The apparatus of claim 7, wherein the one or more sensors are integrated with the robotic equipment.

11. The apparatus of claim 7, wherein the one or more sensors are positioned or configured to rotate such that the light is captured from substantially an entirety of the environment that surrounds the robotic equipment.

12. The apparatus of claim 7, wherein the at least one analysis module is configured to employ one or more Gaussian distributions to generate the model image.

13. The apparatus of claim 7,

wherein the user interface is coupled to the at least one analysis module and the user interface is configured to display the model image of the environment that surrounds the robotic equipment to an operator of the robotic equipment.

14. A system to implement sensor-based safety features for robotic equipment, the system comprising:

at least one imaging module configured to capture light of an environment that surrounds the robotic equipment through one or more sensors;

at least one modeling module configured to generate a model image of the environment that surrounds the robotic equipment based on the light captured by the at least one imaging module, wherein a time variation of the model image is stored as a plurality of frames;

at least one detection module configured to detect a change in the model image based on a comparison of one or more of the plurality of frames, and to determine whether an unintended object is approaching the robotic equipment based on the detected change; and

at least one controller configured to;

receive, through a user interface that displays the model image, a definition of at least a first boundary and a second boundary within the environment that surrounds the robotic equipment,

wherein a distance between the first boundary and the robotic equipment is less than a distance between the second boundary and the robotic equipment;

instruct the robotic equipment to adjust an operating speed of the robotic equipment based on one or more of a proximity and a speed of approach of the unintended object to the robotic equipment, in response to a determination that the unintended object is approaching the robotic equipment; and

instruct the robotic equipment to increase the operating speed of the robotic equipment in response to detection that the unintended object is leaving the second boundary within the environment that surrounds the robotic equipment,

wherein to detect that the unintended object is leaving the second boundary, the at least one detection module is configured to store the plurality of frames as histograms and to determine an absence of the unintended object in response to a determination that a histogram of a current frame matches a histogram of a last known frame in which the unintended object was absent within the second boundary.

15. The system of claim 14, wherein the at least one modeling module is further configured to:

classify pixels of the model image into one of a foreground and a background of the model image;

group the pixels classified as the foreground of the model image; and

assign a threshold value for a size of the grouped pixels.

16. The system of claim 15, wherein to detect the change in the model image, the at least one detection module is configured to:

determine that the size of the grouped pixels exceeds the threshold value.