Deep machine learning methods and apparatus for robotic grasping
First Claim
1. A method, comprising:
- generating, by one or more processors, a candidate end effector motion vector defining motion to move a grasping end effector of a robot from a current pose to an additional pose;
identifying, by one or more of the processors, a current image captured by a vision sensor associated with the robot, the current image capturing the grasping end effector and at least one object in an environment of the robot;
applying, by one or more of the processors, the current image and the candidate end effector motion vector as input to a trained convolutional neural network;
generating, over the trained convolutional neural network, a measure of successful grasp of the object with application of the motion, the measure being generated based on the application of the current image and the end effector motion vector to the trained convolutional neural network;
generating an end effector command based on the measure, the end effector command being a grasp command or an end effector motion command; and
providing the end effector command to one or more actuators of the robot.
2 Assignments
0 Petitions
Accused Products
Abstract
Deep machine learning methods and apparatus related to manipulation of an object by an end effector of a robot. Some implementations relate to training a deep neural network to predict a measure that candidate motion data for an end effector of a robot will result in a successful grasp of one or more objects by the end effector. Some implementations are directed to utilization of the trained deep neural network to servo a grasping end effector of a robot to achieve a successful grasp of an object by the grasping end effector. For example, the trained deep neural network may be utilized in the iterative updating of motion control commands for one or more actuators of a robot that control the pose of a grasping end effector of the robot, and to determine when to generate grasping control commands to effectuate an attempted grasp by the grasping end effector.
-
Citations
21 Claims
-
1. A method, comprising:
-
generating, by one or more processors, a candidate end effector motion vector defining motion to move a grasping end effector of a robot from a current pose to an additional pose; identifying, by one or more of the processors, a current image captured by a vision sensor associated with the robot, the current image capturing the grasping end effector and at least one object in an environment of the robot; applying, by one or more of the processors, the current image and the candidate end effector motion vector as input to a trained convolutional neural network; generating, over the trained convolutional neural network, a measure of successful grasp of the object with application of the motion, the measure being generated based on the application of the current image and the end effector motion vector to the trained convolutional neural network; generating an end effector command based on the measure, the end effector command being a grasp command or an end effector motion command; and providing the end effector command to one or more actuators of the robot. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
-
-
14. A system, comprising:
-
a vision sensor viewing an environment; a trained convolutional neural network stored in one or more non-transitory computer readable media; at least one processor configured to; generate a candidate end effector motion vector defining motion to move a robotic end effector from a current pose to an additional pose; apply the candidate end motion vector and an image captured by the vision sensor as input to the trained convolutional neural network, the image capturing an end effector and at least one object in an environment of the object; generate, over the trained convolutional neural network, a measure of successful grasp of the object with application of the motion, the measure being generated based on the application of the image and the end effector motion vector to the trained convolutional neural network; generate an end effector command based on the measure, the end effector command being a grasp command or an end effector motion command; and provide the end effector command to one or more actuators of the robot.
-
-
15. A method of training a convolutional neural network, comprising:
-
identifying, by one or more processors, a plurality of training examples generated based on sensor output from one or more robots during a plurality of grasp attempts by the robots, each of the training examples including training example input comprising; an image for a corresponding instance of time of a corresponding grasp attempt of the grasp attempts, the image capturing a robotic end effector and one or more environmental objects at the corresponding instance of time, and an end effector motion vector defining motion of the end effector to move from an instance of time pose of the end effector at the corresponding instance of time to a final pose of the end effector for the corresponding grasp attempt, each of the training examples including training example output comprising; a grasp success label indicative of success of the corresponding grasp attempt; training, by one or more of the processors, the convolutional neural network based on the training examples. - View Dependent Claims (16, 17, 18, 19, 20, 21)
-
Specification