Systems and methods for planning a robot grasp that can withstand task disturbances
1. A method for planning a robot grasp, the method comprising:
- measuring interaction forces imposed on an object by an environment while a task is demonstrated by a human user using the object to obtain a disturbance distribution dataset, wherein demonstrating the task comprises the user demonstrating the task with an actual object in an actual environment, wherein the object comprises one or more sensors that measure the forces, or comprises the user demonstrating the task with a virtual object in a virtual environment, wherein the virtual object is controlled in the virtual environment by the user with a haptic device;
a computer modeling a task requirement based upon the disturbance distribution dataset;
the computer identifying multiple robot grasp types stored in computer memory that can be used to satisfy the task requirement;
the computer calculating a grasp wrench space for each identified robot grasp type;
the computer calculating a grasp quality of each robot grasp type; and
the computer selecting the robot grasp type that has the highest grasp quality.
In one embodiment, a system and method for planning a robot grasp involve measuring interaction forces imposed on an object by an environment while a task is demonstrated using the object to obtain a disturbance distribution dataset, modeling a task requirement based upon the disturbance distribution dataset, identifying robot grasp types that can be used to satisfy the task requirement, calculating a grasp wrench space for each identified robot grasp, and calculating a grasp quality of each grasp.
|System and method for automatic motion generation|
Patent #US 6,229,552 B1
Current AssigneeAvid Technology Incorporated
Sponsoring EntityAvid Technology Incorporated
|Range finding method and apparatus|
Patent #US 4,695,156 A
Current AssigneeWestinghouse Electric Company LLC
Sponsoring EntityWestinghouse Electric Company LLC
|METHOD AND SYSTEM FOR TRAINING A ROBOT USING HUMAN-ASSISTED TASK DEMONSTRATION|
Patent #US 20130245824A1
Current AssigneeGM Global Technology Operations LLC
Sponsoring EntityGM Global Technology Operations LLC
- 1. A method for planning a robot grasp, the method comprising:
measuring interaction forces imposed on an object by an environment while a task is demonstrated by a human user using the object to obtain a disturbance distribution dataset, wherein demonstrating the task comprises the user demonstrating the task with an actual object in an actual environment, wherein the object comprises one or more sensors that measure the forces, or comprises the user demonstrating the task with a virtual object in a virtual environment, wherein the virtual object is controlled in the virtual environment by the user with a haptic device; a computer modeling a task requirement based upon the disturbance distribution dataset; the computer identifying multiple robot grasp types stored in computer memory that can be used to satisfy the task requirement; the computer calculating a grasp wrench space for each identified robot grasp type; the computer calculating a grasp quality of each robot grasp type; and the computer selecting the robot grasp type that has the highest grasp quality.
- View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9)
- 10. A non-transitory computer-readable medium comprising:
logic configured to receive interaction forces imposed on an object by an environment to obtain a disturbance distribution dataset, the forces having been measured while a task was demonstrated using the object, wherein demonstrating the task comprises the user demonstrating the task with an actual object in an actual environment, wherein the object comprises one or more sensors that measure the forces, or comprises the user demonstrating the task with a virtual object in a virtual environment, wherein the virtual object is controlled in the virtual environment by the user with a haptic device; logic configured to model a task requirement based upon the disturbance distribution dataset; logic configured to identify robot grasp types that can be used to satisfy the task requirement; logic configured to calculate a grasp wrench space for each identified robot grasp type; logic configured to calculate a grasp quality of each robot grasp type, and logic configured to select the robot grasp type that has the highest grasp quality.
- View Dependent Claims (11, 12, 13, 14)
- 15. A system for planning a robot grasp, the system comprising:
a force measurement device comprising one or more sensors configured to measure interaction forces imposed on an actual object when the object is used by a human user to perform a task in an actual environment or a haptic device configured to measure interaction forces imposed on a virtual object when the haptic device is used by the human user to perform a task with the virtual object in a virtual environment; a computer having a non-transitory computer-readable medium that stores a grasp planning system including; logic configured to receive the interaction forces measured by the force measurement device to obtain a disturbance distribution dataset, logic configured to model a task requirement based upon the disturbance distribution dataset, logic configured to identify robot grasp types that can be used to satisfy the task requirement, logic configured to calculate a grasp wrench space for each identified robot grasp type, logic configured to calculate a grasp quality of each robot grasp type, and logic configured to select the robot grasp type that has the highest grasp quality.
- View Dependent Claims (16, 17, 18, 19, 20, 21, 22)
This application claims priority to U.S. Provisional Application Ser. No. 61/915,812, filed Dec. 13, 2013, which is hereby incorporated by reference herein in its entirety.
Manipulation and grasp have been active research areas in robotics for several decades. One of the primary goals of the research is the choice of an appropriate grasp, in terms of task requirement and stability properties, given an object associated with a manipulation task to be performed. Such a problem is referred to as the grasp synthesis problem. To solve this problem, different approaches and algorithms have been developed for the robotic hand to execute a stable manipulation task.
One solution to the grasp synthesis problem is grasp planning. Grasp planning uses optimization mathematics to search for the optimal contact placement on an object, which gives rise to difficulty in choosing a quality criterion for the optimization procedure. One widely-used quality criterion is the force closure that measures the capability of a grasp to apply a certain amount of force on an object to resist disturbances in any direction, defined as the radius of the largest six-dimensional wrench space sphere centered at the origin and enclosed with the unit grasp wrench space (GWS).
The above approach is task-independent and, therefore, an evenly distributed disturbance in all directions is assumed. In many manipulation tasks, however, such as drinking, writing, and handling a screwdriver, a task-related grasp criterion must be applied for the choice of appropriate grasp configurations for different task requirements. A manipulation task refers to the process of moving or rearranging objects within an environment. To perform a task, the object to be manipulated would interact with the environment and suffer from outside disturbance. For example, the screwdriver can experience disturbance in the form of resistance forces of a screw that is being driven by the screwdriver. A task-oriented grasp should be able to resist the force disturbance required for a task. That is, the grasp should be maintained (i.e., the object should not be dropped) in spite of the force disturbance.
One typical task-oriented grasp method is to choose a suitable task wrench space (TWS) and then measure how well it can be fitted into a GWS. A challenge with this approach is that it is difficult to model the disturbance associated with the task to obtain the TWS and sensors are required to measure the contact regions and contact normals in human demonstration to obtain the TWS in reality. For this reason, most approaches empirically approximate the TWS rather than actually measure it.
Another difficulty of task-oriented grasp planning is the computational complexity of searching in a high-dimensional hand configuration space. It is, therefore, natural to introduce human experience relative to a task. Data gloves have been used in prior research to map human-hand to robotic-hand workspace and capture the TWS in virtual reality. A database of candidate grasps was considered, and grasps were evaluated by a task-related quality measure. However, the correspondence problem has been a crucial issue to map between two different configuration spaces of the human hand and the robotic hand. Other research has involved searching for candidate grasps using a shape-matching algorithm and evaluating the grasps using a task-oriented criterion. However, the TWS is also modeled by experience rather than actually measuring it from the demonstrated task.
From the above discussion, it can be appreciated that it would be desirable to have an alternative approach for planning a task-oriented grasp that can withstand disturbance from the environment associated with performing the task.
The present disclosure may be better understood with reference to the following figures. Matching reference numerals designate corresponding parts throughout the figures, which are not necessarily drawn to scale.
As described above, it would be desirable to have an alternative approach for planning a task-oriented grasp that can withstand disturbance associated with performing the task. Disclosed herein are systems and methods for planning task-oriented robot grasps based upon measurement of a distribution of task disturbance. Once the disturbance distribution has been measured, a search can be conducted for a grasp that covers the most significant part of the disturbance distribution. Rather than using a uniformly distributed task wrench space (TWS), the disclosed approach models a manipulation task with a non-parametric statistical distribution model built from the disturbance data captured during the task demonstrations. By maximizing a grasp quality criterion, the resulting grasp is more likely to cover the most frequent disturbances. In some embodiments, the candidate grasps are computed from a reduced configuration space that is confined by a set of given thumb placements and thumb directions to reduce the computational complexity of the search in a high-dimensional robotic hand configuration space, as well as to avoid the correspondence problem.
In some embodiments, the disclosed approach involves measuring the task property (force distributions) from human demonstration data instead of estimating or approximating the task property. Such demonstration can be performed within an actual environment or within a virtual environment. In either case, the task-related disturbance from the environment acting on the manipulated object is captured during execution of the task.
The task requirement can be modeled based on the distribution of the measured task disturbance. Instead of assuming an evenly-distributed TWS, the disclosed approach takes into account the task disturbance distribution measured from human demonstration, since it is possible that disturbance wrenches occur more frequently in some areas than others. Therefore, the importance of the wrenches can be weighted with the disturbance distribution and more weights can be applied to wrenches with high disturbance distribution rather than to wrenches that occur less frequently. For instance, a knife can be used for cutting as well as spreading butter. Because the interactive forces on the knife in the two tasks have two different sets of distributions, two different grasps are usually used in the two tasks.
In some embodiments, a task-oriented grasp quality metric, referred to herein as the task coverage grasp quality metric, can be used. In some cases, the planned grasp will not necessarily cover the entire TWS, which contains noise from the measurement of the disturbance. The metric is quantified by the proportion of disturbance that the grasp covers, which takes into account the noise of real TWS data. As a result, the targeted grasp can cover the most frequent disturbances, instead of covering the area with scattered distributed disturbance (which occurs rarely and is possibly noise).
In some embodiments, the computational complexity of the search in high-dimensional robotic hand configuration space, as well as to avoid a correspondence problem, can be reduced by computing the candidate grasp from a set of given thumb placements rather than contact points on an object surface. One advantage of thumb placement is that it is independent of the physical constraints of a given hand, which has the problem of solving the inverse kinematics that satisfies the constraints imposed by contact points. Every thumb placement is associated with the direction the thumb should point to, which further reduces the search space of wrist positions and orientations.
In the following disclosure, various specific embodiments are described. It is to be understood that those embodiments are example implementations of the disclosed inventions and that alternative embodiments are possible. All such embodiments are intended to fall within the scope of this disclosure.
Considering a multi-fingered robotic hand grasping an object, a grasp comprises multiple contact points. Assuming a hard finger model of the grasp, i.e., point contact with friction (PCWF), the most common friction model is Coulomb'"'"'s friction model in which, at each local contact, the tangential force is bounded by the normal force, ft≦μf n, where ft is the tangential force component, fn is the normal force component, and μ is the coefficient of friction. Thus, all feasible contact forces are constrained to the friction cone.
where the coefficient αj≧0, and Σj=1m aj=1.
The three-dimensional force vector fi and torque vector πi can be written as a wrench wi and each contact can be described with a six-dimensional vector of wrench wi:
where di is the vector from global origin of the object to the contact point and λ is the scale factor of torque to force conversion. λ can be set to be the inverse of the maximum radius from the torque origin so that torque is independent of the object scale.
Given n contact points of a grasp, the unit grasp wrench space (GWS), written as W(G), can be defined as the linear combination of the unit wrench space at each contact:
In other words, the unit GWS is the set of all possible resultant wrenches that can be applied to the object by all the contacts if applying unit magnitude of contact force, i.e., the convex hull of the contact wrenches. This is shown in
A typical way of evaluating grasp quality is to compute force-closure, i.e., the ability of a grasp to equilibrate external force and moment in any direction by applying appropriate forces. It implies that, if the origin of the wrench space is in the convex hull, then the grasp is force-closure. Similar to the GWS, a task can also be described as the space of disturbance wrenches that must be applied to the object. Ferrari and Canny quantified the force-closure property by the magnitude of the contact wrenches that can compensate the disturbance wrench in the worst case. If no task-oriented information is provided to form a subset of the whole space of wrenches, a typical TWS is a six-dimensional ball Tball centered at the wrench space origin, where external disturbance is uniformly weighted (left figure in
In other words, km(G) is the minimum magnitude of contact force in order to resist all task wrenches. The larger km is, the greater is the effort needed for a grasp to encounter the task wrench along the weakest direction. The grasp planning is to find the maximum Q(G), which is the reciprocal of km(G).
Measuring the Task Wrench
Instead of using a uniform ball, the quality measure in Equation 4 can also be used for different task requirements. Li and Sastry developed a quality criterion to measure the ability of a grasp to perform a TWS using a six-dimensional wrench space ellipsoid to better approximate a task (right figure in
Most of the existing work relied on experience to estimate TWS and predict the contact disturbance. In tool manipulation, such as with a pen, screwdriver, scoop, fork, toothbrush, etc., for example, the contact disturbance is expected to be applied on the tip of those tools. Then, the empirical task-oriented disturbance wrench space is a friction cone applied to the tip. The wrench space is assumed to be uniformly distributed in the space. However, even if the disturbance is applied to the same location of different tools, the disturbance wrench can distribute unevenly over the whole TWS. Consider, for example, a writing task versus manipulation of a screwdriver. Although both require the grasp to resist disturbance force applied to the tip, they have different disturbance distributions. As illustrated in
In some embodiments, the distribution of the disturbance wrench space can be measured in a virtual environment. In such a case, a user interface, such a haptic device, can be used by a user to control a virtual object (tool) within a virtual environment.
where w(t) is a wrench at time t, wc(t) is the contact wrench of the tool with the environment, and wn(t) is a non-contact wrench. The non-contact wrench wn(t) is an offset wrench that includes forces that are not acting on the surface of the object, such as gravity and the inertial force. In some embodiments, for simplicity, these forces can be considered to include only gravity if the user moves the tool smoothly and slowly, in which case the motion of the tool can be assumed to be pseudostatic. Gravity can be considered as the force acting on the center of mass of the object. If the center of mass is set as the torque origin, the wrench compensated by the gravity is a wrench with zero torque. If no contact occurs during the manipulation, only gravity is required to be compensated, e.g., when lifting a book on an open palm, where the task wrench stabilizes the effect of gravity along a single direction. Note that the direction of the gravity disturbance relative to the object coordinate frame changes with the motion of the object, e.g., when rotating a book by a hand, where the task wrench stabilizes the effect of gravity along multiple directions.
Verification of Simulated Disturbance Data
Since the disclosed grasp planning approach relies heavily on disturbance measure, it is necessary to validate how realistic the simulated disturbance distribution is. Thus, the distribution of simulated disturbance was compared to real measurement on a physical tool. To measure a real task disturbance, a physical tool was designed that incorporated a six-axis Nano17 force sensor connecting a handle and a stick. This tool can mimic a long-shaped tool that has a handle, such as a screwdriver, a knife, a fork, a scoop and a marker, etc. The same manipulation tasks can be demonstrated both in simulation and using the physical tool. The captured disturbance data was compared in the two environments.
Studying the Distribution Model
The distribution data was compared with a standard uniform distribution and a normal distribution by a Q-Q plot to study the distribution of the disturbance.
Because the probability distribution model of disturbance is unknown and the shapes of Q-Q plot change with tasks, a non-parametric statistical distribution of the disturbance was built from the dataset of TWS measured by demonstration for each task. Then, to reduce the computational complexity, a smaller set of data points could be randomly downsampled based on the non-parametric statistical distribution. In alternative embodiments, the task requirement can be modeled as a parametric statistical distribution, such as a Gaussian mixture model with expectation and maximization.
Quality Measure Based on Distribution of Task Disturbance
The quality measure km in Equation 4 measures how much effort a grasp needs to cover the whole required TWS, which quantifies a constraint in the worst case where the robot should not drop the object.
The quality measure km is not a reasonable constraint to the noisy TWS, given that the task is being modeled from noisy measurement data and the worst case guarantee is not a real guarantee since one can never guarantee that the data match the real world. To make km less sensitive to noise, one may consider km a coverage of the majority (e.g., 90%) of TWS, instead of the whole TWS. However, the computational effort to find the value of such a km would be a critical issue. Furthermore, without considering distribution of a task, it cannot distinguish quality between two task wrenches of the same volume but with different distributions. Consider the scenario of two different GWS for the same TWS shown in
As a result, this quality measure takes neither noise nor the distribution of real measure into account. Therefore, a quality measure must be developed for real captured task requirements. When developing a grasp quality measurement for a task-wrench distribution, the different abilities of a grasp to apply forces along different force directions should be considered. It is preferred that less effort is required of a grasp to apply forces along directions where the disturbance force frequently happens, considering the efficiency of power consumption. The GWS is not necessary to cover the whole TWS, because less ability is required to apply forces for some force directions where force magnitude is large but rarely occurs. Therefore, it makes more sense to try to capture as much of the TWS as possible, weighted by the data distribution, than it does to try to provide a guarantee that the entire TWS is captured. Intuitively, the grasp quality can be defined as the ratio of TWS that can be covered by the scaled GWS W(G), given a scale k. The set of task wrenches that is in the scaled GWS is represented as:
To make the computation easier, TWS can be scaled instead of GWS. Then, the equation can be written as:
The grasp quality can be represented as:
where |W| is the size, number of data samples, of the task wrenches covered by the scaled GWS, and |TWS| is the size of total task wrenches; 0≦Q(G)≦1. The larger Q(G) is, the more disturbance wrenches can be resisted by the grasp G. Therefore, a goal of the grasp planning is to find the optimal grasp that maximizes Q(G).
It is noted that as k increases, Q is not linearly increasing with k, and the increasing rate of Q is not the same for different grasps (
The scale k identifies for the amount of force the robotic hand is expected to apply. A scale k0 can be determined by considering both the capability of the robotic hand and the task requirement. Suppose a unit vector ŵ; stands for a fixed direction for the disturbance wrench w(t). Let a(t)=∥(t)∥, the magnitude of w(t), so that the disturbance wrench can be written as w(t)=a(t)ŵ(t). For a given task wrench set, k0 is determined by the smaller value between the maximum magnitude a(t) of all wrenches in the task and the maximum forces that can be applied by the robotic hand, typically the capability ωmax of robot motors, can be written as:
for all t=1, . . . , T, where T is the number of data samples. In experiments described herein, a Barrett hand was used. The maximum finger force of the Barrett hand was 20 N, so ωmax was set to ωmax=20 in order to bound k0. k0 can also be set to other empirical values, e.g., the amount of force that humans usually apply in a manipulation.
Computational Efficiency of Grasp Quality
To compute the quality of a grasp, the convex hull needs to be computed from the contact normals, then every sample of the TWS must be checked to determine if they are inside the scaled GWS. Convex hull can be computed by a quick hull algorithm using a Qhull C++ library, where the average case complexity is considered to be O(nlog(n)), where n is the number of contact wrenches.
To check if a point is inside the scaled GWS, one can test if the query point lies in the inward area of each facet of the convex hull. Comparing the point with one facet of the convex hull takes constant time. Thus, comparing a point with all facets of the convex hull is the worst case, taking O(K) time, where K is the number of facets of the convex hull. To check if all samples are inside the convex hull takes O(KL), where L is the number of task sampling points from the distribution of the disturbance.
Incorporation of Thumb Placement Constraint into Grasp Planning
Since a number of anthropomorphic hands have a high number of degrees of freedom (DOF) to be as powerful as human hand, thereby introducing complexity to the search in the optimization, much work has focused on providing constraints to the search space to reduce the computational complexity of the search in a high-dimensional robotic hand configuration space, for example, imposing appropriate contact points on the object. The constraint on contact points, however, is assumed to be independent of physical constraints of a given hand. This raises the problem of solving the inverse kinematics that satisfies the constraints imposed by contact points. To avoid the problem given rise by the constraints of contact points, the candidate grasp can be computed from a set of given thumb placements on the object surface, as well as the direction to which the thumb should point. Thumb positions offer a general reference of the body part to be gripped. Thumb directions provide a constraint on wrist positions and orientations. The constraint of thumb placement can be labeled manually on the object or generated automatically from examples.
The above-described approach was tested in simulation for several tasks with different objects. Non-expert subjects were asked to manipulate an object in the user interface via Phantom OMNI. The interaction force between the object and the environment was captured during the demonstration with a sample rate of 100 Hz. The data set of the disturbance, compensated by object gravity, was recorded. Then, from the data set, a non-parametric statistical distribution of the disturbance was built. To reduce the computational complexity, a smaller set of data points was randomly sampled based on the non-parametric statistical distribution.
A Barrett hand model and a Shadow hand model were tested during the simulation for task-oriented grasp planning. The desired grasp type and the constraint area of the thumb location and direction were input into the simulator as well, which highly reduced the search space of the robotic hand configuration. In the simulation, the friction coefficient μ was set to be 1. The friction cone is approximated by an eight-sided pyramid. For each hand configuration, the GWS can be computed from the contact points, and the contact normals can be obtained using the open dynamics library. Grasp quality Q was calculated based on the GWS and the distribution of disturbance. The grasp planning searches the best grasp configuration that maximizes Q.
In a second experiment, grasps were compared for two tasks using a knife. The user was asked to perform a cutting motion along one direction (+x in
In a third experiment, the user was asked to strike a plane with a hammer and the grasp planning was performed to compare results of the Barrett hand model and the Shadow hand model. It can be imagined that the chosen grasp should be excellent for balancing the large pressure on the head of the hammer. As shown in
As concluded from the experiments, the resulting grasp with a higher grasp quality criterion tends to be more efficient to apply frequently-occurring force, using the same magnitude of resultant force as the low-quality grasp, thus improving the efficiency of power consumption.
Experimental Results on Real Platform
Experiments were also performed with a real robot system composed of a Barrett robotic hand and a FANUC robotic arm. The FANUC robotic arm was a FANUC LR Mate 200iC robotic arm with six axes. The Barrett hand has a four degrees of freedom with a three-fingered programmable grasper.
In the experiment, objects and manipulation tasks were carefully selected to evaluate the disclosed approach. Three representative manipulation tasks were selected to evaluate the proposed approach, including: Task 1: move a computer mouse on a table, Task 2: plugging a power adapter into a power strip, and Task 3: screw in a light bulb to a socket. Task 1 represents a sliding interaction with the environment. Similar tasks include using a dry-erase eraser, moving a broom or a vacuum cleaner on the floor, painting, etc. Task 2 represents a peg-in-hole motion. Similar tasks include inserting a key or a flash drive, etc. Task 3 represents a screwing motion. Similar tasks include screwing a screwdriver, jar lid, knob, key, switch, etc.
Each task was executed in 10 trials with independent noise. To minimize the influence of the position error of the object on the success rate of the manipulation, a vision system was not used, which may introduce additional errors into the predicted object position and orientation. Instead, the target object was placed at the same known location and orientation in the robot'"'"'s workspace in each trial, with an estimated position error of 10 mm or less. Before each execution, the robot arm was reset to an initial position and the robotic hand was kept open. The whole execution procedure was divided to four steps. The first step was to approach the target object and reach the wrist position and orientation relative to the object, which resulted from the algorithm. Then, the robotic hand was commanded to close its fingers on the object and lift the object up from the table. These first two steps were performed autonomously by the robot. The following manipulation step was executed either autonomously or guided by humans, depending the complexity of the manipulation. The first task, i.e., moving a mouse around on a table, was relatively simple, so it was executed in a predefined routine, in which the mouse was moved along a squared path on the table. The other two tasks, however, required complicated sensing and manipulating technique to accomplish the task. Therefore, human participation was introduced in completing the remaining task by teleoperating the robot using a PHANTOM Omni haptic device. After the manipulation step was accomplished, the robot hand was then commanded to open the fingers and move up away from the object.
The Omni device was chosen due to its compact design and the intuitive positional abilities are appropriate for the teleoperation of the robotic arm. The FANUC arm and Barrett hand were connected to the same personal computer (PC) with the PHANTOM Omni. The manipulator was teleoperated at a position-based and bilateral mode, for which force feedback was provided to the user. The positions and gimbal angles of Omni stylus were continuously transmitted to the PC server in real time. The position and orientation of Omni stylus were transformed to the corresponding position and orientation of the robot end-effector in its feasible workspace. The robot arm and hand incorporate their own motion controllers. The position commands were streamed from PC server to the robot controller, so the manipulator was able to follow the Omni motion in real time. For safety, the speed was constrained up to a feedrate of 30% of the maximum speed. The force sensed by the force sensor embedded in the robot wrist was feedback to the Omni, so the user was able to feel the contact force when the robotic hand is in contact with the environment.
To evaluate the proposed algorithm on physical robotic platform, the success rate was compared with that of the widely used non-task-oriented planning method that optimizes the epsilon quality ε. The epsilon quality E measures the radius of the largest six-dimensional ball centered at the origin and enclosed with the convex hull of the GWS. The epsilon quality refers to the magnitude of the disturbance wrenches that can be compensated by the grasp in the worst case. The larger the epsilon quality is, the more stable the grasp can be in terms of resisting the worst case disturbance. In a previous study, a grasp with an epsilon quality of 0.1 or greater is considered to be a stable grasp that tends to be robust to uncertainties. The results were not compared with any other task-oriented method because, to the best of the inventors'"'"' knowledge, none of the related research on task-oriented grasp planning has reported any success rate on the execution of manipulation tasks in a physical system. Most work tested on real robotic platforms are pick-up tasks.
The resulting hand configurations from both the proposed algorithm and the non-task-oriented grasp planning algorithm are shown in
The execution on real robotic platform of both the proposed algorithm and the non-task-oriented planning method for all of the three tasks were compared and the success rate of both algorithms was compared in Table 1. If the object is sliding out of the robotic hand during the task execution because of the outside disturbance from collision, it is counted as a failure. Otherwise, if the robot successfully completes the task without dropping the object, it is counted as a success. It can be observed that, overall, hand configurations resulting from the proposed algorithm have a higher success rate (average of 70%), compared to that of the non-task-oriented planning algorithm (average of 43.3%) in executing the manipulation tasks. The results imply that the proposed quality metric is more consistent with the success rate of the manipulation tasks than the non-task-oriented algorithm. The results also verify the effective choice of scale k.
Task 1 required the robot to slide the mouse on a plane while keeping in touch with the plane. The disturbance was distributed mainly on the boundary of the friction cone (
The user interface 24 comprises one or more devices that can be used to enter user inputs into the computer 12. As noted above, the user interface 24 can comprise the haptic device 14 and the display 16 (
The memory 22 (a non-transitory computer-readable medium) stores programs including an operating system 30 and a grasp planning system 32. The operating system 30 controls the general operation of the computer 12, while the grasp planning system 32 comprises algorithms (i.e., logic) that are used to plan a robot grasp for a particular object and a particular task to be performed by the robot using the object. The memory 22 further comprises a database 34, which can be used to store grasps that are formulated by the grasp planning system 32.
Example of Grasp Planning
Beginning with block 40 of
Irrespective of whether the human demonstration is actual or virtual, the interaction forces between the object and the environment can be measured as the task is performed, as indicated in block 42. As described above, these forces can be continually measured during performance of the task so as to obtain a set of task-related disturbance distribution data. This disturbance dataset represents a task wrench space (TWS) that comprises the set of all wrenches acting upon the object as the task is performed. As was also described above, the TWS can be modified to include a non-contact wrench that accounts for forces acting upon the object that are not associated with interaction between the object and another object in the real or virtual environment, such as gravity and inertia.
Referring next to block 44, the task requirement can be modeled based upon the measured disturbance distribution dataset. As described above, a non-parametric statistical model can be generated that comprises a non-parametric statistical distribution of the disturbance. In some embodiments, a smaller set of data points can be downsampled from the non-parametric statistical distribution to reduce computational complexity. As was also described above, the task requirement can alternatively be modeled as a parametric statistical distribution, such as a Gaussian mixture model with expectation and maximization.
At this point, grasp types can be identified that are to be evaluated in relation to satisfying the task requirement, as indicated in block 46. The grasp types can be selected by the user or the grasp planning system can automatically select the grasp types to be evaluated. In some embodiments, the grasp types can be constrained by a human thumb location and direction, which can either be manually input by the user or automatically detected by the grasp planning system from the task demonstration. In either case, the thumb location and direction can significantly reduce the search space for the robotic hand configuration.
With reference next to block 48, the grasp wrench space (GWS) of each identified grasp can be calculated using the contact points and contact normals of the grasp, and the friction coefficient of the object. At this point, the quality Q of each grasp can be calculated, as indicated in block 50. As described above, this grasp quality quantifies the portion of the disturbance distribution that the grasp covers. This is graphically depicted by
Once the grasp qualities have been calculated, the grasp having the highest quality can be identified and stored for later use by a robot, as shown in block 52. In some embodiments, the grasp can later be manually selected by a user for use by a robot in performing a particular task with a particular object. In other embodiments, a robot can automatically select the grasp based upon recognition of one or more of the object and the task to be performed.
Turning to decision block 54, if further grasps are to be planned, flow returns to block 40 at which a new demonstration can be performed and the process can be repeated. If no further grasps are to be planned, however, flow for the session is terminated.