SYSTEM AND METHOD OF IDENTIFYING OBJECTS

• US 20080181485A1
• Filed: 12/14/2007
• Published: 07/31/2008
• Est. Priority Date: 12/15/2006
• Status: Abandoned Application

• Current Assignee: Roboticvisiontech LLC
• Original Assignee: Roboticvisiontech LLC
• Inventors: Beis, Jeffrey S., Habibi, Babak
• Application Number:
  US11/957,258
• Publication Number:
  US 20080181485A1
• Time in Patent Office: Days
• US Class Current: 382/153
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 19/402 characterised by control arrangements for positioning, e.g. centring a tool relative to a hole in the workpiece, additional detection means to correct position G05B19/19 takes precedence
• CPC Class Codes: G05B 2219/37555 Camera detects orientation, position workpiece, points of workpiece
• CPC Class Codes: G05B 2219/39109 Dual arm, multiarm manipulation, object handled in cooperation
• CPC Class Codes: G05B 2219/40584 Camera, non-contact sensor mounted on wrist, indep from gripper
• CPC Class Codes: G05B 2219/45063 Pick and place manipulator

Abstract

A system and method for identifying objects using a robotic system are disclosed. Briefly described, one embodiment is a method that captures a first image of at least one object with an image capture device that is moveable with respect to the object, processes the first captured image to determine a first pose of at least one feature the object, determines a first hypothesis that predicts a predicted pose of the identified feature based upon the determined first pose, moves the image capture device, captures a second image of the object, processes the captured second image to identify a second pose of the feature, and compares the second pose of the object with the predicted pose of the object.

Claims

1. A method for identifying objects with a robotic system, the method comprising:

capturing an image of at least one object with an image capture device that is moveable with respect to the object;

processing the captured image to identify at least one feature of the at least one object; and

determining a hypothesis based upon the identified feature.

2. The method of claim 1, further comprising:

determining a difference between the identified feature and a corresponding reference feature of a known model of the object, such that determining the hypothesis is based at least in part upon the difference between the identified feature and the reference feature.

3. The method of claim 1, further comprising:

processing the captured image to identify a different feature of the at least one object; and

determining a difference between the identified different feature and a corresponding reference feature of the known model of the object, such that determining the hypothesis is based at least in part upon the difference between the identified different feature and the corresponding reference feature.

4. The method of claim 1, further comprising:

moving the image capture device;

capturing a new image of the at least one object with the image capture device;

processing the new captured image to identify the at least one feature; and

determining a pose of the at least one feature based upon the hypothesis and the feature identified in the new captured image.

5. The method of claim 4, further comprising:

determining a confidence level for the hypothesis based upon the determined pose; and

validating the hypothesis in response to the confidence level equaling at least a threshold.

6. The method of claim 5, further comprising:

determining that the pose of the object is valid in response to validation of the hypothesis.

7. The method of claim 4, further comprising:

determining a confidence level of the hypothesis based upon the determined pose;

invalidating the hypothesis in response to the confidence level being less than a threshold;

discarding the hypothesis;

capturing a new image of the object;

processing the new captured image to identify the at least one feature of the object; and

determining a new hypothesis based upon the identified at least one feature.

8. The method of claim 4, further comprising:

determining a new hypothesis based upon the identified feature in the new captured image.

9. The method of claim 1, further comprising:

determining a movement for the image capture device based at least in part on the hypothesis; and

moving the image capture device in accordance with the determined movement.

10. The method of claim 9 wherein moving the image capture device comprises:

changing the position of the image capture device.

11. The method of claim 9 wherein determining the movement for the image capture device comprises:

determining a direction of movement for the image capture device, wherein the image capture device is moved in the determined direction of movement.

12. The method of claim 9, further comprising:

capturing a new image after movement of the image capture device;

processing the captured new image to re-identify the feature;

determining a new movement for the image capture device based at least in part on the re-identified feature; and

moving the image capture device in accordance with the determined new movement.

13. The method of claim 9, further comprising:

determining at least one lighting condition around the object such that the movement for the image capture device is based at least in part upon the determined lighting condition.

14. The method of claim 13 wherein a first direction of movement increases a lighting condition of the object in a subsequently captured image of the object.

15. The method of claim 1 wherein determining the hypothesis based upon the identified feature comprises:

determining a difference between the identified feature and a corresponding reference feature of a known model of the object; and

determining a predicted pose of the object based at least in part upon the difference between the identified feature and the corresponding reference feature, wherein the hypothesis is based at least in part upon the predicted pose.

16. The method of claim 1 wherein the image includes a plurality of objects, further comprising:

processing the captured image to identify the feature associated with at least two of the plurality of objects that are visible in the captured image;

determining at least one initial hypothesis for each of the at least two objects based upon the identified feature;

determining a confidence level for each of the initial hypotheses determined for the at least two objects; and

selecting the initial hypothesis with the greatest confidence level.

17. The method of claim 16, further comprising:

validating the selected initial hypothesis in response to the confidence level equaling at least a threshold.

18. The method of claim 17, further comprising:

determining a pose of the object associated with the validated initial hypothesis.

19. The method of claim 1 wherein the captured image includes an artificial feature on the object, further comprising:

processing the captured image to identify the artificial feature; and

determining the first hypothesis based upon the identified artificial feature.

20. The method of claim 19 wherein the artificial feature is painted on the object.

21. The method of claim 19 wherein the artificial feature is a decal affixed on the object.

22. A robotic system that identifies objects, comprising:

an image capture device mounted for movement with respect to a plurality of objects; and

a processing system communicatively coupled to the image capture device, and operable to;

receive a plurality of images captured by the image captive device;

identify at least one feature for at least two of the objects in the captured images;

determine at least one hypothesis predicting a pose for the at least two objects based upon the identified feature;

determine a confidence level for each of the hypotheses; and

select the hypothesis with the greatest confidence level.

23. The system of claim 22 where, in response to the confidence level being less than a threshold, the processing system is operable to:

determine a movement of the image capture device based upon the selected hypothesis; and

generate a movement command signal, wherein the image capture device is moved in accordance with the movement command signal.

24. The system of claim 23, further comprising:

a robot arm member with the image capture device secured thereon and communicatively coupled to the processing system so as to receive the movement command signal, wherein a robot arm member moves the image capture device in accordance with the movement command signal.

25. The system of claim 22 wherein the processing system is operable to validate the selected hypothesis in response to the corresponding confidence level equaling at least a threshold.

26. The system of claim 25 wherein the processing system is operable to determine a pose of the object in response to validation of the selected hypothesis.

27. A method for identifying objects with a robotic system, the method comprising:

capturing a first image of at least one object with an image capture device that is moveable with respect to the object;

determining a first hypothesis based upon at least one feature identified in the first image, wherein the first hypothesis is predictive of a pose of the feature;

capturing a second image of the at least one object after a movement of the image capture device;

determining a second hypothesis based upon the identified feature, wherein the second hypothesis is predictive of the pose of the feature; and

comparing the first hypothesis with the second hypothesis.

28. The method of claim 27 wherein determining the first and second hypotheses comprises:

determining a difference between the identified feature in the first captured image and a reference feature of a known model of the object;

determining the first hypothesis based at least in part upon the determined difference between the identified feature in the first captured image and the reference feature;

determining a difference between the identified feature in the second captured image and the reference feature of the known model of the object; and

determining the second hypothesis based at least in part upon the determined difference between the identified feature in the second captured image and the reference feature.

29. The method of claim 28 wherein determining the first hypothesis comprises:

processing the first captured image to identify a second feature of the at least one object;

determining a second difference between the identified second feature and a second reference feature of the known model of the object; and

determining the first hypothesis based at least in part upon a difference between the identified second feature and the second reference feature.

30. The method of claim 27, further comprising:

determining a confidence level based upon the first hypothesis and the second hypothesis;

validating the first and second hypotheses in response to the confidence level equaling at least a first threshold; and

invalidating the first and second hypotheses in response to the confidence level being less than a second threshold.

31. The method of claim 30, further comprising:

determining a pose of the object in response to validation of the first and second hypotheses.

32. The method of claim 30 where, in response to invalidating the first and the second hypothesis, the method further comprises:

discarding the first hypothesis and the second hypothesis;

capturing a new first image of the object;

determining a new first hypothesis based upon the at least one feature identified in the new first image, wherein the new first hypothesis is predictive of a new pose of the feature;

capturing a new second image of the at least one object after a subsequent movement of the image capture device;

determining a new second hypothesis based upon the identified feature, wherein the new second hypothesis is predictive of the new pose of the feature; and

comparing the new first hypothesis with the new second hypothesis.

33. The method of claim 30 where, in response to the confidence level being less than the first threshold, the method further comprises:

changing at least a relative pose between the image capture device and the object;

capturing another image of at least the object; and

determining a third hypothesis based upon the identified feature, wherein the third hypothesis is predictive of the pose of the feature.

34. The method of claim 33, further comprising:

selecting one of the first and the second hypotheses;

comparing the third hypothesis with at least the selected hypothesis;

determining a new confidence level of the compared third hypothesis and selected hypothesis; and

validating at least the third hypothesis in response to the new confidence level equaling at least the first threshold.

35. The method of claim 34, further comprising:

determining a pose of the object in response to validation of the third hypothesis.

36. The method of claim 27, further comprising:

determining the movement for the image capture device based at least in part on the first hypothesis; and

moving the image capture device in accordance with the determined movement.

37. A method for identifying one of a plurality of objects with a robotic system, the method comprising:

capturing an image of a plurality of objects;

processing the captured image to identify a feature associated with at least two of the objects visible in the captured image;

determining a hypothesis for the at least two visible objects based upon the identified feature;

determining a confidence level for each of the hypotheses for the at least two visible objects; and

selecting the hypotheses with the greatest confidence level.

38. The method of claim 37 wherein determining the hypothesis for the visible objects based upon the identified feature comprises:

comparing the identified feature of the objects with a corresponding reference feature of a reference object, wherein the reference object corresponds to the plurality of objects.

39. The method of claim 37, further comprising:

validating the selected hypothesis in response to the confidence level of the selected hypothesis equaling at least a threshold.

40. The method of claim 39, further comprising:

determining a pose of the object associated with the selected hypothesis in response to validation of the selected hypothesis.

41. The method of claim 40, further comprising:

moving an end effector physically coupled to a robot arm to the determined pose of the object associated with the selected hypothesis; and

grasping the object associated with the selected hypothesis with the end effector.

42. The method of claim 37, further comprising:

comparing the confidence level of the selected hypothesis with a threshold;

invalidating the selected hypothesis in response to the confidence level being less than the threshold; and

selecting a remaining one of the hypotheses.

43. A method for identifying objects using a robotic system, the method comprising:

capturing a first image of at least one object with an image capture device that is moveable with respect to the object;

determining a first pose of at least one feature of the object from the captured first image;

determining a hypothesis that predicts a predicted pose of the feature based upon the determined first pose;

capturing a second image of the object;

determining a second pose of the feature from the captured second image; and

updating the hypothesis based upon the determined second pose.

44. The method of claim 43, further comprising:

determining a confidence level of the hypothesis;

validating the hypothesis in response to the confidence level equaling at least a threshold; and

determining a pose of the object based upon the predicted pose in response to validation of the hypothesis.

45. The method of claim 44 where, in response to the first confidence level being less than the threshold, the method further comprises:

again moving the image capture device;

capturing a third image of the object;

determining a third pose of the feature from the captured third image; and

updating the hypothesis based upon the third pose.

46. A method for identifying objects using a robotic system, the method comprising:

capturing a first image of at least one object with an image capture device that is moveable with respect to the object;

determining a first view of at least one feature of the object from the captured first image;

determining a first hypothesis based upon the first view that predicts a first possible orientation of the object;

determining a second hypothesis based upon the first view that predicts a second possible orientation of the object;

moving the image capture device;

capturing a second image of the object;

determining a second view of the at least one feature of the object from the captured second image;

determining an orientation of a second view of the at least one feature; and

comparing the orientation of the second view with the first possible orientation of the object and the second possible orientation of the object.

47. The method of claim 46, further comprising:

selecting one of the first hypothesis and the second hypothesis that compares closest to the orientation of the second view.