Methods and apparatus to facilitate operations in image based systems

• US 8,559,699 B2
• Filed: 10/10/2008
• Issued: 10/15/2013
• Est. Priority Date: 10/10/2008
• Status: Active Grant

• Current Assignee: Roboticvisiontech LLC
• Original Assignee: Roboticvisiontech LLC
• Inventors: Boca, Remus
• Primary Examiner(s): Park, Edward
• Application Number:
  US12/249,658
• Publication Number:
  US 20100092032A1
• Time in Patent Office: 1,831 Days
• Field of Search: 382/153, 348/113, 700/245, 700/264
• US Class Current: 382/153
• CPC Class Codes: B25J 9/1679 characterised by the tasks executed
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 2219/39508 Reorientation of object, orient, regrasp object
• CPC Class Codes: G05B 2219/40014 Gripping workpiece to place it in another place

Abstract

Vision based systems may select actions based on analysis of images to redistribute objects. Actions may include action type, action axis and/or action direction. Analysis may determine whether an object is accessible by a robot, whether an upper surface of a collection of objects meet a defined criteria and/or whether clusters of objects preclude access.

Claims

1. A method of operating an image based autonomous object repositioning system, the method comprising:

acquiring data indicative of an image of an area in which a plurality of objects may reside using at least one image sensor;

analyzing the acquired image data for data indicative of a respective representation of any of the objects using at least one processor communicably coupled to the at least one image sensor;

based on the analysis of the acquired image data, determining by the at least one processor a preferred non-object picking physical output action from a plurality of different possible non-object picking physical output actions to cause the physical movement of the objects in the area, without physically picking any of the objects, the determining the preferred non-object picking physical output action based at least in part on a uniformity of height of the plurality of objects over the area;

based on the analysis of the acquired image data, determining by the at least one processor a preferred axis or direction of the preferred non-object picking physical output action in which to tilt, shake, sweep or blow the objects in the area to expose at least one pickable object in the plurality of objects; and

communicatively transmitting by the at least one processor at least one signal to one or more outputs capable of physically displacing at least a portion of the objects in the area to cause the determined preferred non-object picking physical output action to occur along the preferred axis or direction.

2. The method of claim 1, further comprising:

based on the analysis of the acquired image data, determining by the at least one processor whether to cause movement of the objects in the area.

3. The method of claim 2 wherein determining by the at least one processor whether to cause movement of the objects in the area includes determining by the at least one processor whether any of the objects are currently positioned for engagement by a robot member without causing the movement.

4. The method of claim 2 wherein determining by the at least one processor whether to cause movement of the objects in the area includes determining whether an upper surface level of the objects is within a defined threshold across at least a portion of the area.

5. The method of claim 2 wherein determining by the at least one processor whether to cause movement of the objects in the area includes determining by the at least one processor whether data indicative of at least one object appears in the acquired image data and data indicative of at least one object in the acquired image data indicates that a robot member is capable of physically engaging the at least one object.

6. The method of claim 1 wherein determining by the at least one processor a preferred non-object picking physical output action from a plurality of different possible non-object picking physical output actions to cause the physical movement of the objects in the area without physically picking any of the objects includes determining by the at least one processor which of the different possible non-object picking physical output actions has a highest likelihood of exposing at least one of the objects.

7. The method of claim 1 wherein determining a preferred non-object picking physical output action from a plurality of different possible non-object picking physical output actions to cause the physical movement of the objects in the area without physically picking any of the objects includes generating by the at least one processor data indicative of a representation of a current upper surface level of the objects from the acquired image data, and determining at least one movement calculated to move the current upper surface level toward a desired upper surface level.

8. The method of claim 1 wherein determining a preferred non-object picking physical output action from a plurality of different possible non-object picking physical output actions to cause the physical movement the objects in the area without physically picking any of the objects includes determining which of the different possible non-object picking physical output actions has a highest likelihood of distributing the objects to achieve a more uniform upper surface level.

9. The method of claim 1 wherein determining a preferred non-object picking physical output action from a plurality of different possible non-object picking physical output actions to cause the physical movement of the objects in the area without physically picking any of the objects includes determining a preferred action based at least in part on at least one of a geometry of a container that carries the objects, a size of the objects or a geometry of the objects.

10. The method of claim 1 wherein analyzing the acquired image data for data indicative of a respective representation of any of the objects includes identifying a cluster of the objects and determining using the at least one processor a preferred non-object picking physical output action from a plurality of different possible non-object picking physical output actions that cause movement of the objects in the area without physically picking any of the objects includes determining an action to disperse the cluster of the objects.

11. The method of claim 1 wherein communicatively transmitting at least one signal to one or more outputs capable of physically moving the objects to cause the determined preferred non-object picking physical output action to occur along the preferred axis or direction includes transmitting the at least one signal to at least one of a robotic controller that controls a robotic arm or at least one actuator coupled to a move a table or container on which the objects are carried.

12. The method of claim 1 wherein analyzing the acquired image data for data indicative of a respective representation of any of the objects includes performing at least one of a feature recognition, a registration or a pose estimation for at least one of the objects based on the respective representation of the object in the acquired image.

13. An image based system, comprising:

at least one image acquisition device that acquires data indicative of an image of an area in which a plurality of objects may reside;

at least one processor; and

at least one processor-readable storage medium that stores processor executable instructions that cause the processor to;

analyze the acquired image data for data indicative of a respective representation of any of the objects;

based on the analysis of the acquired image data, determine a preferred action from a plurality of different possible non-object picking actions that cause movement of the objects in the area, if any, without physically picking any of the objects, the determining the preferred action based at least in part on a uniformity of height of the plurality of objects over the area;

based on the analysis of the acquired image data, determine a preferred axis or direction in which to tilt, shake, sweep or blow the objects to expose at least one pickable object in the plurality of objects; and

transmit at least one signal to cause the determined preferred action to occur.

14. The image based system of claim 13, further comprising:

at least one robot selectively positionable to physically pick at least one of the objects.

15. The image based system of claim 14 wherein the at least one robot is positionable to cause movement of the objects in accordance with the at least one signal.

16. The image based system of claim 14, further comprising:

a support structure on which the objects are carried; and

at least one actuator coupled to physically move the support structure in accordance with the at least one signal.

17. A method of operating an image based autonomous object repositioning system, the method comprising:

repeatedly acquiring data indicative of a number of images of an area in which a number of objects may reside using at least one image sensor;

identifying one or more clusters of the objects in the acquired image data using at least one processor communicably coupled to the at least one image sensor;

selecting by the at least one processor a preferred non-object picking physical output action to disperse the one or more clusters of the objects from a plurality of different possible non-object picking physical output actions without physically removing any of the objects or any of the one or more objects from the area or displacing any of the objects in the area, based at least in part on a characteristic of the identified one or more clusters and based at least in part on a uniformity of height of the number of objects over the area;

selecting by the at least one processor a preferred axis or direction in which to tilt, shake, sweep or blow the one or more clusters of objects to adjust the upper surface level of the one or more objects toward a defined upper surface level; and

providing a signal that causes the selected action.

18. The method of claim 17 wherein selecting by the at least one processor a preferred non-object picking physical output action to disperse the one or more clusters based at least in part on a characteristic of the identified one or more clusters includes generating data indicative of a representation of an upper surface level of the one or more clusters of objects from the acquired image data by the at least one processor, and determining by the at least one processor at least one direction or axis of movement to adjust the upper surface level of the one or more clusters toward a defined upper surface level.

19. The method of claim 17, further comprising:

engaging at least one of the objects with a robotic member; and

performing the desired action while the robotic member engages at least one of the objects.

20. The method of claim 17 wherein selecting by the at least one processor a preferred non-object picking physical output action to disperse the one or more clusters based at least in part on a characteristic of the identified one or more clusters includes determining by the at least one processor at least one direction or axis of movement based at least in part on at least one representation of at least one surface of a container or a support structure that carries the objects.

21. An image based system, comprising:

at least one image acquisition device that repeatedly acquires data indicative of an image of an area in which a plurality of objects may reside;

at least one processor; and

at least one processor-readable storage medium that stores processor executable instructions that cause the processor to;

identify one or more clusters of the objects in the acquired image data;

select a preferred non-object picking physical output action to disperse the one or more clusters of the objects from a plurality of different possible non-object picking physical output actions that do not physically remove any of the objects from the area, based at least in part on a characteristic of the identified one or more clusters and based at least in part on a uniformity of height of the plurality of objects over the area;

select a preferred axis or direction in which to tilt, shake, sweep or blow the one or more clusters of objects to disperse the one or more clusters of the objects based at least in part on a characteristic of the identified one or more clusters; and

provide a signal that causes the selected action.

22. The image based system of claim 21 wherein the instructions cause the processor to select a preferred non-object picking physical output action to disperse the one or more clusters based at least in part on a characteristic of the identified one or more clusters by generating a representation of an upper surface level of the one or more clusters of objects from the acquired image data, and determining at least one direction or axis of movement to adjust the upper surface level of the one or more clusters toward a defined upper surface level.

23. The image based system of claim 21, further comprising:

at least one robot selectively positionable to physically remove at least one of the objects from the area.

24. The image based system of claim 21 wherein the at least one robot is selectively positionable to cause movement of a support structure on which the one or more objects are carried, in accordance with the at least one signal.

25. The image based system of claim 21, further comprising:

a support structure on which the objects are carried; and

at least one actuator coupled to physically move the support structure in accordance with the at least one signal.