Robotic control system for needle sorting and feeder apparatus

• US 6,115,650 A
• Filed: 01/30/1998
• Issued: 09/05/2000
• Est. Priority Date: 04/30/1997
• Status: Expired due to Term

• Current Assignee: Ethicon Incorporated (Johnson & Johnson)
• Original Assignee: Ethicon Incorporated (Johnson & Johnson)
• Inventors: Shaw, Teresa, Demarest, David D.
• Primary Examiner(s): Grant, William
• Assistant Examiner(s): Garland, Steven R.
• Application Number:
  US09/016,455
• Time in Patent Office: 949 Days
• Field of Search: 700/245, 700/247, 700/259, 700/59, 700/62, 700/56, 700/112-114, 700/213, 700/230, 382/153, 901/7, 901/9, 901/47, 163/5, 414/754, 414/758, 414/783
• US Class Current: 700/259
• CPC Class Codes: A61B 17/06004 Means for attaching suture to needle tipping A61B17/06195; connecting wire to other metallic objects B21F15/00
• CPC Class Codes: A61B 17/06133 of parallelepipedal shape, e.g. made of rectangular or slightly oval panels
• CPC Class Codes: A61B 2017/00526 Methods of manufacturing
• CPC Class Codes: B21G 1/08 of hollow needles or needles with hollow end, e.g. hypodermic needles, larding-needles B21G1/10 takes precedence
• CPC Class Codes: B25J 9/1697 Vision controlled systems
• CPC Class Codes: G05B 19/4182 manipulators and conveyor only
• CPC Class Codes: G05B 2219/37555 Camera detects orientation, position workpiece, points of workpiece
• CPC Class Codes: G05B 2219/45063 Pick and place manipulator

Abstract

A control system for a needle singulation and infeed apparatus that automatically picks up surgical needles in random, un-oriented positions on a transport conveyor or like device, and places them in oriented positions for suture attachment at a fully automated needle swaging station. The apparatus employs one or more robot devices for picking up the needles and placing them in individual precision engagement devices for sequential conveyance to an automatic swaging machine. After gripping the needle, and, depending upon the orientation of the needle with respect to the robot gripper, the robot gripper can pick the needle and move to a location where a mechanical finger is fixedly positioned. The robot is programmed to move to the mechanical finger and tap the needle held by the robot gripper against the finger to orient or rotate the needle in the proper direction prior to placing the needle in the precision engagement device. This, obviates the need for a further needle orienting step downstream of the robot pick and place location and ensures the high-speed transport of highly oriented needles for the automatic swaging operation.

Claims

1. A control system for a needle infeed apparatus having first conveyor means with singulated needles positioned randomly thereon for conveyance from a first location toward a second location, said apparatus having one or more robot devices each having a gripper means for picking said needles from said first conveyor means and placing said needles on a precise engagement device, said control system comprising:

(a) control means for pausing said first conveyor means to create a dwell cycle for said infeed apparatus;

(b) at least one vision tracking means in communication with said control means for generating an image of a selected needle at a predetermined location on said first conveyor means during said dwell cycle and for calculating positional and orientation data for each needle from said generated image;

(c) memory means for temporarily storing said positional and orientation data received from said vision tracking means; and

,

(d) robot control means for accessing said stored positional and orientation data of a selected said imaged needle and enabling a gripper device of one of said one or more robots to pick up said imaged needle in accordance with its respective positional and orientation data and place said needle in a said precision engagement device, wherein based on said orientation of said needle, said robot control means further enabling said robot gripper means to move to a needle orienting location having a needle orientation device located thereat to enable said needle to contact a needle orientation device and rotate said needle in said gripper means prior to placing said needle in said precision engagement device.

2. The control system for a needle infeed apparatus as claimed in claim 1, wherein said needle infeed apparatus includes a drive means under control of said control means for driving said second conveyor means, said robot control-means generating a first signal for receipt by said control means to enable said drive means to index said conveyor means.

3. The control system for a needle infeed apparatus as claimed in claim 2, wherein said control means generates a first signal for receipt by said vision tracking means indicating that said conveyor means has finished indexing and is in said dwell period.

4. The control system for a needle infeed apparatus as claimed in claim 2, wherein said robot control means automatically polls said memory means to obtain current positional and orientation data of each recognized needle for said one or more robots, said robot control means generating said first signal for receipt by said control means to further index said conveyor means when no positional and orientation data is available in said memory means at a current dwell period.

5. The control system for a needle infeed apparatus as claimed in claim 1, wherein said vision tracking means includes one or more camera means for obtaining a video image of said needles on said conveyor means at said predetermined location within a field-of-view of each of said one or more cameras.

6. The control system for a needle infeed apparatus as claimed in claim 5, wherein each of said cameras has a plurality of vision parameters associated therewith, said vision tracking means including means for comparing needle parameters obtained from said video image with one or more acceptable needle parameters associated with a current batch of needles to be processed, said one or more needle parameters selected from the group including needle radius, needle angle, and needle width.

7. The control system for a needle infeed apparatus as claimed in claim 1, wherein said vision tracking means further includes means for automatically enhancing said image of a needle by recording successive images of said needle and adjusting one or more of a plurality of vision parameters between each successive image until said image of said needle is acceptable for obtaining positional coordinate data therefrom.

8. The control system for a needle infeed apparatus as claimed in claim 7, wherein said vision parameters include field of view size, iris control for said camera and vision system lighting control for said camera.

9. The control system for a needle infeed apparatus as claimed in claim 1, wherein said control means includes means for determining a barrel end and a point end of said needle, said orientation data including data indicating location of said barrel end of said selected needle for downloading to said robot control means.

10. The control system for a needle infeed apparatus as claimed in claim 1, wherein said means for determining a barrel end includes means for determining an amount of taper at each end of said needle, said barrel end being an end with less taper than said other end.

11. The control system for a needle infeed apparatus as claimed in claim 10, wherein said precise engagement device is located on a second conveyor means having a drive means associated therewith for indexing said needle to said processing location, said robot control means generating a signal for receipt by said control means requesting said control means to pause said indexing of said second conveyor means to create a second dwell period.

12. The control system for a needle infeed apparatus as claimed in claim 11, wherein said control means generates a signal for receipt by said robot control means indicating that said robot gripper means may place said needle in said engagement apparatus during said second dwell period.

13. The control system for a needle infeed apparatus as claimed in claim 11, wherein said robot control means generates a signal for receipt by said control means indicating that said one or more robots have placed said needles in said engagement apparatus, said control means enabling first and second jaws of said engagement apparatus to grip said needles placed therein.

14. A method for controlling an automatic infeed apparatus for feeding surgical needles from one location to another location, said infeed apparatus having a first conveyor having randomly positioned needles located thereon, a second conveyor having a plurality of precision needle engagement devices located thereon, and one or more robot means each having a gripper means for picking up a needle from said indexing conveyor, said method comprising the steps of:

(a) pausing said first conveyor to create a dwell time for said infeed apparatus;

(b) imaging said needles on said first conveyor with a vision tracking means during said dwell time to create an image of said needles;

(c) calculating positional and orientation data from said image of a selected needle and determining a barrel end and a point end for said selected needle;

(d) locating said gripper means at said barrel end of said needle;

(e) determining a location of said needle point end with respect to said located gripper means;

(f) picking up said selected needle from said first conveyor;

(g) moving said gripper means gripping said needle towards a needle orienting device when a first orientation of said needle point end is determined at step (e);

(h) rotating said needle while gripped by said needle gripper; and

,

(i) placing each needle in said precision engagement device for subsequent conveyance thereof.

15. The method according to claim 14, wherein step (f) further includes the step of placing each needle in said precision engagement device for subsequent conveyance thereof when a second orientation of said needle point end is determined at step (e).

16. The method for controlling an automatic infeed apparatus for feeding surgical needles from one location to another location according to claim 14, wherein said step (a) of pausing said first conveyor further includes the step of generating a first control signal from said robot means requesting a control means for said first conveyor to inhibit motion thereof during said dwell time.

17. The method according to claim 16, wherein said processing step (c) further includes the steps of:

determining one or more needle parameter values for needles identified in said image, said one or more needle parameters selected from the group including needle radius, needle angle, and needle width; and

comparing each of said needle parameters values obtained from said image with a predetermined range of acceptable needle parameter values associated with a current batch of needles being processed.

18. The method according to claim 14, further including the step of generating a second control signal from said robot means for said control means requesting said control means to index said first conveyor means when no acceptable needle locations are available.

19. The method according to claim 14, wherein said step (b) of visually tracking said needles on said conveyor during said dwell time to determine acceptable needle locations for said one or more robot means further comprises the steps of:

generating a signal for said vision tracking means indicating that said first conveyor is in said dwell cycle;

obtaining an image of said needles from one or more camera means each having a field of view at one or more predetermined locations on said conveyor means;

processing said image to determine positional coordinates for recognizable needles present in said image; and

,

inputting said positional coordinates into a memory means for access by said robot means.

20. The method according to claim 14, wherein said step (i) of placing each said needles in an engagement apparatus further includes the steps of:

pausing said second conveyance having said needle engagement device located thereon to create a second dwell time for said infeed system; and

,

generating a control signal indicating to said robot gripper means to place a gripped needle in said needle engagement apparatus during said second dwell time.

21. The method according to claim 20, further including the step of generating a signal for enabling a pair of jaws of said engagement apparatus to grip said needle after placement therein by said robot gripper means.

22. The method according to claim 20, further including the step of actuating a push rod means for retracting one jaw of said pair of engagement jaws for enabling the positioning of said needle therebetween, said actuating step occurring prior to the placement of said needle between said pair of engagement jaws.

23. The method according to claim 14, wherein said step (c) of determining a barrel end and a point end for said selected needle, includes the step of determining an amount of taper at each end of said needle, said barrel end being an end with less taper than said other end.

24. A control system for a surgical needle infeed apparatus having first conveyor means with singulated needles positioned randomly thereon for conveyance from a first location toward a second location, said apparatus having one or more robot devices located intermediate said first and second locations, each robot having a gripper means for picking said needles from said first conveyor means and placing said needles on a second conveyor means for conveyance to said second location, said control system comprising:

(a) control means for pausing said first conveyor means to create a dwell cycle for said infeed apparatus;

(b) at least one vision tracking means in communication with said control means for obtaining an image of a selected needle on said first conveyor means during said dwell cycle, said vision tracking means comprising;

i) means for calculating a first set of data representing a location of a barrel end of said surgical needle from said image; and

,

ii) means for calculating a second set of data representing rotational orientation of said surgical needle on said conveyor from said image;

(c) means for inputting said first and second sets of data to said robot device; and

,

(d) means for enabling said gripper device of one of said one or more robots to pick up said imaged needle in accordance with said first and second sets of data and place said needle directly in said second conveyor means when a first condition is satisfied, and further enabling said gripper device to move to a third location to enable said needle to rotate said needle in said gripper means prior to placing said needle on said second conveyor means when a second condition is satisfied.

25. The control system for a surgical needle infeed apparatus as claimed in claim 24, wherein said first condition is satisfied when said needle is rotationally oriented on said first conveyor in a manner advantageous for said robot device to pick and directly place said needle on to said second conveyor at a first orientation.

26. The control system for a surgical needle infeed apparatus as claimed in claim 25, wherein said second condition is satisfied when said needle is rotationally oriented on said first conveyor in a manner that would enable said robot device to pick and directly place said needle on to said second conveyor in a second orientation.

27. The control system for a surgical needle infeed apparatus as claimed in claim 24, further including a drive means for controlling movement of said first and second conveyor means, said robot device instructing said drive means to pause said first conveyor during said dwell period to enable gripping of said needle thereon, and further enabling said drive means to pause said second conveyor for a dwell period to enable direct needle placement thereon.

28. The control system for a surgical needle infeed apparatus as claimed in claim 24, wherein said needle includes first and second endpoints, said means for calculating said first set of data includes:

means for determining the location of first and second endpoints of said needle;

determining an amount of taper of each said needle endpoint;

comparing said amount of taper determined for each said needle endpoint, and;

determining a barrel end for said needle based on said comparison.

29. The control system for a surgical needle infeed apparatus as claimed in claim 28, wherein said barrel end is determined to be an endpoint having the smallest amount of taper.

30. The control system for a surgical needle infeed apparatus as claimed in claim 24, wherein said vision tracking means includes a vision camera having positional coordinates defined in a field of view, said means for calculating said second set of data includes:

means for determining location of both endpoints in said vision camera'"'"'s field of view and locations of needle edges;

means for calculating a predetermined location associated with said needle;

means for computing a first line capable of intersecting a said endpoint and said predetermined location;

means for computing a second line of predetermined length and predetermined direction from said first line;

means for determining whether said second line intersects a needle edge,

wherein said rotational orientation is determined according to whether said second line intersects a needle edge;

said location of said predetermined.

31. The control system for a surgical needle infeed apparatus as claimed in claim 30, wherein said surgical needle has a plurality of needle parameters associated therewith, said needle parameters including needle radius, needle angle, and needle width.

32. The control system for a surgical needle infeed apparatus as claimed in claim 31, wherein said predetermined length of said second line is determined in accordance with said needle parameters.