Vision-guided electromagnetic robotic system
First Claim
1. A computer-implemented method of detecting and manipulating a plurality of instruments, the computer including a computer processor, and a camera with a barcode reading software module;
- the method comprising the steps of;
creating a template image of each of the instruments by way of the camera and storing the template images in memory;
capturing an input image of the plurality of instruments;
identifying each of the instruments by using the barcode reading software module that (i) reads an identification tag from each of the instruments and (ii) localizes reference points of the identification tag, each of the identification tags encoding a unique identifier;
retrieving the template image from memory that corresponds to the identification tag;
aligning one or more edges of an instrument in the input image with edges of the instrument in the template image to compute an initial affine transformation;
computing an occupancy map that models one or more intersections between the instruments; and
synthesizing occlusion relationships that correspond to one or more hypotheses, wherein the input image is compared against at least a first hypothesis and a second hypothesis;
wherein the template image and the input image are two dimensional and utilized in combination to select a top-most instrument from a cluttered pile.
1 Assignment
0 Petitions
Accused Products
Abstract
Embodiments of the disclosure comprise a vision-based robotic manipulation system that identifies, singulates, and removes an individual surgical instrument from a cluttered environment. A computer-vision algorithm incorporated with the system is robust against optical challenges such as changing light conditions, specularities, and inter-reflections among various surgical instruments. The system estimates 2D pose (as opposed to a more challenging 3D pose) using a camera with normal room lighting to identify each object, including its specified data matrix barcode; and incorporates the use of a robotic arm with a compliant electromagnetic gripper to handle objects having large shape variance. The system can then relocate the objects as desired. The robot manipulator is utilized in hospital and research settings, manufacturing, and sterile environments.
-
Citations
22 Claims
-
1. A computer-implemented method of detecting and manipulating a plurality of instruments, the computer including a computer processor, and a camera with a barcode reading software module;
- the method comprising the steps of;
creating a template image of each of the instruments by way of the camera and storing the template images in memory; capturing an input image of the plurality of instruments; identifying each of the instruments by using the barcode reading software module that (i) reads an identification tag from each of the instruments and (ii) localizes reference points of the identification tag, each of the identification tags encoding a unique identifier; retrieving the template image from memory that corresponds to the identification tag; aligning one or more edges of an instrument in the input image with edges of the instrument in the template image to compute an initial affine transformation; computing an occupancy map that models one or more intersections between the instruments; and synthesizing occlusion relationships that correspond to one or more hypotheses, wherein the input image is compared against at least a first hypothesis and a second hypothesis; wherein the template image and the input image are two dimensional and utilized in combination to select a top-most instrument from a cluttered pile. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14)
- the method comprising the steps of;
-
15. A vision-guided robotic system for instrument singulation comprising:
-
a robotic arm; a camera comprising a barcode reading software module to identify and image the plurality of instruments; one or more instruments positioned in a clutter; a computer processor to implement the steps of; creating a template image of each of the instruments by way of the camera and storing the template images in memory; capturing an input image of the plurality of instruments; identifying each of the instruments by using the barcode reading software module that (i) reads an identification tag from each of the instruments and (ii) localizes reference points of the identification tag, each of the identification tags encoding a unique identifier; retrieving the template image from memory that corresponds to the identification tag; aligning one or more edges of an instrument in the input image with edges of the instrument in the template image to compute an initial affine transformation; computing an occupancy map that models one or more intersections between the instruments; and synthesizing occlusion relationships that correspond to one or more hypotheses, wherein the input image is compared against at least a first hypothesis and a second hypothesis; wherein the template image and the input image are two dimensional and utilized in combination to select a specific instrument from the clutter. - View Dependent Claims (16, 17, 18, 19, 20, 21, 22)
-
Specification