Systems and methods for aligning a tool with a desired location or object

• US 7,962,192 B2
• Filed: 04/28/2006
• Issued: 06/14/2011
• Est. Priority Date: 09/30/2005
• Status: Active Grant

• Current Assignee: Venus Concept, Inc.
• Original Assignee: Restoration Robotics, Inc. (Venus Concept, Inc.)
• Inventors: Gildenberg, Philip L., Caddes, Donald E., Bodduluri, Mohan
• Primary Examiner(s): Jung; Unsu
• Assistant Examiner(s): FONTENOT, NIGEL RAI
• Application Number:
  US11/380,907
• Publication Number:
  US 20070106306A1
• Time in Patent Office: 1,873 Days
• Field of Search: None
• US Class Current: 600/407
• CPC Class Codes: A61B 17/32053 Punch like cutting instruments, e.g. using a cylindrical or oval knife A61B17/326 takes precedence; trepans A61B17/1695; for use on the cornea A61F9/007
• CPC Class Codes: A61B 17/3468 for implanting or removing devices, e.g. prostheses, implants, seeds, wires devices for implanting seeds A61M37/0069
• CPC Class Codes: A61B 2017/00752 Hair removal or transplantation
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2090/064 for measuring force, pressure or mechanical tension
• CPC Class Codes: A61B 2090/363 Use of fiducial points
• CPC Class Codes: A61B 2090/371 with simultaneous use of two cameras
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/32 operating autonomously
• CPC Class Codes: A61B 34/70 Manipulators specially adapted for use in surgery
• CPC Class Codes: A61B 90/36 Image-producing devices or illumination devices not otherwise provided for
• CPC Class Codes: A61B 90/361 Image-producing devices, e.g. surgical cameras
• CPC Class Codes: A61M 37/0076 Tattooing apparatus apparatus for marking animals A01K11/00; vaccine applicators having needles or other puncturing means A61B17/205

Abstract

Systems and methods for aligning a tool positioned on a moveable arm with a location or an object of interest on a body surface. Some methods comprise using one or more cameras and identifying position offsets of the location or the object of interest from the one or more cameras. Some methods comprise using at least two cameras and rectifying the images from the first and the second cameras. A variety of different tools and assemblies may be located on the moveable arm for performing various diagnostic and therapeutic procedures that take advantage of the ability of the moveable arm to rapidly and precisely position the respective tool or assembly at desired locations on a skin.

Claims

1. A method for aligning a tool positioned on a moveable arm of an automated system with a location or an object of interest on a skin, comprising:

determining a fixed relationship between the tool positioned on the moveable arm and at least two cameras positioned on the movable arm, wherein the tool is capable of manipulating tissue;

selecting a location or an object of interest on a skin;

identifying position offsets of the location or the object of interest from the at least two cameras;

automatically moving the moveable arm to align the tool with the location or the object of interest on the skin based, at least in pail, on the identified offsets and on maintaining while aligning the tool the fixed relationship between the at least two cameras and the tool.

2. The method of claim 1, wherein identifying position offsets comprises

determining whether the at least two cameras are aligned with the location or object of interest within acceptable tolerances, and

moving the moveable arm, if the at least two cameras are not aligned with the location or object of interest within acceptable tolerances, until the at least two cameras are aligned with the location or object of interest within acceptable tolerances.

3. The method of claim 1, further comprising determining whether the tool is aligned with the location or the object of interest within acceptable tolerances, and repeating the acts of identifying position offsets and moving the movable arm, if the tool is not aligned with the location or object of interest within acceptable tolerances, until the tool is aligned with the location or object of interest within acceptable tolerances.

4. The method of claim 1, further comprising continually calculating and adjusting position and orientation of the moveable arm as the tool is moved into alignment with the location or the object of interest.

5. The method of claim 1, wherein the object of interest has certain rotational orientation in 3D, and wherein the method comprises identifying 3D orientation offsets of the object of interest.

6. The method of claim 5, wherein the object of interest is a follicular unit and the tool is a hair removal tool, and wherein aligning the tool comprises aligning an elongate axis of the hair removal tool with an elongate axis of the follicular unit.

7. The method of claim 1, wherein the location or the object of interest is selected from the group comprising an injection site, a tattoo placement or removal site, a tissue removal site, a follicular unit, a wrinkle line, a wart, and a mole.

8. The method of claim 6, further comprising operating the hair removal tool to remove the follicular unit.

9. The method of claim 1, wherein the tool comprises an elongated axis and each of the at least two cameras comprises a respective axis, and wherein the tool and the at least two cameras are positioned such that their respective axes are parallel to each other.

10. The method of claim 1, wherein the tool comprises an elongated axis and each of the at least two cameras comprises a respective axis, wherein the tool and the at least two cameras are positioned such that their respective axes are not parallel to each other.

11. An automated system for aligning a tool with a location or an object of interest on a skin, comprising:

a moveable arm;

a tool capable of manipulating tissue and positioned on the moveable arm;

at least two cameras positioned on the moveable arm; and

a processor configured for

processing images of a location or an object of interest on a skin obtained from the at least two cameras,

determining a fixed relationship between the at least two cameras and the tool,

identifying position offsets of the location or the object of interest from the at least two cameras, and

causing the moveable arm to be moved to align the tool with the location or the object of interest based, at least in part, on the identified offsets and on maintaining while aligning the tool the fixed relationship between the at least two cameras and the tool.

12. The system of claim 11, further comprising a controller operatively associated with the processor and configured to maneuver the moveable arm.

13. The system of claim 11, wherein the tool is selected from the group comprising a laser, an RF energy device, an ultrasound device, a cryogenic device, an injection needle, a tissue removal tool, a needle assembly, a cannula, a hair removal tool, and a hair implantation tool.

14. The system of claim 11, wherein the automated system is a robotic system and the moveable arm is a robotic arm.

15. The system of claim 11, wherein the tool comprises a harvesting tool and an implantation tool as an integrated assembly.

16. The system of claim 11, further comprising an air jet configured for directing an air stream at the skin.

17. The system of claim 11, wherein each camera of the at least two cameras having an optical axis and respective camera optical axes are not parallel to each other.

18. An image processor for aligning a tool capable of manipulating tissue and positioned on a moveable arm with an object or location on a skin, the image processor configured for:

processing images of a location or an object of interest on a skin obtained from at least two cameras located on a moveable arm;

determining a fixed relationship between the at least two cameras and the tool;

identifying position offsets of the location or the object of interest from the at least two cameras;

causing a controller that is operatively associated with the processor to move the moveable arm to align the tool with the location or the object of interest based, at least in part, on the identified offsets and on maintaining while aligning the tool the fixed relationship between the at least two cameras and the tool.

19. The image processor of claim 18, wherein the object of interest is at least one hair follicle, and the image processor further configured for identifying 3D orientation offsets of the at least one hair follicle.

20. The image processor of claim 18, further configured for systematically and repeatedly aligning the tool with a plurality of locations or objects of interest on the skin.

21. A method for aligning a tool with a location or an object of interest on a skin, comprising:

determining a fixed relationship between a tool capable of manipulating tissue and a first camera and a second camera;

determining a position of a location or an object of interest on a skin, comprising;

obtaining an image of the location or the object of interest on the skin from the first camera and the second camera;

rectifying the respective images from the first and the second cameras;

identifying a center of the location or the object of interest in the rectified images;

matching the center of the location or the object of interest in the rectified images;

calculating three dimensional coordinates of the center of the location or the object of interest; and

aligning the tool with the location or the object of interest on the skin based, at least in part, on

     1) the determined position of the location or the object of interest on the skin and

     2) maintaining while aligning the tool the fixed relationship between the tool and the first and the second cameras.

22. The method of claim 21, wherein the tool, the first camera and the second camera are located on a moveable arm so that coordinates of the tool are fixed relative to the coordinates of the first and second cameras.

23. The method of claim 21, wherein the object of interest is a hair follicle and wherein the method further comprises matching a center of the hair follicle in the rectified images from the first and the second cameras.

24. The method of claim 21, wherein the object of interest is a hair follicle and wherein the method further comprises

finding a head and a tail of the hair follicle in the respective images of the first camera and the second camera, and

calculating three dimensional coordinates of the head and the tail of the hair follicle.

25. The method of claim 24, wherein the hair follicle has a rotational orientation in 3D and the method further comprises calculating a relative offset between the tool and the position and orientation of the hair follicle.

26. The method of claim 21, further comprising calibrating the first and the second cameras to identify intrinsic and extrinsic parameters of the cameras.

27. An automated system for aligning a tool with a location or an object of interest on a skin, comprising:

a moveable arm;

a tool capable of manipulating tissue and positioned on the moveable arm;

a first camera and a second camera; and

a processor configured for

determining a fixed relationship between the tool and the first and second cameras;

determining a position of a location or an object of interest on a skin, comprising;

rectifying images of the location or the object of interest on the skin obtained from the first and the second cameras,

identifying a center of the location or the object of interest in the rectified images,

matching the center of the location or the object of interest in the rectified images;

calculating three dimensional coordinates of the center of the location or the object of interest; and

aligning the tool with the location or the object of interest based, at least in part, on

     1) the determined position of the location or the object of interest on the skin and

     2) maintaining while aligning the tool the fixed relationship between the tool and the first and second cameras.

28. The system of claim 27, wherein the tool is selected from the group comprising a laser, an RF energy device, an ultrasound device, a cryogenic device, an injection needle, a needle assembly, a cannula, a hair removal tool, a hair implantation tool, and a tissue removal tool.

29. The system of claim 27, wherein the automated system is a robotic system and the moveable arm is a robotic arm.

30. The system of claim 27, wherein the tool comprises a harvesting tool and an implanting tool as an integrated assembly.

31. The system of claim 27, further comprising a controller operatively associated with the processor and configured to maneuver the moveable arm.

32. An image processor for aligning a tool with an object or location on a skin, the image processor configured for:

determining a fixed relationship between a tool capable of manipulating tissue and a first camera and a second camera;

determining a position of a location or an object of interest on a skin, comprising;

receiving and processing images of the location or the object on the skin obtained from the first and the second camera;

rectifying the images obtained from the first and the second cameras;

identifying a center of the location or the object of interest in the rectified images;

matching the center of the location or the object of interest in the rectified images;

calculating three dimensional coordinates of the center of the location or the object of interest; and

aligning the tool with the location or the object of interest based, at least in part, on the determined position of the location or the object of interest on the skin and on maintaining while aligning the tool the fixed relationship between the tool and the first and second cameras.