Systems and methods for identifying and tracking physical objects during a robotic surgical procedure

• US 10,531,926 B2
• Filed: 05/23/2017
• Issued: 01/14/2020
• Est. Priority Date: 05/23/2016
• Status: Active Grant

• Current Assignee: MAKO Surgical Corporation (Stryker Corporation)
• Original Assignee: MAKO Surgical Corporation (Stryker Corporation)
• Inventors: Roessler, Patrick
• Primary Examiner(s): Woldemariam, Aklilu K
• Application Number:
  US15/602,621
• Publication Number:
  US 20170333137A1
• Time in Patent Office: 966 Days
• Field of Search: 382128, 382130, 382131, 382264, 600414, 600407, 600426, 600424, 600427, 600417, 345156, 345634
• CPC Class Codes: A61B 17/0206 with antagonistic arms as supports for retractor elements
• CPC Class Codes: A61B 17/1703 using imaging means, e.g. by X-rays
• CPC Class Codes: A61B 2034/102 Modelling of surgical devices, implants or prosthesis
• CPC Class Codes: A61B 2034/105 Modelling of the patient, e.g. for ligaments or bones
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2034/2048 using an accelerometer or inertia sensor
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2034/2057 Details of tracking cameras
• CPC Class Codes: A61B 2034/2059 Mechanical position encoders
• CPC Class Codes: A61B 2034/2065 Tracking using image or pattern recognition
• CPC Class Codes: A61B 2034/2068 using pointers, e.g. pointers having reference marks for determining coordinates of body points
• CPC Class Codes: A61B 2034/207 Divots for calibration
• CPC Class Codes: A61B 2034/2072 Reference field transducer attached to an instrument or patient
• CPC Class Codes: A61B 2090/3612 with images taken automatically
• CPC Class Codes: A61B 34/10 Computer-aided planning, simulation or modelling of surgical operations
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/32 operating autonomously
• CPC Class Codes: A61B 34/76 Manipulators having means for providing feel, e.g. force or tactile feedback
• CPC Class Codes: A61B 90/90 Identification means for patients or instruments, e.g. tags
• CPC Class Codes: B25J 9/1676 : Avoiding collision or forbidden zones
• CPC Class Codes: B25J 9/1697 : Vision controlled systems

Abstract

Navigation systems and methods for tracking physical objects near a target site during a surgical procedure. The navigation system comprises a robotic device and an instrument attached to the robotic device. A vision device is attached to the robotic device or the instrument and generates vision data sets. The vision data sets are captured from multiple perspectives of the physical object. A computing system associates a virtual object with the physical object based on one or more features of the physical object identifiable in the vision data sets. The virtual object at least partially defines a virtual boundary for the instrument.

Claims

1. A navigation system for tracking a physical object, said navigation system comprising:

a robotic device;

an instrument attachable to said robotic device, the instrument being configured to treat tissue at a target site;

a vision device attached to one of said robotic device or said instrument such that said vision device is movable with said robotic device, said vision device configured to generate vision data sets, wherein said vision data sets are captured from multiple perspectives of the physical object enabled by said vision device being moved in a plurality of degrees of freedom when moving said robotic device, and the physical object being at least one of the target site and an object to be avoided at or near the target site; and

a computing system in communication with said vision device, said computing system having at least one processor configured to associate a virtual object with the physical object based on one or more features of the physical object identifiable in said vision data sets, wherein the virtual object at least partially defines a virtual boundary utilized to provide a constraint on movement of the instrument relative to the physical object.

2. The navigation system as set forth in claim 1, wherein said vision device comprises a camera.

3. The navigation system as set forth in claim 1, wherein said computing system is configured to track a position and orientation of said vision device at each location of said vision device in which said vision data sets are generated.

4. The navigation system as set forth in claim 1, wherein said computing system is configured to determine an identity of the physical object based on said one or more features of the physical object.

5. The navigation system as set forth in claim 4, wherein said computing system is configured to associate said virtual object with the physical object based on the identity of the physical object.

6. The navigation system as set forth in claim 5, wherein said virtual object comprises one or more of a three dimensional model, a two dimensional surface, a point cloud, a voxelized volume, a surface mesh, a plane, a line, or a single point.

7. The navigation system as set forth in claim 1, wherein said one or more features includes a set of features identifiable in said vision data sets and said computing system is configured to determine a position and orientation of the set of features to establish a position and orientation of a first coordinate system associated with the physical object.

8. The navigation system as set forth in claim 7, wherein said vision device is fixed with respect to said one of said robotic device or said instrument so that said vision data sets are transformable into a second coordinate system of said one of said robotic device or said instrument.

9. The navigation system as set forth in claim 8, wherein said computing system includes a coordinate transformer configured to transform one of said coordinate systems so that said computing system can track movement of said instrument relative to the physical object.

10. The navigation system as set forth in claim 9, wherein said computing system is configured to enable movement of said instrument to treat the tissue at the target site while avoiding the physical object by tracking movement of said instrument with respect to the physical object.

11. The navigation system as set forth in claim 1, wherein said computing system is configured to determine information associated with the physical object based on the one or more features.

12. The navigation system as set forth in claim 11, wherein the information comprises one or more of physical object identification, physical object type, physical object size, physical object dimensions, physical object serial number, physical object manufacturer, virtual object identification, virtual object type, virtual object size, or virtual object dimensions.

13. The navigation system as set forth in claim 1, wherein said virtual object is defined by at least one of a navigation pointer or said instrument.

14. The navigation system as set forth in claim 1, wherein said robotic device is configured to operate in an autonomous mode so that said vision device moves autonomously when said robotic device operates in said autonomous mode.

15. A method for tracking a physical object utilizing a robotic device, an instrument attachable to the robotic device and the instrument being configured to treat tissue at a target site, a vision device, and a computing system having at least one processor, and the physical object being at least one of the target site and an object to be avoided at or near the target site, the method comprising the steps of:

generating vision data sets in response to movement of the vision device resulting from movement of the robotic device, wherein the vision data sets are captured from multiple perspectives of the physical object; and

associating a virtual object with the physical object based on one or more features of the physical object identifiable in the vision data sets, wherein the virtual object defines a virtual boundary utilized for providing a constraint on movement of the instrument relative to the physical object.

16. The method as set forth in claim 15, wherein the vision device comprises a camera.

17. The method as set forth in claim 15, comprising tracking a position and orientation of the vision device at each location of the vision device in which the vision data sets are generated.

18. The method as set forth in claim 15, comprising determining an identity of the physical object based on the one or more features of the physical object.

19. The method as set forth in claim 18, comprising associating the virtual object with the physical object based on the identity of the physical object.

20. The method as set forth in claim 19, wherein the virtual object comprises one or more of a three dimensional model, a two dimensional surface, a point cloud, a voxelized volume, a surface mesh, a plane, a line, or a single point.

21. The method as set forth in claim 15, comprising determining a position and orientation of a set of features identifiable in the vision data sets to establish a position and orientation of a first coordinate system associated with the physical object.

22. The method as set forth in claim 21, comprising transforming the vision data sets into a second coordinate system of one of the robotic device or the instrument.

23. The method as set forth in claim 22, comprising transforming one of the coordinate systems so that the computing system can track movement of the instrument relative to the physical object.

24. The method as set forth in claim 23, comprising enabling movement of the instrument to the treat tissue at the target site while avoiding the physical object by tracking movement of the instrument with respect to the physical object.

25. The method as set forth in claim 15, comprising determining information associated with the physical object based on the one or more features.

26. The method as set forth in claim 25, wherein the information comprises one or more of physical object identification, physical object type, physical object size, physical object dimensions, physical object serial number, physical object manufacturer, virtual object identification, virtual object type, virtual object size, or virtual object dimensions.

27. The method as set forth in claim 15, comprising defining the virtual object with at least one of a navigation pointer or the instrument.

28. The method as set forth in claim 15, comprising autonomously moving the robotic device so that the vision device moves autonomously to capture the vision data sets.