SYSTEM AND METHOD FOR CONTROLLING A REMOTE MEDICAL DEVICE GUIDANCE SYSTEM IN THREE-DIMENSIONS USING GESTURES

• US 20130096575A1
• Filed: 12/03/2012
• Published: 04/18/2013
• Est. Priority Date: 07/22/2009
• Status: Active Grant

• Current Assignee: St. Jude Medical (Abbott Laboratories Incorporated)
• Original Assignee: St. Jude Medical (Abbott Laboratories Incorporated)
• Inventors: Olson, Eric S.
• Granted Patent: US 9,439,736 B2
• Time in Patent Office: Days
• US Class Current: 606/130
• CPC Class Codes: A61B 2034/258 providing specific settings for specific users
• CPC Class Codes: A61B 2034/741 Glove like input devices, e.g. "data gloves"
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: A61B 34/74 Manipulators with manual electric input means
• CPC Class Codes: A61B 34/76 Manipulators having means for providing feel, e.g. force or tactile feedback
• CPC Class Codes: A61B 5/283 Invasive
• CPC Class Codes: B25J 9/161 Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
• CPC Class Codes: G06F 3/014 Hand-worn input/output arrangements, e.g. data gloves
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/03545 Pens or stylus
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16

Abstract

A system for enabling a user to remotely control a robotic medical device system includes a motion capture apparatus to capture motion of a user in a sensing volume and generate indicative output data. The system includes a control unit configured to execute gesture recognition logic that recognizes a user gesture based on analysis of the indicative output data. The control unit executes interpreter logic that is configured to translate the recognized user gesture into a corresponding robotic medical device control command configured to control an aspect of the operation of the robotic medical device system.

Claims

1. A system for enabling a user to remotely control a robotic medical device system, comprising:

a motion capture apparatus configured to capture motion of a user in a sensing volume and generate output data indicative of the captured user motion;

an electronic control unit including a processor and a memory;

gesture recognition logic stored in said memory and configured to execute on said processor, said gesture recognition logic being configured to recognize a user gesture based on said output data from said motion capture apparatus; and

interpreter logic stored in said memory and configured to execute said processor, said interpreter logic being configured to translate the user gesture to a corresponding robotic medical device control command wherein said command is configured to control an aspect of the operation of the robotic medical device system, said electronic control unit being configured to communicate said command to the robotic medical device system.

2. The system of claim 1 wherein said output data includes fiducial point tracking data.

3. The system of claim 2 wherein said fiducial point tracking data is associated with a plurality of fiducial points defined with respect to the user, and wherein said fiducial point tracking data includes, for each fiducial point, a respective position.

4. The system of claim 3 wherein each position includes a respective three-dimensional coordinate in a reference coordinate system, and wherein said fiducial point tracking data further includes, for each fiducial point, a respective time-based plurality of positions.

5. The system of claim 4 wherein said gesture recognition logic is configured to:

identify a start pose based on said fiducial point tracking data;

record the motion of a predetermined plurality of fiducial points after the start pose until an end pose is identified based on said fiducial point tracking data;

compare said recorded motion of the predetermined plurality of fiducial points with a plurality of predefined gestures; and

output the user gesture when said recorded motion matches one of the plurality of gestures.

6. The system of claim 5 wherein the start pose corresponds to a start condition where a first set of fiducial points assumes a first relationship therebetween, and the end pose corresponds to an end condition where a second set of fiducial points assumes a second relationship therebetween.

7. The system of claim 1 further comprising a user-actuatable switch coupled to said electronic control unit and having a normally open state, and a user-actuatable closed state, wherein said electronic control unit is configured to at least disable said communication of said command to the robotic medical device system unless said switch is in said closed state.

8. The system of claim 1 wherein said interpreter logic is further configured to selectively translate, based on a state of context switch parameter, the user gesture into one of (i) the robotic medical device control command configured to control an aspect of the operation of the robotic medical device system and (ii) a mapping control command configured to control an aspect of an electro-anatomic mapping system.

9. The system of claim 8 wherein the sensing volume includes a context switching portion, said system for enabling the user to remotely control further comprising context switching logic stored in said memory and configured for execution by said processor and configured to detect a predetermined context switching gesture based on said output data from said motion capture apparatus originating with user-motion occurring in the context switching portion of the sensing volume.

10. The system of claim 3 wherein a characteristic associated said robot catheter control command is a commanded magnitude associated with an action involving one of a catheter and a sheath under control of the robotic medical device system, and wherein the commanded magnitude corresponds to a distance between preselected fiducial points.

11. The system of claim 10 wherein the action is selected from the group comprising a catheter extension, a catheter retraction, a sheath extension, and a sheath retraction.

12. The system of claim 3 wherein a characteristic associated with the robot medical device control command is a commanded rotation associated with an action involving one of a catheter and a sheath under control of the robotic medical device system, and wherein the commanded rotation corresponds to a rotation angle through which a preselected fiducial point is rotated during the user gesture.

13. The system of claim 1 wherein said output data includes acquired imaging of the movements of the user, said gesture recognition logic being configured to determine the gesture from at least said acquired imaging.

14. The system of claim 8 said interpreter logic is further configured to generate said robotic medical device system command with respect to the visible orientation of the then-visible view of an anatomical model of at least a portion of a body of a patient produced by the mapping system.

15. The system of claim 1 wherein said motion capture apparatus comprises an optical sub-system configured to optically detect the motion of the user in the sensing volume.

16. The system of claim 1 wherein said motion capture apparatus comprises one of (i) a spatially detected glove configured to be worn by the user, (ii) a spatially detected wand configured to be moved by the user, and (iii) a spatially detected finger instrument configured to be worn on a finger of the user.

17. A system for enabling a user to remotely control a robotic medical device system, comprising:

motion capture means for capturing motion of a user in a sensing volume and generating output data indicative of the captured user motion;

gesture recognition means for recognizing a user gesture based on said output data from said motion capture means;

interpreter means for translating the user gesture to a corresponding robotic medical device control command wherein said command is configured to control an aspect of the operation of the robotic medical device system; and

communication means for communicating said command to the robotic medical device system.

18. The system of claim 17 wherein said output data includes fiducial point tracking data associated with a plurality of fiducial points defined with respect to the user, and wherein said fiducial point tracking data includes, for each fiducial point, a respective position, wherein each position includes a respective three-dimensional coordinate in a reference coordinate system, and wherein said fiducial point tracking data further includes, for each fiducial point, a respective time-based plurality of positions.

19. The system of claim 18 wherein said gesture recognition means is further configured for:

identifying a start pose based on said fiducial point tracking data;

recording the motion of a predetermined plurality of fiducial points after the start pose until an end pose is identified based on said fiducial point tracking data;

comparing said recorded motion of the predetermined plurality of fiducial points with a plurality of predefined gestures; and

outputting the user gesture when said recorded motion matches one of the plurality of gestures.

20. An article of manufacture, comprising:

a computer-readable storage medium having a computer program encoded thereon that, when executed by an electronic control unit, enables a user to remotely control a robotic medical device system, the computer program including code for;

capturing, using a motion capture apparatus, the motion of a user in a sensing volume and generating output data indicative of the captured user motion;

recognizing a user gesture based on said output data from the motion capture apparatus;

translating the user gesture to a corresponding robotic medical device control command wherein said command is configured to control an aspect of the operation of the robotic medical device system; and

communicating said command to the robotic medical device system.