USER INTERFACE FOR MEDICAL ROBOTICS SYSTEM

• US 20150375399A1
• Filed: 05/21/2015
• Published: 12/31/2015
• Est. Priority Date: 06/27/2014
• Status: Abandoned Application

• Current Assignee: Auris Health, Inc. (Johnson & Johnson)
• Original Assignee: Hansen Medical Incorporated (Johnson & Johnson)
• Inventors: Walker, Sean P., Park, June, Balaji, Kamini, Chiu, Joanne
• Application Number:
  US14/719,034
• Publication Number:
  US 20150375399A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: A61B 2017/00207 with hand gesture control or hand gesture recognition
• CPC Class Codes: A61B 34/30 Surgical robots
• CPC Class Codes: B25J 9/1612 characterised by the hand, wrist, grip control
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: Y10S 901/09 Closed loop, sensor feedback controls arm movement

Abstract

An exemplary illustration of a user interface for a medical robotics system may include multiple light sources configured to illuminate a gesture within a field of view. The user interface may further include multiple cameras, which have a field of view and are configured to generate a detection signal in response to detecting the gesture within the field of view. The user interface can also have a controller configured to generate a command signal based on the detection signal. The command signal may be configured to actuate an instrument driver, a display device, a C-arm configured or any combination thereof to perform a function mapped to the gesture.

Claims

1. A user interface for a medical robotics system, the user interface comprising:

multiple light sources configured to illuminate a gesture within a field of view;

multiple cameras having the field of view and being configured to generate a detection signal in response to detecting the gesture within the field of view; and

a controller configured to generate a command signal based on the detection signal, wherein the command signal is configured to actuate at least one of an instrument driver, a display device, or a C-arm configured to perform a function mapped to the gesture.

2. The user interface of claim 1, wherein the multiple cameras are configured to generate the detection signal in response to detecting the gesture provided by a hand or a tool.

3. The user interface of claim 1, further comprising a non-transitory computer readable medium storing a reference lookup table that includes multiple reference command data mapped to multiple reference detection data, such that the controller generates the command signal based on reference command data corresponding with the detection signal.

4. The user interface of claim 3, wherein the multiple cameras are configured to generate the detection signal in response to detecting the gesture that is configured to move a virtual elongate member within the field of view in at least one of a rolling motion, an articulation motion, an insertion motion, or a retraction motion.

5. The user interface of claim 4, wherein the multiple reference detection data correspond with the gesture moving the virtual elongate member in at least one of the rolling motion, the articulation motion, the insertion motion, or the retraction motion.

6. The user interface of claim 5, wherein the multiple reference command data are configured to actuate the instrument driver to move an elongate member in at least one of the rolling motion, the articulation motion, the insertion motion, or the refraction motion.

7. The user interface of claim 3, wherein the multiple cameras are configured to generate the detection signal in response to detecting the gesture that is configured to move a virtual reference frame within the field of view to change a fluoroscopy view angle.

8. The user interface of claim 7, wherein the multiple reference detection data correspond with the gesture that is configured to move the virtual reference frame to change the fluoroscopy view angle.

9. The user interface of claim 8, wherein the multiple reference command data are configured to actuate the display device to change the fluoroscopy view angle.

10. The user interface of claim 3, wherein the multiple cameras are configured to generate the detection signal in response to detecting the gesture that is configured to move a virtual C-arm within the field of view.

11. The user interface of claim 10, wherein the multiple reference detection data correspond with the gesture that is configured to move the virtual C-arm.

12. The user interface of claim 11, wherein the multiple reference command data are configured to actuate the C-arm to move an imaging device carried by the C-arm.

13. The user interface of claim 1, wherein the multiple light sources comprise infrared LEDs, and wherein the multiple cameras comprise infrared cameras.

14. A medical robotics system, comprising:

a user interface, comprising;

multiple light sources;

multiple cameras;

a controller; and

a non-transitory computer readable medium; and

at least one of an instrument driver, a display device, or a C-arm configured to perform a function mapped to a command signal in response to receiving a command signal from the controller,

wherein the multiple light sources are configured to illuminate a gesture within a field of view;

wherein the multiple cameras are configured to generate a detection signal in response to detecting the gesture;

wherein the controller is configured to generate the command signal in response to receiving the detection signal from the multiple cameras; and

wherein the non-transitory computer readable medium stores a reference lookup table that includes multiple reference command data mapped to multiple reference detection data, such that the controller generates the command signal based on the multiple reference command data corresponding with the detection signal.

15. The system of claim 14, wherein the display device is a fluoroscope configured to display a fluoroscopy view angle in response to the command signal.

16. The system of claim 14, wherein the C-arm is configured to move an X-ray imaging device in response to the command signal.

17. The system of claim 14, wherein the controller is configured to generate the command signal to lock at least one of an instrument driver, a display device or a C-arm in a current position.

18. A method for operating a user interface for a medical robotics system, the method comprising:

illuminating a gesture within a field of view;

generating a detection signal in response to detecting the gesture within the field of view;

generating a command signal in response to receiving the detection signal; and

actuating at least one of an instrument driver, a display device, or a C-arm configured to perform a function mapped to the gesture in response to the detection signal.

19. The method of claim 18, further comprising determining the command signal based on matching the detection signal with a corresponding reference detection data and reference command data in a reference lookup table stored within a non-transitory computer readable medium.

20. The method of claim 18, further comprising one or more of:

forming a hand gesture configured in a C-shape to control a C-arm;

forming a hand gesture having a pair of pinching fingers to manipulate a virtual catheter;

forming a hand gesture having a pair of opposing cupped hands to change a view angle on the display device; and

forming a hand gesture having a flat open palm to disable or lock the user interface.