SYSTEM AND METHOD FOR VIRTUALLY TRACKING A SURGICAL TOOL ON A MOVABLE DISPLAY

• US 20130072787A1
• Filed: 09/14/2012
• Published: 03/21/2013
• Est. Priority Date: 09/16/2011
• Status: Active Grant

• Current Assignee: Auris Health, Inc. (Johnson & Johnson)
• Original Assignee: Translucent Medical, Inc.
• Inventors: Grogan, Aaron, Wallace, Dan, Stahler, Greg, Waydo, Jaime
• Granted Patent: US 9,918,681 B2
• Time in Patent Office: Days
• US Class Current: 600/424
• CPC Class Codes: A61B 2034/2048 using an accelerometer or inertia sensor
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 2034/2059 Mechanical position encoders
• CPC Class Codes: A61B 2090/365 augmented reality, i.e. correlating a live optical image with another image
• CPC Class Codes: A61B 2090/372 Details of monitor hardware
• CPC Class Codes: A61B 2090/376 using X-rays, e.g. fluoroscopy
• CPC Class Codes: A61B 2090/3966 Radiopaque markers visible in an X-ray image
• CPC Class Codes: A61B 2090/504 with a counterweight
• CPC Class Codes: A61B 2090/508 with releasable brake mechanisms
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 5/062 using magnetic field
• CPC Class Codes: A61B 5/066 Superposing sensor position on an image of the patient, e.g. obtained by ultrasound or x-ray imaging
• CPC Class Codes: A61B 6/032 Transmission computed tomography [CT]
• CPC Class Codes: A61B 6/037 Emission tomography
• CPC Class Codes: A61B 6/12 Arrangements for detecting or locating foreign bodies
• CPC Class Codes: A61B 6/4405 the apparatus being movable or portable, e.g. handheld or mounted on a trolley
• CPC Class Codes: A61B 6/4417 related to combined acquisition of different diagnostic modalities
• CPC Class Codes: A61B 6/4441 the rigid structure being a C-arm or U-arm
• CPC Class Codes: A61B 6/461 Displaying means of special interest
• CPC Class Codes: A61B 6/462 : characterised by constructional features of the display
• CPC Class Codes: A61B 6/463 : characterised by displaying multiple images or images and diagnostic data on one display
• CPC Class Codes: A61B 6/467 : characterised by special input means
• CPC Class Codes: A61B 6/486 : involving generating temporal series of image data
• CPC Class Codes: A61B 6/487 : involving fluoroscopy
• CPC Class Codes: A61B 6/5235 : combining images from the same or different ionising radiation imaging techniques, e.g. PET and CT
• CPC Class Codes: A61B 6/547 : involving tracking of position of the device or parts of the device
• CPC Class Codes: A61B 8/00 : Diagnosis using ultrasonic, sonic or infrasonic waves imaging of objects using sonar G01S15/00
• CPC Class Codes: A61B 8/0841 : for locating instruments
• CPC Class Codes: A61B 8/4218 : characterised by articulated arms
• CPC Class Codes: A61B 8/4245 : involving determining the position of the probe, e.g. with respect to an external reference frame or to the patient
• CPC Class Codes: A61B 8/4254 : using sensors mounted on the probe
• CPC Class Codes: A61B 8/4263 : using sensors not mounted on the probe, e.g. mounted on an external reference frame
• CPC Class Codes: A61B 8/4416 : related to combined acquisition of different diagnostic modalities, e.g. combination of ultrasound and X-ray acquisitions
• CPC Class Codes: A61B 8/461 : Displaying means of special interest
• CPC Class Codes: A61B 8/462 : characterised by constructional features of the display
• CPC Class Codes: A61B 90/37 : Surgical systems with images on a monitor during operation
• CPC Class Codes: A61B 90/50 : Supports for surgical instruments, e.g. articulated arms
• CPC Class Codes: F04C 2270/041 : Controlled or regulated

Abstract

The invention comprises a virtual window system that creates a visual coherency between the image of the patient and his or her anatomy and the patient by aligning the image of the patient anatomy on the display to the patient and presenting the image to the user that feels is if the user is looking directly into the patient through the display. The invention is designed to also display medical devices, such as a minimally invasive tool. The system substantially unifies the coordinate systems of the patient, the medical device, the display, and the physician'"'"'s hands. The invention creates a visual coherency between the motion of the medical device in the image and the motion of the physician'"'"'s hands manipulating the device. This invention also creates a visual coherency between the motion of the image in the display and display.

Claims

1. A system for displaying an image of a tool on an image of a patient on a movable display, said system comprising:

a display screen configured to be moved and aligned with target regions on a patient'"'"'s body; and

a processor configured to receive data representing the patient'"'"'s anatomy, data representing a position of a tool introduced to the patient'"'"'s body in real time, and data representing a position of the display screen in real time;

wherein the processor is configured to deliver to the display screen an image of the patient anatomy having an image of the tool superimposed thereon, wherein a position of the image of the tool on the image of the anatomy is updated in real time and a region of the anatomy which is on the display is selected by moving the display screen relative to the region.

2. A system as in claim 1, further comprising an external tracker for tracking a position of the tool in the patient'"'"'s body.

3. A system as in claim 2, wherein the external tracker is configured to track an electromagnetic sensor on the tool.

4. A system as in claim 1, further comprising a support for moveably holding the display screen relative to the patient'"'"'s body.

5. A system as in claim 4, wherein the support comprises an articulated arm.

6. A system as in claim 5, wherein the support is sensored to produce the data representing a position of the display screen.

7. A system as in claim 1, further comprising an external tracker for tracking the display screen and producing the data representing a position of the display screen.

8. A system as in claim 1, wherein the processor is configured to receive a static image of the patient anatomy from a data file.

9. A system as in claim 1, wherein the processor is configured to receive a real time image of the patient anatomy from a patient image device.

10. A system as in claim 1, further comprising an external tracker configured to track movement of the patient'"'"'s body, wherein the processor is further configured to receive data from the tracker representing the position of the patient'"'"'s body and to adjust the position of the patient anatomy image on the screen in response to the body movements in real time.

11. A system as in claim 1, further comprising a single external tracker configured to track movement of both the display screen and the tool.

12. A system as in claim 11, wherein the single external tracker is further configured to track movement of the patient'"'"'s body.

13. A method for displaying an image of a tool on an image of a patient on a moveable display screen, said method comprising:

aligning the display screen with a region of the patient'"'"'s body proximate a target anatomy of the patient;

displaying an image of the target anatomy on the display screen; and

superimposing an image of a tool on the image of the target anatomy, wherein a position of the tool image corresponds to a position of an actual tool in or on the actual patient anatomy;

wherein the position of the tool image on the anatomy image is updated in real time as the position of the actual tool changes relative to the actual patient anatomy and wherein a region of the anatomy on the display can be changed by moving the display screen relative to the region.

14. A method as in claim 13, wherein displaying the image of the target anatomy comprises displaying a pre-operative static image.

15. A method as in claim 13, wherein displaying the image of the target anatomy comprises displaying a real-time image.

16. A method as in claim 13, further comprising tracking a position of the tool in real-time in the actual patient anatomy.

17. A method as in claim 14, further comprising tracking a position of the display screen in real-time in a coordinate system which is registered with the patient'"'"'s actual anatomy.

18. A method as in claim 17, further comprising tracking a position of the patient'"'"'s actual anatomy in real time and shifting the coordinate system in response to changes in position of the actual anatomy.

19. A system for displaying an image of a patient anatomy on a moveable display, said system comprising:

a display screen configured to be moved and aligned with target regions on an exterior of a patient'"'"'s body;

a processor configured to receive data representing the patient'"'"'s anatomy and data representing a position of the display screen in real time, wherein the processor is configured to deliver to the display an image of the patient anatomy which changes in real time in response to movement of the display screen in accordance with a spatial relationship between a position of the patient'"'"'s actual anatomy and the position of the display screen in real time; and

means on the display screen allowing a user to adjust the spatial relationship between the position of the display screen and the position of the image of the anatomy represented on the display screen.

20. A system as in claim 19, wherein the display screen means allows a user to selectively interrupt the spatial relationship so that the image of the patient'"'"'s anatomy remains unchanged while the display screen is moved and to thereafter resume the spatial relationship so that the image of the patient'"'"'s anatomy can resume moving on the display screen.

21. A system as in claim 20, wherein the spatial relationship is altered such that upon resumption of the spatial relationship any changes in the position and the orientation of the display screen made during a period of time when the spatial relationship was interrupted are neglected and do not affect the image delivered by the processor to the display.

22. A system as in claim 19, wherein the display screen means allows a user to adjust the scale of the spatial relationship so that movement of the display screen through a distance results in movement of the image of the anatomy on the screen over a distance which is greater than or less than that of the display screen.

23. A system as in claim 19, wherein the display screen means comprises a user input device on the display screen.

24. A system as in claim 23, wherein the user input device comprises at least one of a tracking pad, roller ball, and a joy stick.

25. A system as in claim 23, wherein the user input device comprises a touch screen.

26. A system as in claim 23, wherein the user input device is voice-activated.

27. A system as in claim 19, further comprising a support for movably holding the display screen relative to the patient'"'"'s body.

28. A system as in claim 27, wherein the support comprises an articulated arm.

29. A system as in claim 27, wherein the support is sensored to produce the data representing a position of the display screen.

30. A system as in claim 19, further comprising an external tracker for tracking the screen and producing the data representing a position of the display screen.

31. A system as in claim 19, wherein the processor is configured to receive a static image of the patient anatomy from a data file.

32. A system as in claim 19, wherein the processor is configured to receive a real time image of the patient anatomy from a patient imaging device.

33. A system as in claim 19, further comprising an external tracker for tracking movement of the patient'"'"'s body, wherein the processor is further configured to receive data from the tracker representing the position of the patient'"'"'s body and to adjust the position of the patient anatomy image on the screen in response to the body movements in real time.

34. A method for displaying an image of a patient anatomy on a moveable display, said method comprising:

aligning the display screen with a region of the patient'"'"'s body proximate a target anatomy of the patient;

displaying an image of the target anatomy on the display screen, wherein the image of the target anatomy has a spatial relationship between a position of the patient'"'"'s actual anatomy and the position of the display screen in real time as the display screen is moved relative to the actual anatomy; and

adjusting the spatial relationship in real time so that at least one of a scale of the spatial relationship or continuity of the spatial relationship is changed.

35. A method as in claim 34, wherein adjusting comprises interrupting the spatial relationship so that the display screen can be moved without moving the image of the patient anatomy on the display screen.

36. A method as in claim 35, wherein the spatial relationship is further adjusted to neglect any movement of the display made when the spatial relationship was interrupted upon a resuming of the spatial relationship.

37. A method as in claim 34, wherein adjusting comprises changing the scale of the spatial relationship so that movement of the display screen through a distance results in movement of the image of the anatomy on the screen over a distance which is greater than or less than that of the display screen.

38. A method as in claim 34, wherein displaying the image of the target anatomy comprises displaying a pre-operative static image.

39. A method as in claim 34, wherein displaying the image of the target anatomy comprises displaying a real-time image.

40. A method as in claim 34, further comprising tracking a position of the display screen in real-time in a coordinate system which is registered with the patient'"'"'s actual anatomy.

41. A method as in claim 40, further comprising tracking a position of the patient'"'"'s actual anatomy in real time and shifting the coordinate system in response to changes in position of the actual anatomy.

42. A system for displaying an image from a repositionable patient imaging device on a moveable display screen;

said system comprising;

a display screen configured to be moved and aligned with target regions on an exterior of a patient'"'"'s body;

a repositionable imaging device configured to be moved and aligned with target regions on an exterior of a patient'"'"'s body; and

a processor is configured to coordinate the movement of the display screen and of the repositionable imaging device so that the display screen is positioned in a desired relationship with the target region which is being imaged by the repositionable imaging device.

43. A system as in claim 42, wherein the processor is configured to reposition the imaging device in response to movement of the display screen.

44. A system as in claim 42, wherein the processor is configured to move the display screen in response to repositioning of the imaging device.

45. A system as in claim 42, further comprising a support for moveably holding the display screen relative to the patient'"'"'s body.

46. A system as in claim 45, wherein the support comprises an articulated arm.

47. A system as in claim 45, wherein the support is sensored to produce the data representing a position of the display screen.

48. A system as in claim 42, further comprising an external tracker for tracking the screen and producing the data repositioning a position of the display screen.

49. A system as in claim 42, wherein the processor is configured to receive a static image of the patient anatomy from a data file.

50. A system as in claim 42, wherein the processor is configured to receive a real time image of the patient anatomy from a patient imaging device.

51. A system as in claim 42, further comprising an external tracker for tracking movement of the patient'"'"'s body, wherein the processor is further configured to receive data from the tracker representing the position of the patient'"'"'s body and to adjust the position of the patient anatomy image on the screen in response to the body movements in real time.

52. A method for displaying an image from a repositionable patient imaging device on a moveable display screen, said method comprising

repositioning the imaging device to image different target regions on a patient'"'"'s body;

and

moving the display screen to position the screen in a desired relationship with the target region being imaged by the imaging device;

wherein (1) repositioning the imaging device causes the display screen to move to maintain the desired relationship or (2) moving the display screen causes the imaging device to reposition to maintain the desired relationship.

53. A method as in claim 52, wherein moving the display screen repositions the imaging device.

54. A method as in claim 52, wherein repositioning the imaging device moves the display screen.

55. A method as in claim 52, wherein the movement of the display screen and the repositioning of the imaging device are scaled.

56. A method as in claim 55, wherein the scale is 1:

1.

57. A method as in claim 55, wherein the scale is other than 1:

1.

58. A method as in claim 52, wherein displaying the image of the target anatomy comprises displaying a pre-operative static image.

59. A method as in claim 52, wherein displaying the image of the target anatomy comprises displaying a real-time image.

60. A method as in claim 52, further comprising tracking a position of the display screen in real-time in a coordinate system which is registered with the patient'"'"'s actual anatomy.

61. A method as in claim 60, further comprising tracking a position of the patient'"'"'s actual anatomy in real time and shifting the coordinate system in response to changes in position of the actual anatomy.