Predictive visualization of medical imaging scanner component movement

• US 10,573,023 B2
• Filed: 04/09/2018
• Issued: 02/25/2020
• Est. Priority Date: 04/09/2018
• Status: Active Grant

• Current Assignee: Globus Medical, Inc.
• Original Assignee: Globus Medical, Inc.
• Inventors: Crawford, Neil, Johnson, Norbert
• Primary Examiner(s): Yang, Yi
• Application Number:
  US15/948,348
• Publication Number:
  US 20190311490A1
• Time in Patent Office: 687 Days
• CPC Class Codes: A61B 6/102 Protection against mechanical damage, e.g. anti-collision devices
• CPC Class Codes: A61B 6/4405 the apparatus being movable or portable, e.g. handheld or mounted on a trolley
• CPC Class Codes: A61B 6/4441 the rigid structure being a C-arm or U-arm
• CPC Class Codes: A61B 6/461 Displaying means of special interest
• CPC Class Codes: A61B 6/547 involving tracking of position of the device or parts of the device
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0304 Detection arrangements using opto-electronic means constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042
• CPC Class Codes: G06F 3/0346 with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 2207/10081 Computed x-ray tomography [CT]
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80

Abstract

An augmented reality system is provided for use with a medical imaging scanner. The AR system obtains a digital image from a camera, and identifies a pose of a gantry of the medical imaging scanner based on content of the digital image. The gantry includes a movable C-arm supporting an imaging signal transmitter and a detector panel that are movable along an arc relative to a station. A range of motion of the movable C-arm along the arc is determined based on the pose. A graphical object is generated based on the range of motion and the pose, and is provided to a display device for display as an overlay relative to the medical imaging scanner.

Claims

1. An augmented reality (AR) system for use with a medical imaging scanner, the AR system comprising:

a processor; and

a memory comprising program code that is executable by the processor to perform operations comprising;

obtaining a digital image from a camera;

identifying a pose of a gantry of the medical imaging scanner based on content of the digital image, the gantry including a movable C-arm supporting an imaging signal transmitter and a detector panel that are movable along an arc relative to a station;

determining a range of motion of the movable C-arm along the arc based on the pose;

generating a graphical object based on the range of motion and the pose; and

providing the graphical object to a display device for display as an overlay relative to the medical imaging scanner,

wherein the operation of identifying the pose of the gantry comprises identifying a pose of at least one of the imaging signal transmitter and the detector panel,

wherein the operation of determining the range of motion of the movable C-arm along the arc comprises determining a range of motion of the at least one of the imaging signal transmitter and the detector panel along the arc; and

wherein the operation of providing the graphical object to the display device comprises providing an arcuate object for display as an overlay relative to the gantry and with a pose that indicates the range of motion of the at least one of the imaging signal transmitter and the detector panel along the arc;

wherein the operation of providing the arcuate object for display comprises providing a circular object for display as an overlay relative to the gantry and with a pose that indicates an interior region of the movable C-arm that will not be contacted by the at least one of the imaging signal transmitter and the detector panel when moved along the arc through the range of motion.

2. The AR system of claim 1, further comprising:

determining that a physical object which is separate from the gantry has a surface that extends from a location within the circular object displayed on the display device to another location that is outside the circular object; and

performing a collision alert action responsive to the determination.

3. The AR system of claim 2, wherein:

performing the collision alert action comprises providing another graphical object for display as an overlay relative to the physical object and that identifies the physical object as being a collision risk for the at least one of the imaging signal transmitter and the detector panel when moved along the arc through the range of motion.

4. The AR system of claim 2, wherein:

performing the collision alert action comprises communicating a command to the medical imaging scanner that disables electronic movement of the movable C-arm at least in a direction that may collide with the physical object.

5. The AR system of claim 1, wherein:

the operation of determining the range of motion of the movable C-arm along the arc comprises determining a range of motion of an end of the movable C-arm along the arc; and

the operation of providing the graphical object to the display device comprises providing the graphical object to the display device as an overlay that extends from a present location of the end of the movable C-arm to a spaced-apart location along the arc that is within the range of motion of the end of the movable C-arm.

6. The AR system of claim 5, further comprising:

determining that a physical object that is separate from the gantry has a surface that is intersected by the graphical object; and

providing another graphical object for display as an overlay relative to the physical object and that identifies the physical object as being a collision risk for the end of the movable C-arm when moved along the arc through the range of motion.

7. The AR system of claim 5, further comprising:

determining that a physical object that is separate from the gantry has a surface that is intersected by the graphical object; and

communicating a command to the medical imaging scanner that temporarily disables electronic movement of the movable C-arm at least in a direction that may collide with the physical object, responsive to the determination that the surface of the physical object is intersected by the graphical object.

8. The AR system of claim 1, wherein:

the operation of identifying the pose of the gantry comprises identifying a pose of at least one of the imaging signal transmitter and the detector panel;

the operation of generating the graphical object based on the range of motion and the pose, comprises generating a first graphical object that represents the imaging signal transmitter and has a first pose that is rotated and offset to a first location along the arc relative to a present location of the imaging signal transmitter, generating a second graphical object that represents the detector panel and has a second pose that is rotated and offset to a second location along the arc relative to a present location of the detector panel; and

the operation of providing the graphical object to the display device comprises providing the first and second graphical objects for display as an overlay relative to the gantry and with the respective first and second poses.

9. The AR system of claim 8, further comprising:

repeating the operations of generating the graphical object for a plurality of locations along the arc and the providing the graphical object to the display device to animate repetitive movement of the imaging signal transmitter and the detector panel through at least part of the range of motion along the arc.

10. The AR system of claim 1, wherein:

the operation of identifying the pose of the gantry comprises identifying present locations of the imaging signal transmitter and the detector panel;

the operation of generating the graphical object, comprises generating a first virtual imaging signal beam and a second virtual imaging signal beam; and

the operation of providing the graphical object to the display device comprises;

providing the first virtual imaging signal beam for display as an overlay relative to the gantry and extending from an earlier defined location of the imaging signal transmitter to an earlier defined location of the detector panel; and

providing the second virtual imaging signal beam for display as another overlay relative to the gantry and extending from a present location of the imaging signal transmitter to a present location of the detector panel.

11. The AR system of claim 10, wherein:

the first and second virtual imaging signal beams are displayed rotationally offset from each other by about 90 degrees.

12. The AR system of claim 1, wherein:

the operation of identifying the pose of the gantry comprises identifying within the digital image a defined feature of the gantry, and identifying a pose of at least one of the imaging signal transmitter and the detector panel based on a pose of the defined feature within the digital image.

13. The AR system of claim 1, wherein:

the operation of identifying the pose of the gantry comprises identifying within the digital image a location and orientation of a plurality of spaced-apart navigation markers connected to the gantry, and identifying a pose of at least one of the imaging signal transmitter and the detector panel based on the location and orientation of the plurality of spaced-apart tracking markers.