Surgeon head-mounted display apparatuses
First Claim
1. An augmented reality surgical system comprising:
- a display comprising a see-through display screen that display images while allowing transmission of ambient light therethrough;
a motion sensor connected to the display and configured to output a motion signal indicating measured movement of the display;
at least one camera coupled to a portion of the display and configured to observe reference markers connected to a patient, and reference markers connected to a surgical tool located within a surgical room; and
a position tracking system configured to track the location of the surgical tool and/or a prosthetic, the display, a surgical site and a target location on the patient,computer equipment configured to;
compute the relative location and orientation of the display and the reference markers connected to the patient based on processing an image signal from the at least one camera;
generate a three dimensional anatomical image using patient data created by medical imaging equipment that has imaged a portion of the patient;
rotate and scale at least a portion of the three dimensional anatomical model based on the relative location and orientation of the reference markers connected to the head mounted display and the reference markers connected to the patient, and further rotate at least a portion of the three dimensional anatomical model based on the head motion signal to track measured movement of the head mounted display; and
generate a video signal based on at least a portion of the three dimensional anatomical image and the location and orientation of the reference makers coupled to the patient and an active surgical instrument,output a video signal to the display screen on the display,wherein the video signal is a graphical representation of a virtual trajectory of a surgical instrument that intersects a target location on a patient, the target location corresponds to a point the surgical instrument impacts the three dimensional anatomical image which is oriented and scaled to match the patient'"'"'s anatomy, andwherein the virtual trajectory is continuously updated and dynamically displayed to a surgeon who repositioning and reorienting the surgical instrument.
1 Assignment
0 Petitions
Accused Products
Abstract
An augmented reality surgical system includes a head mounted display (HMD) with a see-through display screen, a motion sensor, a camera, and computer equipment. The motion sensor outputs a head motion signal indicating measured movement of the HMD. The computer equipment computes the relative location and orientation of reference markers connected to the HMD and to the patient based on processing a video signal from the camera. The computer equipment generates a three dimensional anatomical model using patient data created by medical imaging equipment, and rotates and scales at least a portion of the three dimensional anatomical model based on the relative location and orientation of the reference markers, and further rotate at least a portion of the three dimensional anatomical model based on the head motion signal to track measured movement of the HMD. The rotated and scaled three dimensional anatomical model is displayed on the display screen.
-
Citations
20 Claims
-
1. An augmented reality surgical system comprising:
-
a display comprising a see-through display screen that display images while allowing transmission of ambient light therethrough; a motion sensor connected to the display and configured to output a motion signal indicating measured movement of the display; at least one camera coupled to a portion of the display and configured to observe reference markers connected to a patient, and reference markers connected to a surgical tool located within a surgical room; and a position tracking system configured to track the location of the surgical tool and/or a prosthetic, the display, a surgical site and a target location on the patient, computer equipment configured to; compute the relative location and orientation of the display and the reference markers connected to the patient based on processing an image signal from the at least one camera; generate a three dimensional anatomical image using patient data created by medical imaging equipment that has imaged a portion of the patient; rotate and scale at least a portion of the three dimensional anatomical model based on the relative location and orientation of the reference markers connected to the head mounted display and the reference markers connected to the patient, and further rotate at least a portion of the three dimensional anatomical model based on the head motion signal to track measured movement of the head mounted display; and
generate a video signal based on at least a portion of the three dimensional anatomical image and the location and orientation of the reference makers coupled to the patient and an active surgical instrument,output a video signal to the display screen on the display, wherein the video signal is a graphical representation of a virtual trajectory of a surgical instrument that intersects a target location on a patient, the target location corresponds to a point the surgical instrument impacts the three dimensional anatomical image which is oriented and scaled to match the patient'"'"'s anatomy, and wherein the virtual trajectory is continuously updated and dynamically displayed to a surgeon who repositioning and reorienting the surgical instrument. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. An augmented reality surgical system for displaying multiple video streams to a user, the augmented reality surgical system comprising:
-
a display comprising a see-through display screen that display images while allowing transmission of ambient light therethrough; at least one camera coupled to a portion of the display and configured to observe reference markers connected to a patient, and reference markers connected to a surgical tool located within a surgical room, a motion sensor connected to the display and configured to output a motion signal indicating measured movement of the display; and
a computer equipment configured to receive a plurality of video streams from one or more source devices and to control which of the video streams are output as a video signal to the display screen based on the head motion signal,wherein the display is configured as glasses; and a position tracking system configured to track the location of the surgical tool and/or a prosthetic, the display, a surgical site and a target location on the patient wherein the video signal is a graphical representation of a virtual trajectory of a surgical instrument that intersects a target location on a patient, the target location corresponds to a point the surgical tool impacts the three dimensional anatomical image which is oriented and scaled to match the patient'"'"'s anatomy, and wherein the virtual trajectory is continuously updated and dynamically displayed to a surgeon who repositioning and reorienting the surgical instrument. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19, 20)
-
Specification