SURGEON HEAD-MOUNTED DISPLAY APPARATUSES
First Claim
1. An augmented reality surgical system comprising:
- a head mounted display comprising a see-through display screen that display images while allowing transmission of ambient light therethrough;
a motion sensor connected to the head mounted display and configured to output a head motion signal indicating measured movement of the head mounted display; and
at least one camera configured to observe reference markers connected to the head mounted display, reference markers connected to a patient, and reference markers connected to a surgical tool located within a surgical room;
computer equipment configured to;
compute the relative location and orientation of the reference markers connected to the head mounted display and the reference markers connected to the patient based on processing a video signal from the at least one camera;
generate a three dimensional anatomical model using patient data created by medical imaging equipment that has imaged a portion of the patient;
rotate and scale at least a portion of the three dimensional anatomical model based on the relative location and orientation of the reference markers connected to the head mounted display and the reference markers connected to the patient, and further rotate at least a portion of the three dimensional anatomical model based on the head motion signal to track measured movement of the head mounted display; and
generate a video signal based on the rotated and scaled three dimensional anatomical model;
output the video signal to the display screen of the head mounted display.
3 Assignments
0 Petitions
Accused Products
Abstract
An augmented reality surgical system includes a head mounted display (HMD) with a see-through display screen, a motion sensor, a camera, and computer equipment. The motion sensor outputs a head motion signal indicating measured movement of the HMD. The computer equipment computes the relative location and orientation of reference markers connected to the HMD and to the patient based on processing a video signal from the camera. The computer equipment generates a three dimensional anatomical model using patient data created by medical imaging equipment, and rotates and scales at least a portion of the three dimensional anatomical model based on the relative location and orientation of the reference markers, and further rotate at least a portion of the three dimensional anatomical model based on the head motion signal to track measured movement of the HMD. The rotated and scaled three dimensional anatomical model is displayed on the display screen.
-
Citations
31 Claims
-
1. An augmented reality surgical system comprising:
-
a head mounted display comprising a see-through display screen that display images while allowing transmission of ambient light therethrough; a motion sensor connected to the head mounted display and configured to output a head motion signal indicating measured movement of the head mounted display; and at least one camera configured to observe reference markers connected to the head mounted display, reference markers connected to a patient, and reference markers connected to a surgical tool located within a surgical room; computer equipment configured to; compute the relative location and orientation of the reference markers connected to the head mounted display and the reference markers connected to the patient based on processing a video signal from the at least one camera; generate a three dimensional anatomical model using patient data created by medical imaging equipment that has imaged a portion of the patient; rotate and scale at least a portion of the three dimensional anatomical model based on the relative location and orientation of the reference markers connected to the head mounted display and the reference markers connected to the patient, and further rotate at least a portion of the three dimensional anatomical model based on the head motion signal to track measured movement of the head mounted display; and generate a video signal based on the rotated and scaled three dimensional anatomical model; output the video signal to the display screen of the head mounted display. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10)
-
-
11. An augmented reality surgical system for displaying multiple video streams to a user, the augmented reality surgical system comprising:
-
a head mounted display comprising a see-through display screen that display images while allowing transmission of ambient light therethrough; a motion sensor connected to the head mounted display and configured to output a head motion signal indicating measured movement of the head mounted display; and a computer equipment configured to receive a plurality of video streams from one or more source devices and to control which of the video streams are output as a video signal to the display screen based on the head motion signal. - View Dependent Claims (12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31)
-
Specification