Image viewing application and method for orientationally sensitive display devices

• US 9,111,386 B2
• Filed: 03/14/2011
• Issued: 08/18/2015
• Est. Priority Date: 11/02/2010
• Status: Active Grant

• Current Assignee: Covidien LP (Medtronic Plc)
• Original Assignee: Covidien LP (Medtronic Plc)
• Inventors: Averbuch, Dorian
• Primary Examiner(s): Tung, Kee M
• Assistant Examiner(s): Du, Haixia
• Application Number:
  US13/047,725
• Publication Number:
  US 20120105436A1
• Time in Patent Office: 1,618 Days
• Field of Search: 345/419
• US Class Current: 1/1
• CPC Class Codes: A61B 2034/107 Visualisation of planned trajectories or target regions
• CPC Class Codes: A61B 2034/2048 using an accelerometer or inertia sensor
• CPC Class Codes: A61B 2034/2051 Electromagnetic tracking systems
• CPC Class Codes: A61B 2034/2055 Optical tracking systems
• CPC Class Codes: A61B 34/20 Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
• CPC Class Codes: A61B 34/25 User interfaces for surgical systems
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G06T 15/08 Volume rendering
• CPC Class Codes: G06T 15/20 Perspective computation
• CPC Class Codes: G06T 15/506 Illumination models
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2215/12 Shadow map, environment map
• CPC Class Codes: G06T 2215/16 Using real world measurements to influence rendering
• CPC Class Codes: G06T 2219/2016 Rotation, translation, scaling

Abstract

A system and method for presenting three-dimensional image volume data utilizing an orientationally-sensitive display device whereby the image volume is navigable simply by tilting, raising and lowering the display device. Doing so presents an image on the screen that relates to the angle and position of the display device such that the user gets the impression that the device itself is useable as a window into the image volume, especially when the device is placed on or near the source of the image data, such as a patient.

Claims

1. A method of displaying images on an orientationally-sensitive display device, the method comprising:

providing three-dimensional image volume data to said device relating to an object;

registering a location and orientation of said device with said object;

displaying on said display device an image of the object generated from said three-dimensional image volume data and corresponding to said location and orientation of said device with respect to said object; and

generating new images based on changes in location and orientation of said device relative to said object;

indicating a plane of the displayed image relative to the object;

wherein a user continuously views the object in three dimensions, from a plurality of angles, in real-time, as the display device moves around the object, based on the location and orientation of the device relative to the object; and

wherein the displayed image includes at least a portion of a pre-planned path, marked with a plurality of markers by the user, from an entry point in the three-dimensional image volume data to a target identified by the user, the pre-planned path being automatically illuminated in response to placement of the plurality of markers.

2. The method of claim 1, wherein said registering a location and orientation of said device with said object comprises placing at least one marker on said object and tracking a position of said device relative to said at least one marker.

3. The method of claim 2, wherein tracking a position of said device relative to said at least one marker comprises using an optical tracking mechanism to locate said markers.

4. The method of claim 2, wherein tracking a position of said device relative to said at least one marker comprises using a magnetic positioning system.

5. The method of claim 1, wherein said registering a location and orientation of said device with said object comprises using a global positioning system incorporated into said device, said global positioning system providing locations of said object and said device.

6. The method of claim 1, further comprising giving the user the option to lock the present image such that further moving of said device does not result in a changing of said image until desired.

7. The method of claim 1, further comprising providing a fly-through feature whereby, when selected, an animated representation of said pre-planned path is displayed from a viewpoint along said pre-planned path.

8. A method of presenting three-dimensional image volume data relating to an object, the method comprising:

accessing said three-dimensional image volume data using an orientationally-sensitive display device;

registering a location and orientation of said device with said object;

displaying on said display device an image of the object generated from said three-dimensional image volume data and corresponding to said location and orientation of said device with respect to said object; and

continuously generating new images based on changes in location and orientation of said device relative to said object;

indicating a plane of the displayed image relative to the object;

wherein a user continuously views the object in three dimensions, from a plurality of angles, in real-time, as the display device moves around the object, based on the location and orientation of the device relative to the object; and

wherein the displayed image includes at least a portion of a pre-planned path, marked with a plurality of markers by the user, from an entry point in the three-dimensional image volume data to a target identified by the user, the pre-planned path being automatically illuminated in response to placement of the plurality of markers.

9. The method of claim 8, wherein said registering a location and orientation of said device with said object comprises placing at least one marker on said object and tracking a position of said device relative to said at least one marker.

10. The method of claim 9, wherein tracking a position of said device relative to said at least one marker comprises using an optical tracking mechanism to locate said markers.

11. The method of claim 9, wherein tracking a position of said device relative to said at least one marker comprises using a magnetic positioning system.

12. The method of claim 8, wherein said registering a location and orientation of said device with said object comprises using a global positioning system incorporated into said device, said global positioning system providing locations of said object and said device.

13. The method of claim 8, further comprising giving the user the option to lock the present image such that further moving of said device does not result in a changing of said image until desired.

14. The method of claim 8, further comprising providing a fly-through feature whereby, when selected, an animated representation of said pre-planned path is displayed from a viewpoint along said pre-planned path.

15. A system for use in presenting internal images of an object, the system comprising:

an orientationally-sensitive display device;

three-dimensional image volume data relating to said object accessible by said device; and

a computer program executable by said device that receives input from orientation sensors of said device, as well as registration data between a location and orientation of said device relative to a location and orientation of said object, and generates on a display of said display device, an internal image of said object that is based on changes in said location and orientation of said display device relative to said object, such that a user continuously views the object in three dimensions, from a plurality of angles, in real-time, as the display device moves around the object, based on the location and orientation of the device relative to the object;

wherein the image includes at least a portion of a pre-planned path, marked with a plurality of markers by the user, from an entry point in the three-dimensional image volume data to a target identified by the user, the pre-planned path being automatically illuminated in response to placement of the plurality of markers; and

an indication mechanism to illustrate a plane of the image relative to the object.

16. The system of claim 15, wherein said computer program updates said internal image of said object that corresponds to said location and orientation of said display device relative to said object whenever movement of said display device is detected.

17. The system of claim 16, wherein said computer program further provides an option to lock said image of said object such that movement of said device does not result in a changed image.

18. The system of claim 16, further comprising a slider feature that, when moved, causes said device to generate image changes that correspond to a hypothetical movement along an axis that is normal to a plane of said display screen.

19. The system of claim 15, further comprising providing a fly-through feature whereby, when selected, generates an animated representation of said pre-planned path displayed from a viewpoint along said pre-planned path.