Video-based interactive viewing along a path in medical imaging

• US 10,204,449 B2
• Filed: 09/01/2015
• Issued: 02/12/2019
• Est. Priority Date: 09/01/2015
• Status: Active Grant

• Current Assignee: Siemens Healthcare GMBH (Siemens AG)
• Original Assignee: Siemens Healthcare GMBH (Siemens AG)
• Inventors: Hong, Wei, Paulus, Jr., John, Yu, Daphne, Petkov, Kaloian
• Primary Examiner(s): Vu, Kieu D
• Assistant Examiner(s): Calderon, IV, Alvaro R
• Application Number:
  US14/842,410
• Publication Number:
  US 20170061687A1
• Time in Patent Office: 1,260 Days
• CPC Class Codes: G06F 3/04815 Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06T 19/003 Navigation within 3D models or images
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G11B 27/309 Table of contents

Abstract

For interactive viewing in medical imaging, pre-rendering is used to provide greater quality of the rendered images with a given computational power. For interactivity in viewing direction, two or more views are rendered for each location and provided as video to the viewer. The viewer selects a viewing direction interactively, and an image at the selected view direction is formed from the previously rendered images of the videos for the two or more views. In an alternative or additional approach, a video graph is created with videos of different segments of the graph. The videos are sequences of images representing change in location, camera parameters (e.g., zooming), and/or rendering settings. Once provided to the user, interactivity is provided by selection of the appropriate node of the video graph. The user may interactively select different locations and/or rendering settings available in the video graph, allowing interaction while benefiting from quality provided by pre-rendered video.

Claims

1. A non-transitory computer readable storage medium having stored therein data representing instructions executable by a programmed processor for interactive viewing in medical imaging, the storage medium comprising instructions for:

generating a video graph having multiple segments, at least a first segment of the segments branching at a first exit node into a second segment of the segments and a third segment of the segments, the first segment comprising a first plurality of sequential nodes including at least an entrance node and the first exit node, the second segment comprising a second plurality of sequential nodes including at least the first exit node and a second exit node, and the third segment comprising a third plurality of sequential nodes including at least the first exit node and a third exit node;

rendering images for each of the first plurality of sequential nodes, the second plurality of sequential nodes, and the third plurality of sequential nodes from the medical scan data representing a volume;

encoding with a video codec, the images of the entrance node, the first exit node, the second exit node, and the third exit node as intra-encoded frames and the images of others of the nodes of the first plurality of sequential nodes, the second plurality of sequential nodes, and the third plurality of sequential nodes as predictive-encoded frames;

generating a first video from images of the first plurality of sequential nodes, a second video from images of the second plurality of sequential nodes, and a third video from images of the third plurality of sequential nodes;

and serving the first, second, and third videos to a client device upon request based on user selection of nodes, wherein only the entrance node, the first exit node, the second exit node, and the third exit node are user selectable.

2. The non-transitory computer readable storage medium of claim 1 wherein generating comprises generating the first and second segments as different spatial paths through the volume.

3. The non-transitory computer readable storage medium of claim 1 wherein generating comprises generating the first segment as a spatial path through the volume and generating the second segment as a change in rendering parameters for a location on the spatial path.

4. The non-transitory computer readable storage medium of claim 1 wherein encoding comprises encoding separate video streams for the first segment with the separate video streams progressing along the first segment in different directions.

5. The non-transitory computer readable storage medium of claim 1 wherein rendering the images comprises rendering multiple of the images for one of the nodes of the first plurality of nodes or the second plurality of nodes, the multiple images for the one node comprising fisheye images from different directions at a location.