Medical image display system and method

• US 10,169,534 B2
• Filed: 09/30/2013
• Issued: 01/01/2019
• Est. Priority Date: 09/30/2013
• Status: Active Grant

• Current Assignee: Toshiba Medical Systems Corporation (Canon Inc.)
• Original Assignee: Toshiba Medical Systems Corporation (Canon Inc.)
• Inventors: Day, Timothy, Ashmole, Dominic
• Primary Examiner(s): Taylor, Jr., Duane N
• Application Number:
  US14/041,291
• Publication Number:
  US 20150091778A1
• Time in Patent Office: 1,919 Days
• Field of Search: None
• CPC Class Codes: A61B 5/6898 Portable consumer electronic devices, e.g. music players, telephones, tablet computers
• CPC Class Codes: A61B 5/7425 Displaying combinations of multiple images regardless of image source, e.g. displaying a reference anatomical image with a live image geometric transforms for image registration G06T3/14; enhancement using image fusion G06T5/50; algorithms for registration of images G06T7/30
• CPC Class Codes: G06F 3/1446 display composed of modules, e.g. video walls
• CPC Class Codes: G06F 3/1454 involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2210/41 Medical
• CPC Class Codes: G09G 2340/0492 Change of orientation of the displayed image, e.g. upside-down, mirrored
• CPC Class Codes: G09G 2370/042 for monitor identification
• CPC Class Codes: G09G 2380/08 Biomedical applications
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: G16H 40/63 for local operation
• CPC Class Codes: G16Z 99/00 Subject matter not provided for in other main groups of this subclass

Abstract

A display system for displaying medical images across a plurality of separate display devices, the system comprising a layout unit for providing layout data concerning the display of medical images, an allocation unit for allocating a plurality of medical images amongst a plurality of separate display devices according to the layout data, wherein at least one of the plurality of display devices comprises a mobile display device.

Claims

1. A display system for collaboratively displaying a plurality of separate medical images across a plurality of separate display devices, the system comprising:

processing circuitry connected to the plurality of separate display devices and configured to

provide a hanging protocol specifying the plurality of separate medical images to be displayed and a relative position in which each of the separate medical images is to be displayed;

obtain, from each of the plurality of separate display devices, consent to join a collaboration with other of the plurality of separate display devices, in order to display the plurality of separate medical images across the plurality of separate display devices, wherein establishment of the collaboration is dependent on at least one of user preference, group preference, a person using at least one of the separate display devices, and particular protocols used;

automatically assign each of the plurality of separate medical images to one of the plurality of separate display devices according to the hanging protocol; and

in response to receiving the consent to join the collaboration from each of the plurality of separate display devices, display the plurality of separate medical images concurrently on the plurality of separate display devices to obtain a collaborative display of the medical images across the plurality of separate display devices,

wherein at least one of the plurality of separate display devices comprises a mobile display device, and

the processing circuitry is further configured to

automatically detect a physical location of each of the plurality of separate display devices,

select, based on the detected physical location of each of the separate display devices, at least one device out of the detected plurality of separate display devices, and

after receiving the consent from the selected at least one device, assign at least one of the plurality of separate medical images for display on the selected at least one device, based on the hanging protocol and at least one of the detected location and an orientation of the selected at least one device with respect to the mobile display device.

2. The display system according to claim 1, wherein the processing circuitry is configured to provide the hanging protocol, which comprises a plurality of rules concerning at least one of allocation to a display device, image appearance, image orientation, image data selection, image rendering, image importance, and image navigation.

3. The display system according to claim 1, wherein the processing circuitry is configured to provide the hanging protocol, which is a DICOM-compliant hanging protocol.

4. The display system according to claim 1, wherein the processing circuitry is further configured to obtain the plurality of separate medical images, which comprise at least one of:

a plurality of views of the same anatomical region of a patient;

a plurality of views obtained using different imaging modalities; and

a plurality of different views obtained by differently rendering a same volumetric image data.

5. The display system according to claim 1, wherein the display system comprises at least one position determination sensor configured to detect the physical location of each of the plurality of separate display devices.

6. The display system according to claim 1, wherein the processing circuitry is further configured to allocate the plurality of medical images to the display devices according to the hanging protocol and the physical locations of the display devices.

7. The display system according to claim 1, wherein in selecting the at least one device out of the detected plurality of separate display devices, the processing circuitry is further configured to select those previously-detected display devices that are separated by less than a threshold distance from each other.

8. The display system according to claim 1, further comprising a user input device, wherein the processing circuitry is further configured to output a list of available display devices, to receive user input via the user input device selecting a plurality of the available display devices, and to assign the plurality of medical images to the selected display devices according to the hanging protocol.

9. The display system according to claim 8, wherein the list of available display devices is a list of further display devices that are within a threshold distance of a first display device.

10. The display system according to claim 1, wherein the processing circuitry is further configured to generate automatically a corresponding change in a medical image displayed on one of the display devices in response to a change in a medical image displayed on at least one other one of the display devices.

11. The display system according to claim 10, wherein the processing circuitry is configured to generate the change in the medical image, which comprises at least one of a change in rendering, a change in position, orientation or co-ordinates, a change in view point, a change in position of a pointer, and at least one of a pan, scale, position and scale adjustment.

12. The display system according to claim 1, wherein the processing circuitry is further configured to provide synchronized navigation of the medical images across the plurality of display devices.

13. The display system according to claim 1, wherein the processing circuitry is configured to provide the hanging protocol, which assigns a relative importance to each of the plurality of medical images.

14. The display system according to claim 13, wherein the processing circuitry is further configured to determine, for at least one medical image of the medical images, in dependence on the relative importance of the one medical image and a number of the display devices available, whether to display the one medical image.

15. The display system according to claim 13, wherein the processing circuitry is further configured to determine, in dependence on the relative importance of the medical images and in dependence on at least one property of the display devices, at least one of position, scale, and resolution of the display device assigned to each medical image.

16. The display system according to claim 1, wherein at least one display device of the plurality of display devices comprises a sensor configured to determine the orientation of the display device and to update an orientation of the medical image displayed on that display device automatically in response to a change in orientation of the display device.

17. The display system according to claim 1, wherein the processing circuitry is further configured to assign the plurality of medical images to the plurality of display devices subject to the plurality of display devices subscribing to a collaboration, and the circuitry is further configured to at least one of reassign the images and alter at least one display parameter in response to one of the plurality of display devices leaving the collaboration.

18. The display system according to claim 1, further comprising a server that includes the processing circuitry, and that further includes a circuit configured to render the medical images and distribute the rendered medical images to at least some of the plurality of display devices.

19. The display system according to claim 1, wherein at least one of the display devices comprises the processing circuitry, and is configured to control the assignment of the images to the other display devices.

20. The display system according to claim 1, further comprising a device discovery circuit configured to discover display devices in a vicinity of the display system.

21. The display system according to claim 19, wherein the device discovery circuit is configured to discover display devices using at least one of Global Positioning System (GPS), near field or short range wireless communication apparatus, a Bluetooth, ZigBee or Infrared (IR) communication apparatus, a camera, and an accelerometer.

22. The system according to claim 1, wherein the processing circuitry is further configured to dynamically modify or replace a layout of the medical images in response to at least one of changes in display devices available to the system and user input.

23. The system according to claim 1, wherein the processing circuitry is further configured to assign at least one medical image to a display device having a location relative to the other display device that best matches the relative image position specified in the hanging protocol for the assigned medical image.

24. The system according to claim 1, wherein the processing circuitry is further configured to select the hanging protocol according to at least one of a user'"'"'s expertise, a user'"'"'s job title, a user'"'"'s interest in a field, user preferences, a display resolution or other display property of at least one of the display devices, a display size of at least one of the display devices, whether a display device is a color or grayscale display device, whether a display device has an electronic ink display, a modality associated with the images, a procedure associated with the images, and anatomy associated with the images.

25. The system according to claim 1, wherein the processing circuitry is further configured to select or change the hanging protocol based on usable system resources, including at least one of CPU usage and memory usage.

26. A mobile display device for collaboratively displaying a plurality of separate medical images in collaboration with at least one other display device, the mobile display device comprising:

a display;

a communication circuit configured to communicate with the at least one other display device;

processing circuitry connected to the at least one other display device and configured to

provide a hanging protocol specifying the plurality of separate medical images to be displayed and a relative position in which each of the separate medical images is to be displayed;

obtain, from each of the at least one other display device, consent to join a collaboration with the mobile display device and other of the at least one other display device, in order to display the plurality of separate medical images across the at least one other display device and the display, wherein establishment of the collaboration is dependent on at least one of user preference, group preference, a person using the at least one of the at least one other display device, and particular protocols used;

automatically assign each of the plurality of separate medical images to one of the mobile display device and at least one other display device according to the hanging protocol; and

in response to receiving the consent to join the collaboration from each of the at least one other display device, cause the display to display the plurality of separate medical images concurrently on the display and the at least one other display device to obtain a collaborative display of the medical images across the at least one other display device and the display,

wherein the processing circuitry is further configured to

automatically detect a physical location of each of the at least one other display device,

select, based on the detected physical location of each of the separate display devices, a device from among the detected at least one other display device, and

after receiving the consent from the selected device, assign at least one of the plurality of separate medical images for display on the selected device, based on the hanging protocol and at least one of the detected physical location and an orientation of the selected device with respect to the mobile display device.

27. The mobile display device according to claim 26, wherein the mobile display device comprises at least one position determination sensor configured to detect the physical location of the mobile display device and the at least one other display device, and the processing circuitry is further configured to assign each of the plurality of separate medical images to one of the mobile display device and the at least one other display device according to the hanging protocol and the physical locations of the display devices.

28. A server for processing a plurality of separate medical images, the server comprising:

processing circuitry configured to

provide a hanging protocol specifying the plurality of separate medical images to be displayed and a relative position in which each of the separate medical images is to be displayed;

obtain, from each of a plurality of separate display devices, consent to join a collaboration with other of the plurality of separate display devices, in order to display the plurality of separate medical images across the plurality of separate display devices, wherein establishment of the collaboration is dependent on at least one of user preference, group preference, a person using at least one of the separate display devices, and particular protocols used;

automatically assign each of the plurality of separate medical images to one of the plurality of separate display devices according to the hanging protocol; and

in response to receiving the consent to join the collaboration from each of the plurality of separate display devices, display the plurality of separate medical images concurrently on the plurality of separate display devices to obtain a collaborative display of the medical images across the plurality of separate display devices,

wherein at least one of the plurality of display devices comprises a mobile display device, and the processing circuitry is further configured to

automatically detect a physical location of each of the plurality of separate display devices,

select, based on the detected physical location of each of the separate display devices, at least one device out of the detected plurality of separate display devices, and

after receiving the consent from the selected at least one device, assign at least one of the plurality of separate medical images for display on the selected at least one device, based on the hanging protocol and at least one of the detected physical location and an orientation of the selected at least one device with respect to the mobile display device.

29. The server according to claim 28, wherein the server comprises at least one position determination sensor configured to detect the physical location of each of the plurality of display devices;

and

the processing circuitry is further configured to assign each of the plurality of separate medical images to one of the plurality of display devices according to the hanging protocol and the detected physical locations of the display devices.

30. A method for collaboratively displaying a plurality of separate medical images across a plurality of separate display devices, the method comprising:

providing a hanging protocol specifying the plurality of medical images to be displayed and a relative position in which each of the separate medical images is to be displayed;

obtain, from each of the plurality of separate display devices, consent to join a collaboration with other of the plurality of separate display devices, in order to display the plurality of separate medical images across the plurality of separate display devices, wherein establishment of the collaboration is dependent on at least one of user preference group preference, a person using at least one of the separate display devices, and particular protocols used;

automatically assigning each of the plurality of separate medical images to one of the plurality of separate display devices according to the hanging protocol,

in response to receiving the consent to join the collaboration from each of the plurality of separate display devices, displaying the plurality of separate medical images concurrently on the plurality of separate display devices to obtain a collaborative display of the medical images across the plurality of separate display devices,

wherein at least one of the plurality of display devices comprises a mobile display device, and the method further includes

automatically detecting a physical location of each of the plurality of separate display devices,

selecting, based on the detected physical location of each of the separate display devices, at least one device out of the detected plurality of separate display devices, and

after receiving the consent from the selected at least one device, assigning at least one of the plurality of separate medical images for display on the selected at least one device, based on the hanging protocol and at least one of the detected physical location and an orientation of the selected at least one device with respect to the mobile display device.

31. The method according to claim 30, further comprising detecting a physical location of each the plurality of display devices, wherein assigning the plurality of separate medical images comprises assigning the plurality of separate medical images to the display devices according to the hanging protocol and the detected physical locations of the display devices.

32. A display system for collaboratively displaying a plurality of separate medical images across a plurality of separate display devices, the system comprising:

processing circuitry connected to the plurality of separate display devices and configured to

provide a hanging protocol specifying the plurality of medical images to be displayed and a relative position in which each of the separate medical images is to be displayed;

obtain, from each of the plurality of separate display devices, consent to join a collaboration with other of the plurality of separate display devices, in order to display the plurality of separate medical images across the plurality of separate display devices, wherein establishment of the collaboration is dependent on at least one of user preference, group preference, a person using at least one of the separate display devices, and particular protocols used;

automatically assign each of the plurality of separate medical images to one of the plurality of separate display devices according to the hanging protocol; and

in response to receiving the consent to join the collaboration from each of the plurality of separate display devices, display the plurality of separate medical images concurrently on the plurality of separate display devices to obtain a collaborative display of the medical images across the plurality of separate display devices,

wherein at least one of the plurality of separate display devices comprises a mobile display device, and the processing circuitry is further configured to

automatically detect a physical location of each of the plurality of separate display devices,

select, based on the detected physical location of each of the separate display devices, at least one device out of the detected plurality of separate display devices, and

after receiving the consent from the selected at least one device, assign at least one of the plurality of separate medical images for display on the selected at least one device, based on the hanging protocol and at least one of the detected physical location and an orientation of the selected at least one device with respect to the mobile display device.