Digital pathology system

• US 8,463,741 B2
• Filed: 09/04/2009
• Issued: 06/11/2013
• Est. Priority Date: 09/04/2009
• Status: Active Grant

• Current Assignee: Fujifilm Healthcare Americas Corporation (Fujifilm Holdings Corporation)
• Original Assignee: Omnyx, LLC
• Inventors: Ehlke, Joshua Todd, Khandavilli, Narasimha Rao, McVaugh, Timothy Mark, Melanson, Anthony J., Selzer, Steve, Stratman, Curtis, Virnik, Marina, Weihnacht, Joshua Micah, Zeng, Zhen
• Primary Examiner(s): Trujillo, James
• Assistant Examiner(s): VU, THONG H
• Application Number:
  US12/554,276
• Publication Number:
  US 20110060766A1
• Time in Patent Office: 1,376 Days
• Field of Search: 382/240, 382/151, 382/128, 378/37, 707/802, 707/10, 707/661
• US Class Current: 707/608
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04845 for image manipulation, e.g. dragging, rotation, expansion or change of colour
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G16H 10/40 for data related to laboratory analysis, e.g. patient specimen analysis
• CPC Class Codes: G16H 15/00 ICT specially adapted for medical reports, e.g. generation or transmission thereof
• CPC Class Codes: G16H 30/00 ICT specially adapted for the handling or processing of medical images computed tomography A61B6/03
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS

Abstract

A digital pathology system has a central workflow server hosting digital pathology application services and supporting one or more pathology workstations. The digital pathology system may include one or more image servers, providing digital images of sample specimen slides that are associated with medical cases. Residing at and executing on each pathology workstation is a digital pathology application client, which is the counterpart of digital pathology application services at the central workflow server. The combination of digital pathology application services at the central workflow server and digital pathology application client at each pathology workstation support a pathology workflow software module and a slide viewer software module. The present disclosure also describes a method of operation and/or use of the digital pathology system.

Claims

1. A data processing system for aid of a medical pathologist in manual workflow involving review, analysis and reporting on patient case information that includes microscopic images of patient tissue specimens and associated data, comprising:

a source of digital image data representing images of the patient tissue specimens to a plurality of predetermined magnifications for plural patients including cases to be reviewed by the pathologist;

an image database containing memory for storing the digital image data and associated data relating to the image data;

a workflow database containing memory for storing case management data associated with examination of the specimens by the pathologist;

a computer workstation comprising at least one digital processor coupled to the image database and to the workflow database, the processor being operable under control of a user to obtain and display selected ones of the images for review, to accept user input from the pathologist relating to the examination of the specimens by the pathologist and to record in at least one of the image database and the workflow database information resulting from the user input;

wherein the digital processor is programmed, in conjunction with display of the image data, to present to the user a display based on the case management data in the workflow database, said display comprising visual representations emulating contents of physical case packages that include paper files and glass slides forming workload units to be reviewed, wherein the processor is programmed to select and to display digital image data obtained from the physical case packages upon user selection and manipulation of display icons that graphically represent depictions of the physical case packages and depictions of the glass slides, said emulating including visually representing the case packages and the glass slides as icons that are visibly changed in at least one of a shape, size, orientation, color, blinking characteristic, animation characteristic and position of the icons in a user display for manipulating the case management data in the workflow database to virtually correspond with user handling of physical case packages and the glass slides;

wherein the processor is operable when accessing at least one of the image database and the workflow database, to associate the images with digital document data relating to the case package and its associated patient.

2. The data processing system of claim 1, wherein the handling comprises selecting and organizing case packages and steps in a predetermined sequence for diagnostic microscopic viewing and reporting on the case packages by the pathologist via the workstation, upon tissue samples associated with patient case files, wherein each of said patient case files has images of one or more tissue samples and specimen slides, wherein each said specimen slide has plural image views stored in the image database and the views are selectable by the pathologist controlling the workstation to select among the case packages via the workstation and to present selected locations on slides in the case packages on the user display at selected levels of magnification.

3. The data processing system of claim 2, wherein the images comprise all viewable locations on the specimens on the slides to the predetermined magnification;

wherein the processor is programmed to display to the user a location selected by the user; and

wherein the processor is operable for fetching and processing from the image database, and for displaying to the user an image at the selected location and level of magnification selected by the user.

4. The data processing system of claim 2, wherein the image views selectable by the pathologist via the workstation include a substantially full scale view of a slide including a label, the label defining a changeable orientation of the slide, and one or more slides in a case package being reoriented by the processor and selection by the user of an input to indicate passing a step in a workflow.

5. The data processing system of claim 4, wherein the digital processor is programmed to present the image data to the user on a display under user operated inputs, for emulating use of an optical microscope by selection of a viewing location on a specimen mounted on a slide and selecting a magnification.

6. The data processing system of claim 2, wherein the information resulting from the user input comprises location information and annotations on the image at the selected location, and is stored in the image database.

7. The data processing system of claim 6, wherein the processor is operable when accessing at least one of the image database and the workflow database, to associate images and data from information on a slide label, information on one of a stored document and a stored document image, a stored medical image, a gross specimen image and information relating to one of a case package and an associated patient, wherein the processor is further operable to revert to display of selected ones of said images and said information due to an association with at least one of said information of the document, document image, medical image, gross specimen image, case package and patient.

8. The data processing system of claim 6, wherein the information resulting from the user input comprises case management information including selections generated by the user in preparation to view the images and status information generated by the digital processor reflecting progress in completion of the steps in the predetermined sequence, and wherein said case management information is stored by the digital processor in the workflow database.

9. The data processing system of claim 2, wherein the digital processor is programmed to associate data in the image database and in the workflow database for plural case packages that are related based on at least one variable value stored in one of the image database and the workflow database, and to present during presentation of one of said plural case packages, an option to switch to presentation of at least one other of the case packages that are related, whereby a user can select the option to compare said related case packages.

10. The data processing system of claim 9, wherein the plural case packages are related as case packages of at least one of a same patient, a same body part, a same block, a same pathological condition, and review by a same user.

11. The data processing system of claim 9, wherein the plural case packages are related as case packages of a same patient, and wherein the digital processor is programmed to sort the plural case packages in a time sequence, whereby the user can compare the case packages for changes occurring over the time sequence.

12. The data processing system of claim 1, wherein the digital processor is programmed to present at least one display based on said case management data, wherein at least one representation of a slide in the case package is a distinct place holder representation of a slide that has been identified as prospectively expected or as formerly available, but for which at least one form of image is not presently available for viewing.

13. The data processing system of claim 1, wherein the information resulting from the user input comprises case management information including selections generated by the user in preparation to view the images and status information generated by the digital processor reflecting progress in completion of the steps in the predetermined sequence;

wherein said case management information is stored by the digital processor in the workflow database; and

wherein the digital processor is programmed to generate displays selected by the user input representing parts of the case management information, the status information and the working information, for managing operations related to plural said slides and plural said case packages.

14. The data processing system of claim 13, wherein said displays selected by the user input include a display of a selected one of the case packages, together with associated slide images, that are selected from one of the image database and the workflow database, due to a relationship with at least one of a currently active case package, currently active slide image, current annotation, currently displayed information and currently displayed document, according at least one variable value stored in the workflow database.

15. The data processing system of claim 14, wherein the processor is controllable by said user input to suspend operations on the currently active one of the case packages, slide images, annotations, information and related documents, to store in the workflow database a record of a current state of review of said currently active one, and to revert to the selected one of the case packages, slide images, annotations, information and related documents, and thereafter to return to the current state of review of said currently active one under control of said user input, whereby the user is able to maintain plural case packages in progress while switching to displays of other case packages that can be related to the currently active one or unrelated to the currently active one.

16. The data processing system of claim 1, wherein manipulation of the case package icons associated with the files and slides via the user input includes categorizing the case packages to define piles by selecting and grouping icons visually resembling files and icons visually resembling slides, and wherein case packages are added to and removed from the piles under control by user input and by operation of the processor.

17. The data processing system of claim 16, wherein the processor is programmed to identify at least one of a categorization of the case packages and a categorization of the slides, by altering at least one of a size, shape, color parameter, orientation and location of associated said icons.

18. The data processing system of claim 17, wherein said categorization of the case packages and slides within said case packages by the processor includes visually distinguishing the associated said icons in a display presented to a user so as to distinguish a reviewed and not-reviewed state of the case packages and the slides.

19. The method of claim 18, wherein the plural case packages are related as case packages of a same patient, and wherein the digital processor is programmed to sort the plural case packages in a time sequence, whereby the user can compare the case packages for changes in said same patient occurring over the time sequence.

20. The data processing system of claim 17, wherein the workstation comprises at least two distinct display areas respectively configured for display of image data, controlled by the user input to emulate the controls of an optical microscope, and for display of the case management information, controlled by at least one of a keyboard and location selecting device.

21. The data processing system of claim 20, wherein the at least two distinct display areas have differently configured manual controls for at least one of positioning, selecting and variably controlling and entering of information for emulating the controls of an optical microscope, and for display of the case management information.

22. The data processing system of claim 1, wherein the computer workstation is one of a plurality of workstations coupled over a network to at least one server, maintaining at least one said image database, at least one said workflow database and a medical information database, and wherein the workflow database and the image database are accessible to plural users, whereby the image data and the information resulting from the user input are useful for later reference, and further professional review.

23. A method for digitally managing selection, examination and reporting upon patient samples by manual operations of a human pathologist or technician user assisted by computational steps using at least one programmed digital processor, comprising:

obtaining tissue samples and preparing the samples on slides;

scanning and digitally encoding microscopic images of at least a predetermined area of the slides encompassing the tissue samples so as to enable display of images of the tissue at plural selectable magnifications;

associating data and images defining a pathology patient, a patient case, and a patient tissue sample, and storing digital data files representing the microscopic images in at least one database with associated information identifying the data and images and associated patient information, the images from the slides and the associated patient information defining a virtual case package containing digital images and digital information including the associated patient information;

presenting to the user a display interface that emulates a pathologist'"'"'s physical case package containing papers and a set of tissue samples mounted on glass slides by visual presentation of digital icons graphically representing the case package on the display interface as a case package selectable from a pile of case packages, and digital icons graphically representing the digital images in the case package as slides selectable from a slide tray;

managing selection and presentation of the digital images and the digital information on the display interface by user manipulation of the digital icons, thereby emulating manipulation and microscopic examination of the slides by manipulation of the digital icons representing the slides, and emulating handling of the case package by revising the digital information during review of the digital images and the digital information during a workflow accomplished by the user;

making the digital images and the digital information accessible over a network to plural users of the network;

providing a workstation having a digital processor in data communication with the at least one database, at least one display and input devices arranged for manipulation by the user;

operating the digital processor under control of a stored data processing program to present to the user options for selection of case packages and portions of case packages forming a workload, displaying to the user on the workstation the images and information and recording and storing in the at least one database, data generated as a result of review of the images and information by the user.

24. The method of claim 23, further comprising recording annotations by the user connected to one of a slide image and one or more locations on a slide image, and storing data representing the annotations in the at least one database.

25. The method of claim 23, further comprising determining a dimension defined by a plurality of said locations.

26. The method of claim 25, comprising associating one of said locations with the data generated as said result of the images and information by the user, and further comprising reverting to display of one of images and data upon encountering a user input selection and a workflow step associated with the one of said locations.

27. The method of claim 23, wherein the presentation and manipulation of images as controlled by the processor and as advanced by user input defines a workflow having at least certain predetermined steps including a concluding step selectable by the user.

28. The method of claim 27, wherein the workflow comprises a step of validating slides presented as constituting a patient sample by displaying slide icons and slide images for comparison with respect to attributes including at least one of tissue shape, size, color, character and associated information.

29. The method of claim 23, comprising associating plural case packages in the database based on a relationship of at least one variable value stored therein, presenting to a user of the workstation an option during presentation of one of said plural case packages to switch to presentation of at least one other of the case packages that is related to said one, and upon selection of said option to permit the user to compare said related case packages.

30. The method of claim 29, wherein the plural case packages are related as case packages of at least one of a same patient, a same body part, a same block, a same pathological condition, and review by a same user.

31. The method of claim 23, wherein said presentation and manipulation of the case packages comprises presentation by the digital processor to the user of a set of icons representing virtual case package files and slides in a virtual carrier and further comprising user manipulation of the icons to make selections and alteration of the icons by the digital processor to reflect progress of review of the images by the user.

32. The method of claim 31, wherein the user manipulation of the icons includes selection of case packages for review and at least one of categorization of case packages, establishing an order for the case packages, temporarily suspending review of a given one of said case packages while retaining information thereon during review of a different one of the case packages, sending and receiving case packages, and signing off on a case package after review.

33. The method of claim 31, wherein manipulation of the icons includes selection of one of case packages and information within case packages for action, and alteration of one of size, shape, position, location and appearance of an icon, by at least one of the user and the digital processor;

and the user manipulation of the icons includes selection of case packages for review and at least one of categorization of case packages, establishing an order for the case packages, temporarily suspending review of a case package while retaining information thereon, sending and receiving case packages, and signing off on a case package after review.

34. The method of claim 31, wherein the user review of the case packages comprises, in addition to displaying microscopic images, the user selecting among altering a presentation of at least one icon associated with one of a slide and a case package, annotating one of said images, entering data respecting the image, storing data representing at least one location on the image, processing the image to determine a dimension, altering a display parameter of the image with respect to one of orientation, staged position and magnification, and reverting to a same point in a series of workflow steps, with previously completed steps stored and identified as completed after temporarily departing from an associated case package.

35. The method of claim 23, wherein said presentation and manipulation of the images and the case packages includes representing plural case packages in at least one identified pile from which individual case packages can be selected, and representing plural associated slides as a slide rack from which individual slides can be selected.

36. The method of claim 35, further comprising presenting at least one display for said case packages, containing at least one representation of a slide in the case package shown as a distinct place holder representation, in a case of a slide that has been identified as prospectively expected or a formerly available, but for which at least one form of image is not presently available for viewing.

37. The method of claim 35, comprising representing said plural case packages in piles of which at least certain piles are defined by programming of the digital processor and further comprising assigning the case packages to the piles as a step in a workflow.

38. The method of claim 35, comprising representing the individual slides using icons that are one of highlighted, distinctly colored, distinctly oriented and distinctly located to identify a status that is changed by one of the processor and the user as a case package proceeds through said examination.

39. The method of claim 23, further comprising recording in the database information provided by the user during said examination, wherein the user concludes the examination of case packages by entering data indicating completion.

40. The method of claim 23, further comprising recording in the workflow database a current status of the examination of each said case package under review;

suspending review of a given case package under operation by the user of an input to switch to review of a different case package;

saving said current status of the given case package; and

thereafter switching back to said current status of the given case package to resume the examination where left off.