Echocardiography workstation

• US 6,674,879 B1
• Filed: 11/16/2000
• Issued: 01/06/2004
• Est. Priority Date: 03/30/1998
• Status: Expired due to Fees

• Current Assignee: Echovision Incorporated
• Original Assignee: Echovision Incorporated
• Inventors: Weisman, Jeffrey, Zietz, Stanley
• Primary Examiner(s): Patel, Jayanti K.
• Application Number:
  US09/647,438
• Time in Patent Office: 1,146 Days
• Field of Search: 382/100, 382/128-134, 128/653.1, 128/920, 378/98.2, 600/301, 600/509, 434/262
• US Class Current: 382/128
• CPC Class Codes: A61B 5/1075 for measuring dimensions by non-invasive methods, e.g. for determining thickness of tissue layer A61B8/0858 takes precedence
• CPC Class Codes: A61B 5/7289 Retrospective gating, i.e. associating measured signals or images with a physiological event after the actual measurement or image acquisition, e.g. by simultaneously recording an additional physiological signal during the measurement or image acquisition
• CPC Class Codes: A61B 6/503 for diagnosis of the heart
• CPC Class Codes: A61B 8/06 Measuring blood flow measuring volume flow in general G01F, e.g. G01F1/66, G01F1/72; measuring speed of fluids in general G01P5/00
• CPC Class Codes: A61B 8/08 Detecting organic movements or changes, e.g. tumours, cysts, swellings A61B8/02 - A61B8/06 take precedence
• CPC Class Codes: A61B 8/0883 for diagnosis of the heart
• CPC Class Codes: A61B 8/463 characterised by displaying multiple images or images and diagnostic data on one display composite display in devices for short-range imaging with acoustic waves G01S7/52074
• CPC Class Codes: A61B 8/465 adapted to display user selection data, e.g. icons or menus
• CPC Class Codes: A61B 8/488 involving Doppler signals
• CPC Class Codes: G01S 7/52025 for pulse systems G01S7/52034 takes precedence
• CPC Class Codes: G01S 7/52046 Techniques for image enhancement involving transmitter or receiver image enhancement by image data processing G06T5/00
• CPC Class Codes: G01S 7/52066 Time-position or time-motion displays
• CPC Class Codes: G01S 7/52074 Composite displays, e.g. split-screen displays; Combination of multiple images or of images and alphanumeric tabular information
• CPC Class Codes: G01S 7/52088 involving retrospective scan line rearrangements medical diagnostic ultrasound devices involving retrospective matching to a physiological signal A61B8/5284

Abstract

A digital image processing system for enhancing the image quality and diagnostic capabilities of conventional medical diagnostic ultrasound imaging systems and, more particularly, to an echocardiography workstation (10) which provides speckle reduction (200), edge detection, color quantitation (306), automatic diagnostic features. a built-in Help system for echocardiography, automatic quantitative analysis of left ventricular function, tomographic perfusion display (36), 3-D analysis, and report generation for improved analysis of echocardiograms.

Claims

1. An image processing system, comprising:

an image source which digitizes and outputs a video image of a patient'"'"'s anatomy;

an image processor which implements at least one software process for processing a video image from said image source, said at least one software process removing noise from said video image while enhancing the definition of an internal structure of the patient'"'"'s anatomy, automatically identifying a border of said internal structure and tracking movement of said border, and quantitating the extent of motion of at least said border of said internal structure of the patient'"'"'s anatomy during a selected time frame; and

a display which displays a processed video image representative of the extent of motion of at least said border of said internal structure of the patient'"'"'s anatomy during said selected time frame,

wherein said at least one software process removes noise from said video image while enhancing the definition of an internal structure of the patient'"'"'s anatomy by performing a non-linear gray scale transformation on a received video image, recursively averaging the transformed video image, and performing a non-linear gray scale morphology operation on said transformed and averaged video image.

2. An image processing system as in claim 1, further comprising a spatial locator that registers position and orientation of a two-dimensional video image from said image source in a three-dimensional spatial coordinate system by overlaying a three-dimensional model of the patient'"'"'s anatomy onto the two-dimensional video image at a corresponding position in three-dimensional space on said display.

3. An image processing system as in claim 1, further comprising a graphical user interface which permits a viewer to perform at least one of the following functions:

view video images output by said image source, select alternative processing options, consult reference sources, generate reports from pull-down menus, and store, retrieve, and transmit said video images and reports including diagnostic interpretation of said video images by the viewer.

4. An image processing system as in claim 1, wherein said at least one software process implements said morphology operation by performing the steps of choosing a neighborhood of each pixel to be displayed, selecting rays of points in N directions in said neighborhood of said each pixel to be displayed, determining a minimum pixel value and a maximum pixel value along each ray of points and a minimum of the maximum pixel values for said rays of points (MIN of MAX) and a maximum of said minimum pixel values for said rays of points (MAX of MIN), and replacing said each pixel to be displayed with an average of said MIN of MAX and MAX of MIN values.

5. An image processing system, comprising:

an image source which digitizes and outputs a video image of a patient'"'"'s heart, said image source including an image sequence synchronizer that synchronizes respective video image sequences of different frame lengths of the patient'"'"'s heart;

an image processor which implements at least one software process for processing a video image from said image source, said at least one software process removing noise from said video image while enhancing the definition of a heart wall of the patient'"'"'s heart, automatically identifying a border of said heart wall and tracking movement of said heart wall, and quantitating the extent of motion of at least said border of said heart wall of the patient'"'"'s heart during a selected time frame; and

a display which displays a processed video image representative of the extent of motion of at least said border of said heart wall of the patient'"'"'s heart during said selected time frame, wherein said image processor presents said respective synchronized video images of the patient'"'"'s heart to said display for concurrent display, and

wherein said respective video image sequences of the patient'"'"'s heart are synchronized by a synchronization process which computes target frame counts for systolic and diastolic portions of a cardiac cycle of the patient'"'"'s heart, where the target systolic frames are {square root over (3.6N)} and the target diastolic frames are N minus the target systolic frames, where N is the number of frames in a cardiac cycle of the patient'"'"'s heart having a maximum number of frames in the cardiac cycle.

6. An image processing system as in claim 5, wherein said respective video image sequences of the patient'"'"'s heart are synchronized and displayed in a synchronized M frame representation on said display whereby up to M viewing windows may be viewed concurrently.

7. An image processing system as in claim 5, wherein said synchronization process computes, for each cardiac cycle of each sequence of video images, a number of systolic frames to add as the target systolic frame minus {square root over (3.6N)} (ADD SYS FRAMES), and a number of diastolic frames to add as the target diastolic frame minus (N−

{square root over (3.6N)}) (ADD DIA FRAMES), where N is the number of frames in the cardiac cycle.

8. An image processing system as in claim 7, wherein said synchronization process repeats the systolic frames from frame {square root over (3.6N)} to frame {square root over (3.6N)} minus ADD SYS FRAMES, repeats the diastolic frames for all frames in a range {square root over (3.6N)}+1 to N by int(ADD DIA FRAMES/{square root over (3.6N)}) times, and repeats all frames in a range M={square root over (3.6N)}+[N−

{square root over (3.6N)}−

MOD(ADD DIA FRAMES)* {square root over (3.6N)})]/2 to M+MOD(ADD DIA FRAMES, {square root over (3.6N)}) one time each, whereby each sequence of images of a cardiac cycle has the same number of image frames.

9. An image processing system, comprising:

an image source which digitizes and outputs a video image of a patient'"'"'s heart;

an image processor which implements at least one software process for processing a video image from said image source, said at least one software process removing noise from said video image while enhancing the definition of a heart wall of the patient'"'"'s heart, automatically identifying a border of said heart wall and tracking movement of said heart wall, and quantitating the extent of motion of at least said border of said heart wall of the patient'"'"'s heart during a selected time frame; and

a display which displays a processed video image representative of the extent of motion of at least said border of said heart wall of the patient'"'"'s heart during said selected time frame,

wherein said at least one software process automatically identifies a border of said heart wall and tracks movement of said border by tagging frames of said video image of the patient'"'"'s heart corresponding to end-diastole (ED frame) and end-systole (ES frame) of a selected cardiac cycle, tracing said border of said heart wall, computing thresholds from the ED frame, and detecting borders of said heart wall for all frames of said selected cardiac cycle using said thresholds.

10. An image processing system as in claim 9, wherein said at least one software process further comprises an expert system that evaluates data from said video image of the patient'"'"'s heart and suggests to a viewer various diagnostic possibilities that are consistent with the video image of the patient'"'"'s heart.

11. An image processing system as in claim 9, wherein said at least one software process further comprises a report generator which accepts diagnostic data from a viewer and generates a diagnostic report including at least said diagnostic data.

12. An image processing system as in claim 9, wherein said at least one software process further comprises a help system which is accessed by a viewer during interpretation of the video image of the patient'"'"'s heart to provide descriptions of abnormalities of the heart structure with lists of known causes of said abnormalities.

13. An image processing system as in claim 9, wherein said at least one software process computes thresholds from the ED frame by setting a preset threshold, projecting N rays spaced at approximately equal angles from a centroid of the traced heart wall border, determining image gray-level gradient points along each of said N rays in a vicinity of said traced heart wall border, and, if a gradient point larger than said preset threshold is found along one of said N rays, replacing a border point on said traced heart wall border with said gradient point.

14. An image processing system as in claim 13, wherein said at least one software process further determines, for each of M threshold values from 0 to a maximum gray level, a histogram equalized threshold value, thresholds said ED frame using the histogram equalized threshold value, performs morphological opening and closing of said ED frame, finds, starting on the traced heart wall border for said ED frame, a change state point on each ray where the thresholded ED image changes state, and, for each ray, determines an error distance between the change state point and the traced heart wall border.

15. An image processing system as in claim 14, wherein said at least one software process further divides a portion of said ED frame representing a left ventricle of the patient'"'"'s heart into N pie-shaped segments, identifies a selected threshold for each segment that corresponds to a smallest average error distance for that segment, compares the video image to the selected threshold, computes an average error distance for each segment using the selected threshold, and, for each segment, saves the selected threshold value as representative of said border of the patient'"'"'s heart wall if all points of the selected threshold are outside of an existing heart wall border or if the average error distance is less than a previous average error distance multiplied by a predetermined weighting factor.

16. An image processing system as in claim 15, wherein said at least one software process compares an image in each frame of said video signal in said selected cardiac cycle to selected threshold values for each segment, performs morphological opening and closing of each image, and invalidates border points which are beyond said selected threshold values for the segment containing said border points.

17. An image processing system as in claim 16, wherein said at least one software process finds image gray-level gradient points along each ray in the vicinity of the border of the previous frame of said video signal for invalidated border points and invalidates the gradient points if a border point on a corresponding ray of the previous frame of said video signal was also determined from image gray-level gradient points.

18. An image processing system as in claim 16, wherein said at least one software process fills in gaps less than a predetermined number of rays in said traced border of the heart wall by interpolating border points on said traced border of the heart wall on either side of the gaps.

19. An image processing system as in claim 18, wherein said at least one software process fills in gaps greater than said predetermined number of rays in said traced border of the heart wall by replacing border points in said gaps with Doppler-predicted border points for a present frame of said video signal.

20. An image processing system as in claim 18, wherein said at least one software process fills in gaps greater than said predetermined number of rays in said traced border of the heart wall by computing for two rays on either side of a gap distances from the centroid of the traced heart wall border to the border points on either side of said gap for a present frame and a previous frame of said video signal, computing an average distance of said distances for said present frame and said previous frame, and replacing all border points in said gap with points that are the same distance from the centroid of the traced heart wall border as the border points on corresponding rays of the previous frame plus the computed average distance difference.

21. An image processing system as in claim 20, wherein said at least one software process replaces invalid border points in said traced border of the heart wall with border points from said previous frame of said video signal after correction for translation of the centroid of the traced heart wall border from said previous frame to the present frame of said video signal.

22. An image processing system as in claim 21, wherein said at least one software process translates the centroid of the traced heart wall border from said previous frame to the present frame of said video signal by tracing the heart wall border for said previous frame and said present frame of said video signal, dividing the traced heart wall border from said previous frame into a first predetermined number of border points equally spaced with respect to arc length along said traced heart wall border, dividing the traced heart wall border of said present frame into a second predetermined number of border points equally spaced with respect to arc length along said traced wall border, finding, for each point on the traced heart wall border of the previous frame, a replacement point on the traced heart wall border of the present frame which is nearest to a point on the traced heart wall border of the previous frame, and replacing a corresponding border point of the previous frame with the replacement point.

23. An image processing system, comprising:

an image source which digitizes and outputs a video image of a patient'"'"'s heart;

an image processor which implements at least one software process for processing a video image from said image source, said at least one software process removing noise from said video image while enhancing the definition of a heart wall of the patient'"'"'s heart, automatically identifying a border of said heart wall and tracking movement of said heart wall, and quantitating the extent of motion of at least said border of said heart wall of the patient'"'"'s heart during a selected time frame; and

a display which displays a processed video image representative of the extent of motion of at least said border of said heart wall of the patient'"'"'s heart during said selected time frame,

wherein said at least one software process quantitates the extent of motion of at least said border of said heart wall from end-diastole to end-systole by representing frame to frame movement of said heart wall in a color which is different from a background color of the processing video image displayed on said display.

24. An image processing system as in claim 23, wherein said at least one software process color codes the extent of radial movement of radial sections of said heart wall from end-diastole to end-systole.

25. A method of imaging internal anatomy of a patient, comprising the steps of:

capturing a video image of the patient'"'"'s internal anatomy;

removing noise from said video image while enhancing the definition of an internal structure of the patient'"'"'s anatomy;

automatically identifying a border of said internal structure;

tracking movement of said border;

quantitating the extent of motion of at least said border of said internal structure of the patient'"'"'s anatomy during a selected time frame; and

displaying processed video images representative of the extent of motion of at least said border of said internal structure of the patient'"'"'s anatomy during said selected time frame,

wherein said noise removing step comprises the steps of performing a non-linear gray scale transformation on a captured video image, recursively averaging the transformed video image, and performing a non-linear gray scale morphology operation on said transformed and averaged video image.

26. A method as in claim 25, further comprising the step of registering a position and orientation of two-dimensional captured video images in a three-dimensional spatial coordinate system by overlaying a three-dimensional model of the patient'"'"'s anatomy onto the two-dimensional video image at a corresponding position in three-dimensional space for display in three dimensions.

27. A method as in claim 26, further comprising the step of calculating a 3-D volume of the patient'"'"'s anatomy from said two-dimensional captured video images and said three-dimensional model of the patient'"'"'s anatomy.

28. A method as in claim 26, further comprising the steps of injecting a contrast agent into the patient'"'"'s bloodstream after capturing the video image of the patient'"'"'s internal anatomy at M degree intervals, capturing a video image of the patient'"'"'s internal anatomy at said M degree intervals at points of injection of said contrast agent, and tomographically displaying said video images before and after injection of said contrast agent.

29. A method as in claim 25, wherein morphology operation comprises the steps of choosing a neighborhood of each pixel to be displayed, selecting rays of points in N directions in said neighborhood of said each pixel to be displayed, determining a minimum pixel value and a maximum pixel value along each ray of points and a minimum of the maximum pixel values for said rays of points (MIN of MAX) and a maximum of said minimum pixel values for said rays of points (MAX of MIN), and replacing said each pixel to be displayed with an average of said MIN of MAX and MAX of MIN values.

30. A method of imaging internal anatomy of a patient, comprising the steps of:

capturing a video image of the patient'"'"'s internal anatomy;

removing noise from said video image while enhancing the definition of an internal structure of the patient'"'"'s anatomy;

automatically identifying a border of said internal structure;

tracking movement of said border;

quantitating the extent of motion of at least said border of said internal structure of the patient'"'"'s anatomy during a selected time frame; and

displaying processed video images representative of the extent of motion of at least said border of said internal structure of the patient'"'"'s anatomy during said selected time frame,

wherein said video image is a video image of a patient'"'"'s heart and said step of automatically identifying a border of said internal structure comprises the steps of tagging frames of said video image of the patient'"'"'s heart corresponding to end-diastole (ED frame) and end-systole (ES frame) of a selected cardiac cycle, tracing said border of said heart wall, computing thresholds from the ED frame, and detecting borders of said heart wall for all frames of said selected cardiac cycle using said thresholds.

31. A method as in claim 30, wherein said step of automatically identifying a border of said internal structure comprises the steps of computing thresholds from the ED frame by setting a preset threshold, projecting N rays spaced at approximately equal angles from a centroid of the traced heart wall border, determining image gray-level gradient points along each of said N rays in a vicinity of said traced heart wall border, and, if a gradient point larger than said preset threshold is found along one of said N rays, replacing a border point on said traced heart wall border with said gradient point.

32. A method as in claim 31, comprising the further step of determining, for each of M threshold values from 0 to a maximum gray level, a histogram equalized threshold value, thresholding said ED frame using the histogram equalized threshold value, performing morphological opening and closing of said ED frame, finding, starting on the traced heart wall border for said ED frame, a change state point on each ray where the thresholded ED image changes state, and, for each ray, determining an error distance between the change state point and the traced heart wall border.

33. A method as in claim 32, comprising the further steps of dividing a portion of said ED frame representing a left ventricle of the patient'"'"'s heart into N pie-shaped segments, identifying a selected threshold for each segment that corresponds to a smallest average error distance for that segment, comparing the video image to the selected threshold, computing an average error distance for each segment using the selected threshold, and, for each segment, saving the selected threshold value as representative of said border of the patient'"'"'s heart wall if all points of the selected threshold are outside of an existing heart wall border or if the average error distance is less than a previous average error distance multiplied by a predetermined weighting factor.

34. A method as in claim 33, comprising the further steps of comparing an image in each frame of said video signal in said selected cardiac cycle to selected threshold values for each segment, performing morphological opening and closing of each image, and invalidating border points which are beyond said selected threshold values for the segment containing said border points.

35. A method as in claim 34, comprising the further steps of finding image gray-level gradient points along each ray in the vicinity of the border of the previous frame of said video signal for invalidated border points and invalidating the gradient points if a border point on a corresponding ray of the previous frame of said video signal was also determined from image gray-level gradient points.

36. A method as in claim 34, comprising the additional step of filling in gaps less than a predetermined number of rays in said traced border of the heart wall by interpolating border points on said traced border of the heart wall on either side of the gaps.

37. A method as in claim 36, comprising the additional step of filling in gaps greater than said predetermined number of rays in said traced border of the heart wall by replacing border points in said gaps with Doppler-predicted border points for a present frame of said video signal.

38. A method as in claim 36, comprising the additional step of filling in gaps greater than said predetermined number of rays in said traced border of the heart wall by computing for two rays on either side of a gap distances from the centroid of the traced heart wall border to the border points on either side of said gap for a present frame and a previous frame of said video signal, computing an average distance of said distances for said present frame and said previous frame, and replacing all border points in said gap with points that are the same distance from the centroid of the traced heart wall border as the border points on corresponding rays of the previous frame plus the computed average distance difference.

39. A method as in claim 38, wherein said step of replacing invalid border points comprises the step of replacing invalid border points in said traced border of the heart wall with border points from said previous frame of said video signal after correcting for translation of the centroid of the traced heart wall border from said previous frame to the present frame of said video signal.

40. A method in claim 39, wherein said step of translating the centroid of the traced heart wall border from said previous frame to the present frame of said video signal comprises the steps of tracing the heart wall border for said previous frame and said present frame of said video signal, dividing the traced heart wall border from said previous frame into a first predetermined number of border points equally spaced with respect to arc length along said traced heart wall border, dividing the traced heart wall border of said present frame into a second predetermined number of border points equally spaced with respect to arc length along said traced wall border, finding, for each point on the traced heart wall border of the previous frame, a replacement point on the traced heart wall border of the present frame which is nearest to a point on the traced heart wall border of the previous frame, and replacing a corresponding border point of the previous frame with the replacement point.

41. A method of imaging internal anatomy of a patient, comprising the steps of:

capturing a video image of the patient'"'"'s internal anatomy;

removing noise from said video image while enhancing the definition of an internal structure of the patient'"'"'s anatomy;

automatically identifying a border of said internal structure;

tracking movement of said border;

quantitating the extent of motion of at least said border of said internal structure of the patient'"'"'s anatomy during a selected time frame; and

displaying processed video images representative of the extent of motion of at least said border of said internal structure of the patient'"'"'s anatomy during said selected time frame,

wherein said video image is a video image of a patient'"'"'s heart, and said step of quantitating the extent of motion of at least said border of said heart wall from end-diastole to end-systole during said selected time frame comprising the step of quantitating the extent of motion of at least said border by representing frame to frame movement of said heart wall in a color which is different from a background color of the processing video image displayed on said display.

42. A method as in claim 41, wherein said quantitating step comprises the additional step of color coding the extent of radial movement of radial sections of said heart wall from end-diastole to end-systole.

43. A method of imaging a patient'"'"'s heart, comprising the steps of:

capturing respective video images of a patient'"'"'s heart;

removing noise from said video images while enhancing the definition of an internal structure of the patient'"'"'s heart;

automatically identifying a border of said heart;

tracking movement of said border;

quantitating the extent of motion of at least said border of said heart during a selected time frame;

synchronizing said respective video images of the patient'"'"'s heart; and

displaying said synchronized images in a synchronized M frame representation whereby up to M viewing windows may be viewed concurrently,

wherein said synchronization step comprises the step of computing target frame counts for systolic and diastolic portions of a cardiac cycle of the patient'"'"'s heart, where the target systolic frames are {square root over (3.6N)} and the target diastolic frames are N minus the target systolic frames, where N is the number of frames in a cardiac cycle of the patient'"'"'s heart having a maximum number of frames in the cardiac cycle.

44. A method in claim 43, wherein said synchronization step comprises the additional step of computing, for each cardiac cycle of each sequence of video images, a number of systolic frames to add as the target systolic frame minus {square root over (3.6N)} (ADD SYS FRAMES), and a number of diastolic frames to add as the target diastolic frame minus (N−

{square root over (3.6N)}) (ADD DIA FRAMES), where N is the number of frames in the cardiac cycle.

45. A method as in claim 44, wherein said synchronization step comprises the steps of repeating the systolic frames from frame {square root over (3.6N)} to frame {square root over (3.6N)} minus ADD SYS FRAMES, repeating the diastolic frames for all frames in a range {square root over (3.6N)}+1 to N by int(ADD DIA FRAMES/{square root over (3.6N)}) times, and repeating all frames in a range M={square root over (3.6N)}+[N−

{square root over (3.6N)}−

MOD(ADD DIA FRAMES)*{square root over (3.6N)}]/2 to M+MOD(ADD DIA FRAMES, {square root over (3.6N)}) one time each, whereby each sequence of images of a cardiac cycle has the same number of image frames.

46. A method as in claim 43, comprises the additional steps of accepting diagnostic data from a viewer and generating a diagnostic report including at least said diagnostic data.