Marking 3D locations from ultrasound images

• US 20030055335A1
• Filed: 08/16/2002
• Published: 03/20/2003
• Est. Priority Date: 08/16/2001
• Status: Active Grant

• Current Assignee: Siemens Healthcare GMBH (Siemens AG)
• Original Assignee: Siemens Corporate Research Incorporated (Siemens AG)
• Inventors: Sauer, Frank, Khamene, Ali, Bascle, Benedicte
• Granted Patent: US 7,251,352 B2
• Time in Patent Office: Days
• US Class Current: 600/443
• CPC Class Codes: A61B 17/3403 Needle locating or guiding means guides for suture needles A61B17/0482; guiding or tracking by nuclear magnetic resonance G01R33/285
• CPC Class Codes: A61B 2017/3413 guided by ultrasound
• CPC Class Codes: A61B 8/00 Diagnosis using ultrasonic, sonic or infrasonic waves imaging of objects using sonar G01S15/00
• CPC Class Codes: A61B 8/462 characterised by constructional features of the display
• CPC Class Codes: A61B 8/466 adapted to display 3D data 3D imaging and stereoscopic displays in devices for short-range imaging with acoustic waves G01S15/8993 and G01S7/52068, respectively; 3D image rendering G06T15/00; manipulating 3D models for computer graphics G06T19/00
• CPC Class Codes: A61B 8/467 characterised by special input means
• CPC Class Codes: A61B 8/483 involving the acquisition of a 3D volume of data
• CPC Class Codes: A61B 8/5238 for combining image data of patient, e.g. merging several images from different acquisition modes into one image for image registration algorithms G06T7/30
• CPC Class Codes: A61B 90/37 Surgical systems with images on a monitor during operation
• CPC Class Codes: G01S 15/8993 Three dimensional imaging systems
• CPC Class Codes: G01S 7/52034 Data rate converters
• CPC Class Codes: G01S 7/52036 using analysis of echo signal for target characterisation
• CPC Class Codes: G01S 7/52068 Stereoscopic displays; Three-dimensional displays; Pseudo 3D displays G01S15/8993 takes precedence
• CPC Class Codes: G01S 7/52084 related to particular user interfaces special user input means for ultrasonic medical diagnostic devices A61B8/467

Abstract

A method for marking three-dimensional (3D) locations from images obtained from an ultrasound imaging system including a transducer, comprising the steps of: tracking the pose of the transducer with respect to an external 3D coordinate system; obtaining a two-dimensional (2D) ultrasound image from the transducer; marking a desired target with a marker on the 2D ultrasound image; and calculating the 3D position of the marker utilizing data from the step of tracking.

Claims

1. A method for marking three-dimensional (3D) locations from images obtained from an ultrasound imaging system including a transducer, comprising the steps of:

tracking the pose of said transducer with respect to an external 3D coordinate system;

obtaining a two-dimensional (2D) ultrasound image from said transducer;

marking a desired target with a marker on said 2D ultrasound image; and

calculating the 3D position of said marker utilizing data from said step of tracking.

2. A method for marking as recited in claim 1, wherein said step of marking comprises marking said desired target while keeping said transducer stationary.

3. A method for marking as recited in claim 2, wherein said step of marking comprises depressing a switch to initiate said marking.

4. A method for marking as recited in claim 2, wherein said step of depressing a switch to initiate said marking comprises depressing a switch attached to said transducer.

5. A method for marking as recited in claim 2, wherein said step of depressing a switch to initiate said marking comprises depressing a foot-switch.

6. A method for marking as recited in claim 2, wherein said step of marking comprises actuating a voice recognition system to initiate said marking.

7. A method for marking as recited in claim 1, wherein said step of marking comprises making a recorded image of said 2D ultrasound image, recording appurtenant pose data from said step of tracking;

and marking said recorded image.

8. A method for marking as recited in claim 7, wherein said step of marking comprises depressing a switch to initiate said marking.

9. A method for marking as recited in claim 8, wherein said step of depressing a switch to initiate said marking comprises depressing a switch attached to said transducer.

10. A method for marking as recited in claim 8, wherein said step of depressing a switch to initiate said marking comprises depressing a foot-switch.

11. A method for marking as recited in claim 8, wherein said step of marking comprises actuating a voice recognition system to initiate said marking.

14. A method for marking as recited in claim 1, wherein said step of automatically locating a potential target comprises using a search algorithm for denoising said 2D ultrasound image around said center-line.

12. A method for marking three-dimensional (3D) locations from images obtained from an ultrasound imaging system including a transducer, comprising the steps of:

tracking the pose of said transducer with respect to an external 3D coordinate system;

obtaining a two-dimensional (2D) ultrasound image from said transducer;

adjusting said pose of said transducer such that said target coincides with a designated part of said 2D ultrasound image for indicating marker location;

automatically locating a potential target of a predetermined type in said designated part;

marking said target with a marker; and

calculating the 3D position of said marker utilizing data from said tracking step.

13. A method for marking as recited in claim 12, wherein said step of adjusting said pose of said transducer comprises adjusting said pose of said transducer such that said target is on a center-line of said 2D ultrasound image.

15. A method for marking as recited in claim 13, wherein said step of automatically locating a potential target comprises using a search algorithm for de-noising said 2D ultrasound image around said center-line by using a median filter.

16. A method for marking as recited in claim 13, wherein said step of automatically locating a potential target comprises using a search algorithm for denoising said 2D ultrasound image around said center-line by using a median filter.

17. A method for marking as recited in claim 13, wherein said step of automatically locating a potential target comprises scanning along said center-line and verifying the existence of a target by using a Hough transform.

18. A method for marking as recited in claim 12, wherein said step of adjusting said pose of said transducer comprises adjusting said pose of said transducer such that said target coincides with a vertical off-center line in said 2D ultrasound image for indicating marker location.

19. A method for marking as recited in claim 12, wherein said step of adjusting said pose of said transducer comprises adjusting said pose of said transducer such that said target coincides with a line tilted at an angle in said 2D ultrasound image for indicating marker location.

20. A method for marking as recited in claim 12, wherein said step of adjusting said pose of said transducer comprises adjusting said pose of said transducer such that said target is on a designated line in said 2D ultrasound image for indicating marker location.

21. A method for marking as recited in claim 20, including a step of inputting target position along said designated line by a pointing device.

22. A method for marking as recited in claim 20, wherein said step of inputting target position comprises inputting target position by operating a thumb wheel.

23. A method for marking as recited in claim 12, wherein said step of adjusting said pose of said transducer comprises adjusting first and second poses of said transducer such that said target is on a designated line for indicating target location in said 2D ultrasound image for each pose.

24. A method for marking as recited in claim 12, wherein said step of adjusting said pose of said transducer comprises adjusting said pose of said transducer such that said target is on first and second designated intersecting lines in said 2D ultrasound image for indicating target location.

25. A method for marking three-dimensional (3D) locations from images obtained from an ultrasound imaging system including a transducer, comprising the steps of:

tracking the pose of said transducer with respect to an external 3D coordinate system;

obtaining a two-dimensional (2D) ultrasound image from said transducer;

automatically locating a potential target of a predetermined type in said 2D ultrasound image; and

optionally marking said target with a marker; and

calculating the 3D position of markers marked with a marker, utilizing data from said tracking step.

26. A method for marking three-dimensional (3D) locations from images obtained from an augmented reality (AR) ultrasound imaging system including a transducer, comprising the steps of:

tracking the pose of said transducer with respect to an external 3D coordinate system from the viewpoint of a user;

obtaining a two-dimensional (2D) ultrasound image of an object from said transducer;

overlaying said ultrasound image onto a view of said object in registration therewith from said viewpoint;

marking a desired target with a marker on said 2D ultrasound image; and

calculating the 3D position of said marker utilizing data from said step of tracking.

27. A method for marking three-dimensional (3D) locations from images obtained from a stereoscopic augmented reality (AR) ultrasound imaging system including a transducer, comprising the steps of:

tracking the pose of said transducer with respect to an external 3D coordinate system from the viewpoint of a user;

obtaining a two-dimensional (2D) ultrasound image of an object from said transducer;

overlaying said ultrasound image onto a view of said object in registration therewith from said viewpoint of a user;

marking a desired target with a marker on said 2D ultrasound image;

calculating the 3D position of said marker utilizing data from said step of tracking; and

providing stereoscopic visualization of said marker utilizing said 3D position of said marker.

28. Apparatus for marking three-dimensional (3D) locations from images obtained from an ultrasound imaging system including a transducer, comprising:

means for tracking the pose of said transducer with respect to an external 3D coordinate system;

means for obtaining a two-dimensional (2D) ultrasound image from said transducer;

means for marking a desired target with a marker on said 2D ultrasound image; and

means for calculating the 3D position of said marker utilizing data from said step of tracking.

29. Apparatus for marking as recited in claim 28, wherein said means for marking comprises a switch to initiate said marking.

30. Apparatus for marking as recited in claim 29, wherein said switch is attached to said transducer.

31. Apparatus for marking as recited in claim 29, wherein said switch comprises a foot-switch.

32. Apparatus for marking as recited in claim 28, wherein said means for marking comprises a voice recognition system to initiate said marking.

33. Apparatus for marking as recited in claim 28, wherein said means for marking comprises means for making a recorded image of said 2D ultrasound image and recording appurtenant pose data from said means for tracking and said means for marking includes means for marking said recorded image.

34. Apparatus for marking three-dimensional (3D) locations from images obtained from an ultrasound imaging system including a transducer, comprising:

means for tracking the pose of said transducer with respect to an external 3D coordinate system;

means for obtaining a two-dimensional (2D) ultrasound image from said transducer;

means for adjusting said pose of said transducer such that said target coincides with a designated part of said 2D ultrasound image for indicating marker location;

means for automatically locating a potential target of a predetermined type in said designated part;

means for marking said target with a marker; and

means for calculating the 3D position of said marker utilizing data from said tracking step.

35. Apparatus for marking as recited in claim 34, wherein said designated part comprises a center-line of said 2D ultrasound image.

36. Apparatus for marking as recited in claim 34, wherein said means for automatically locating a potential target comprises a search algorithm for de-noising said 2D ultrasound image around said center-line.

37. Apparatus for marking as recited in claim 34, wherein said means for automatically locating a potential target comprises a search algorithm for de-noising said 2D ultrasound image around said center-line by using a median filter.

38. Apparatus for marking as recited in claim 34, wherein said step of automatically locating a potential target comprises means for scanning along said center-line and verifying the existence of a target by using a Hough transform.

39. Apparatus for marking as recited in claim 34, wherein said wherein said designated part comprises a vertical off-center line in said 2D ultrasound image for indicating marker location.

40. Apparatus for marking as recited in claim 34, wherein said designated part comprises a line tilted at an angle in said 2D ultrasound image for indicating marker location.

41. Apparatus for marking as recited in claim 34, wherein said designated part comprises a designated line in said 2D ultrasound image for indicating marker location.

42. Apparatus for marking as recited in claim 41, including a pointing device for inputting marker location.

43. A method for marking as recited in claim 42, wherein said pointing device comprises a thumbwheel.

44. Apparatus for marking three-dimensional (3D) locations from images obtained from an ultrasound imaging system including a transducer, comprising:

means for tracking the pose of said transducer with respect to an external 3D coordinate system;

means for obtaining a two-dimensional (2D) ultrasound image from said transducer;

means for automatically locating a potential target of a predetermined type in said 2D ultrasound image;

means for optionally marking said target with a marker; and

means for calculating the 3D position of markers marked with a marker, utilizing data from said tracking step.

45. Apparatus for marking three-dimensional (3D) locations from images obtained from an augmented reality (AR) ultrasound imaging system including a transducer, comprising:

means for tracking the pose of said transducer with respect to an external 3D coordinate system from the viewpoint of a user;

means for obtaining a two-dimensional (2D) ultrasound image of an object from said transducer;

means for overlaying said ultrasound image onto a view of said object in registration therewith from said viewpoint;

means for marking a desired target with a marker on said 2D ultrasound image; and

means for calculating the 3D position of said marker utilizing data from said step of tracking.

46. Apparatus for marking three-dimensional (3D) locations from images obtained from a stereoscopic augmented reality (AR) ultrasound imaging system including a transducer, comprising:

means for tracking the pose of said transducer with respect to an external 3D coordinate system from the viewpoint of a user;

means for obtaining a two-dimensional (2D) ultrasound image of an object from said transducer;

means for overlaying said ultrasound image onto a view of said object in registration therewith from said viewpoint of a user;

means for marking a desired target with a marker on said 2D ultrasound image;

means for calculating the 3D position of said marker utilizing data from said step of tracking; and

means for providing stereoscopic visualization of said marker utilizing said 3D position of said marker.