Stereo image processing for radiography

• US 6,862,364 B1
• Filed: 01/18/2001
• Issued: 03/01/2005
• Est. Priority Date: 10/27/1999
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Berestov, Alexander
• Primary Examiner(s): Ahmed, Samir
• Assistant Examiner(s): BHATNAGAR, ANAND P
• Application Number:
  US09/767,210
• Time in Patent Office: 1,503 Days
• Field of Search: 382/128, 382/132, 382/154, 382/209, 382/286, 382/287, 382/288, 382/289, 382/218, 382/274, 378/8, 378/41, 378/98, 378/98.2, 378/42, 358/487, 356/12, 348/42, 348/45, 345419-427
• US Class Current: 382/132
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/10116 X-ray image
• CPC Class Codes: G06T 5/94 based on local image properties, e.g. for local contrast enhancement
• CPC Class Codes: G06V 2201/03 Recognition of patterns in medical or anatomical images
• CPC Class Codes: G06V 2201/12 Acquisition of 3D measurements of objects
• CPC Class Codes: H04N 13/128 Adjusting depth or disparity
• CPC Class Codes: H04N 13/133 Equalising the characteristics of different image components, e.g. their average brightness or colour balance

Abstract

Pairs of stereo Xray radiographs are obtained from an X-ray imaging system and are digitized to form corresponding pairs of stereo images (602, 604). The pairs of stereo images (602, 604) are adjusted (410) to compensate for gray-scale illumination differences by grouping and processing pixel groups in each pair of images. Distortion in the nature of depth plane curvature and Keystone distortion due to the toed-in configuration of the X-ray imaging system are eliminated (412). A screen parallax for the pair of stereo images is adjusted (414) to minimize depth range so as to enable a maximum number of users to view the stereoscopic image, and particular features of interest.

Claims

1. A method for stereo image processing of an object, the method comprising the step of:

obtaining at least a pair of stereo images each having a plurality of pixels, said stereo images being a digital representation of a corresponding pair of stereo radiographs taken of the object, wherein the pair of stereo radiographs is obtained by an X-ray imaging device, the X-ray imaging device being moved between times when a first radiograph of the pair of stereo radiographs and a second radiograph pair of the stereo radiographs are obtained;

correcting illumination errors within the pair of stereo images;

removing distortions from the pair of stereo images;

combining pixels of the pair of stereo images into a composite image; and

adjusting a corresponding screen parallax for the composite image, wherein the step of correcting illumination errors comprises the substeps of;

selecting a first group of pixels from a first image of the pair of stereo images;

selecting a second group of pixels from a second image of the pair of stereo images, the second group of pixels being respectively associated with the first group of pixels;

determining an intensity level for each pixel in both groups of pixels;

determining a mean intensity level and a variance for each of the first and second groups of pixels;

equalizing the mean intensity level and the variance of the first group and the second group; and

adjusting the pixels to equalize the mean intensity level and the variance of the first group and the second group.

2. The method of claim 1, wherein the substep of equalizing the mean intensity level and the variance comprises the further substeps of:

determining a new intensity level for a first pixel and a second pixel, the first pixel being in the first group and the second pixel being in the second group, the new intensity level being between the intensity level of the first pixel and the intensity level of the second pixel; and

altering the intensity levels of the first and second pixels to be the new intensity level.

3. The method of claim 1, wherein the intensity level determined for each pixel comprises grayscale values.

4. The method of claim 1, wherein the substep of selecting a first group of pixels comprises the further substep of:

selecting a plurality of pixels forming a line perpendicular to a direction of motion of the X-ray imaging device.

5. The method of claim 1, wherein the substep of selecting a first group of pixels comprises the further substep of:

selecting a plurality of pixels forming a longitudinal line perpendicular to a direction of motion of the X-ray imaging device at a point where the longitudinal line intersects a direction of motion of the X-ray imaging device.

6. The method of claim 1, wherein the stereo radiographs comprise X-ray images.

7. The method of claim 1, wherein the step of correcting illumination errors comprises the substeps of:

selecting a first group of pixels from a first image of the pair of stereo images;

selecting a second group of pixels from a second image of the pair of stereo images;

determining a first mean intensity value and a first variance for the first group of pixels;

determining a second mean intensity value and a second variance for the second group of pixels;

equalizing the first mean intensity value and first variance with the second mean intensity value and second variance, respectively; and

adjusting at least one of the first group of pixels and the second group of pixels in response to the substep of equalizing the first mean intensity value and first variance with the second mean intensity value and second variance.

8. The method of claim 7, wherein the substep of adjusting at least one of the first group pf pixels and the second group of pixels comprises the further substep of:

adjusting pixel intensities for the first group of pixels such that the first mean intensity value is equal to the second mean intensity value.

9. The method of claim 7, wherein the substep of adjusting at least one of the first group of pixels and the second group of pixels comprises the further substep of:

adjusting pixel intensities for the second group of pixels such that the second mean intensity value is equal to the first mean intensity value.

10. The method of claim 7, further comprising the substeps of:

determining a third mean intensity value by adjusting pixel intensities of he first group of pixels, wherein said third mean intensity value is between the first mean intensity value and the second mean intensity value;

adjusting the first mean intensity value to be equal to the third mean intensity value; and

adjusting the second mean intensity value to be equal to the third mean intensity value.

11. The method of claim 1, wherein the step of adjusting a corresponding screen parallax comprises the substeps of:

displaying the composite image on a display device, the display device including a viewing surface; and

locating the object in the composite image near the viewing surface in order to minimize depth range.

12. A method for stereo image processing of an object, the method comprising the step of:

obtaining at least a pair of stereo images each having a plurality of pixels, said stereo images being a digital representation of a corresponding pair of stereo radiographs taken of the object;

correcting illumination errors within the pair of stereo images;

removing distortions from the pair of stereo images;

combining pixels of the pair of stereo images into a composite image; and

adjusting a corresponding screen parallax for the composite image,

wherein the step of removing distortions comprises the substeps of;

removing depth plane curvature amongst the pair of stereo images by determining a representation of the pair of stereo images, the representation having a parallel geometry with respect to one or more X-ray sources used to form the radiographs, and determining pixel values for the representation based on the plurality of pixels in the pair of stereo images; and

removing keystone distortion amongst the pair of stereo images.

13. The method of claim 12, wherein the substep of removing keystone distortion comprises the substeps of:

determining epipolar geometry amongst the pair of images; and

shifting the pixels to remove the keystone distortion based on the epipolar geometry determined.

14. The method of claim 13, wherein the substep of determining epipolar geometry comprises the substeps of:

creating a left and right search column on at least one of the pair of stereo images, wherein the pair of stereo images includes at least some overlap area and at least one of the columns includes at least part of the overlap area;

creating two sets of gray-scale sub-images, one set of the sub-images for each of the pair of stereo images;

running a matching algorithm on each point in the right and left search column of each sub-image pair;

calculating a vertical shift between points identified as matching by the matching algorithm;

selecting points with identical vertical shift values; and

aligning the points that were not selected in the images by interpolating amongst vertical shift values for each column.

15. The method of claim 12, wherein the object comprises at least one item disposed therewithin, and wherein the substep of removing keystone distortions includes further sub steps, comprising:

determining a location of at least one physical pointer disposed around the object, said physical pointer being captured in the pair of radiographs and represented in the pair of stereo images;

estimating epipolar geometry and horizontal and vertical distortions using the location of the physical pointer in the pair of stereo images; and

adjusting at least one image of the pair of stereo images vertically and horizontally to correct for any estimated distortions.

16. The method of claim 15, further comprising the substep of calculating a location of the item.

17. The method of claim 15, wherein the physical pointer comprises an ink mark.

18. The method of claim 15, wherein the physical pointer comprises a metal ball.

19. The method of claim 15, wherein the physical pointer comprises a foil sticker.

20. The method of claim 15, wherein the physical pointer is disposed within the object.

21. A method for stereo image processing of an object, the method comprising the step of:

obtaining at least a pair of stereo images each having a plurality of pixels, said stereo images being a digital representation of a corresponding pair of stereo radiographs taken of the object;

correcting illumination errors within the pair of stereo images;

removing distortions from the pair of stereo images;

combining pixels of the pair of stereo images into a composite image; and

adjusting a corresponding screen parallax for the composite image,

wherein the object comprises at least one item, and the step of removing distortions from the pair of stereo images includes sub steps for adjusting the radiographs, the substeps comprising;

locating at least one physical pointer disposed around the object;

capturing the pair of stereo radiographs using a radiograph imaging device, wherein the physical pointer is captured in the pair of radiographs;

determining a location of the physical pointer;

estimating geometry and horizontal and vertical distortions using the location of the physical pointer appearing in the pair of stereo images; and

adjusting at least one image of the pair of stereo images vertically and horizontally to correct for any estimated distortions.

22. The method of claim 21, wherein the physical pointer is disposed within the object, and the geometry includes epipolar geometry.

23. The method of claim 21, wherein the object comprises a body, and the item is selected from the group comprising a bullet, bone, muscle and tissue.

24. A computer-implemented method for stereo image processing of at least one pair of stereo images of an object, wherein the pair of stereo images includes a plurality of pixels and is obtained from a pair of stereo radiographs taken of the object, the method comprising the steps of:

correcting illumination errors within the pair of stereo images;

removing distortions from the pair of stereo images; and

adjusting a corresponding screen parallax for a composite image, the composite image being a combination of the pixels of the pair of stereo images,

wherein the step of removing distortions from the pair of stereo images comprises the substeps of;

removing depth plane curvature amongst the first and second images by determining a representation of the pair of stereo images, the representation having a parallel geometry with respect to one or more X-ray sources used to form the radiographs and determining pixel values for the representation based on the plurality of pixels in the pair of stereo images; and

removing keystone distortion amongst the first and second images.

25. The computer-implemented method of claim 24, wherein the step of correcting illumination errors comprises the substeps of:

selecting a first group of pixels from a first image of the pair of stereo images;

selecting a second group of pixels from a second image of the pair of stereo images, the second group of pixels being associated with the first group of pixels;

determining an intensity level for each pixel in both groups of pixels;

determining a mean intensity level and a variance for each of the first and second groups of pixels;

equalizing the mean intensity level and the variance of the first group and the second group; and

adjusting the pixels to equalize the mean intensity level and the variance of the first group and the second group.

26. The computer-implemented method of claim 25, wherein the substep of equalizing the mean intensity level and the variance comprises the further substeps of:

determining a new intensity level for a first pixel and a second pixel, the first pixel being in the first group and the second pixel being in the second group, the new intensity level being between the intensity level of the first pixel and the intensity level of the second pixel; and

altering the intensity level of the first and second pixels to be the new intensity level.

27. The computer-implemented method of claim 25, wherein the second group of pixels is selected according to a matching algorithm.

28. The computer-implemented method of claim 25, wherein the second group of pixels is selected at a location in the second image corresponding to an equivalent location in the first image of the first group of pixels.

29. The computer-implemented method of claim 25, wherein the intensity level comprises grayscale values.

30. The computer-implemented method of claim 24, wherein the substep of removing keystone distortion comprises the substeps of:

determining epipolar geometry amongst the pair of stereo images; and

shifting the pixels to remove the keystone distortion based on the epipolar geometry determined.

31. The computer-implemented method of claim 24, wherein the step of adjusting a corresponding screen parallax for a composite image comprises the substeps of:

displaying the pair of stereo images on a display device, the display device including a viewing surface; and

locating the object near the viewing surface in order to minimize depth range.

32. A system for stereo image processing of an object, said system comprising:

obtaining means for obtaining at least one pair of stereo images, the stereo images being a digital representation of a corresponding pair of stereo radiographs taken of the object, the pair of stereo images including a first image and a second image both having a plurality of pixels;

communicatively coupled to the obtaining means, means for correcting illumination errors within the pair of stereo images;

coupled to the means for correcting illumination errors, means for removing distortions from the pair of stereo images; and

coupled to the means for removing distortions, means for adjusting a corresponding screen parallax for a composite image, the composite image being a combination of the pixels of the first and second Images,

wherein the means for removing distortions from the pair of stereo images comprises;

means for rotating the first and second images to eliminating depth plane curvature therewithin; and

coupled to the means for eliminating depth plane curvature, means for eliminating keystone distortion within the pair of stereo images.

33. The system of claim 32, wherein the obtaining means comprises an X-ray imaging device for providing the pair of stereo radiographs.

34. The system of claim 33, wherein the obtaining means further comprises a converter coupled to the X-ray imaging device, the converter for converting the pair of stereo radiographs into the pair of stereo images.

35. The system of claim 32, wherein the obtaining means comprises an X-ray imaging device for providing the pair of stereo images.

36. The system of claim 32, further comprising:

coupled to the obtaining means, processing means for processing data representing the stereo images;

coupled to the processor, a storage device;

coupled to the processor, a computer readable medium; and

coupled to the processor, a display device for displaying the composite image.

37. The system of claim 32, wherein the means for correcting illumination errors comprises:

means for selecting a first group of pixels from the first image and for selecting a second group of pixels from the second image, the second group of pixels being associated with the first group of pixels;

coupled to the means for selecting the first group and second group of pixels, means for determining an intensity level for each pixel in both groups of pixels;

coupled to the means for determining an intensity level, means for determining a mean intensity level and a variance for each of the groups of pixels; and

coupled to the means for determining a mean intensity level and a variance, altering means for altering the intensity level of the pixels of the first group to approximate the intensity level of the pixels in the second group.

38. The system of claim 32, wherein the means for eliminating keystone distortion within the pair of stereo images comprises:

means for calculating a shift amount for eliminating the keystone distortion; and

coupled to the means for calculating a shift amount, means for moving the pixels of the first and second images towards each other by the calculated shift amount.

39. The system of claim 32, wherein the means for eliminating keystone distortion within the pair of stereo images comprises:

means for determining a location of at least one physical pointer around the object, wherein the physical pointer is captured in the pair of stereo radiographs;

coupled to the means for determining a location, estimating means for estimating epipolar geometry and horizontal and vertical distortions using the location;

coupled to the estimating means, adjusting means for adjusting at least one of the pair of stereo images vertically and horizontally to correct for any estimated distortions; and

coupled to the adjusting means, means for calculating a location of the item.

40. A computer readable medium containing a computer program product for stereo image processing of an object, the computer program product including instructions for directing a computer to execute operations comprising the steps of:

correcting errors resulting from at least one differently illuminated pair of stereo images of the object, said stereo images including a plurality of pixels and corresponding to a pair of stereo radiographs of the object;

removing depth plane curvature from the pair of stereo images by determining a representation of the pair of stereo images, the representation having a parallel geometry with respect to one or more X-ray sources used to form the radiographs, and determining pixel values for the representation based on the plurality of pixels in the pair of stereo images;

removing keystone distortions from the pair of stereo images; and

adjusting a corresponding screen parallax for a composite image, the composite image being a combination of the pair of stereo images.

41. The computer readable medium of claim 40, wherein the operations further comprise the steps of:

displaying the composite image on a display device, the display device including a viewing surface; and

locating the object appearing in the composite image near the viewing surface in order to minimize depth range.