Mosaicing and enhancement of images for ophthalmic diagnosis and documentation

• US 6,454,410 B1
• Filed: 10/20/2000
• Issued: 09/24/2002
• Est. Priority Date: 10/20/1999
• Status: Expired due to Term

• Current Assignee: Sarnoff Corporation (SRI International, Inc.), Trustees Of The University Of Pennsylvania (University of Pennsylvania)
• Original Assignee: Sarnoff Corporation (SRI International, Inc.), Trustees Of The University Of Pennsylvania (University of Pennsylvania)
• Inventors: Hsu, Steve, Sajda, Paul, Berger, Jeffrey W., Asmuth, Jane
• Primary Examiner(s): MANUEL, GEORGE C
• Application Number:
  US09/692,939
• Time in Patent Office: 704 Days
• Field of Search: 351/206, 351/209, 351/210, 351/211, 351/246, 382/117
• US Class Current: 351/206
• CPC Class Codes: A61B 3/0041 characterised by display arrangements
• CPC Class Codes: A61B 3/0058 for multiple images
• CPC Class Codes: A61B 3/12 for looking at the eye fundus, e.g. ophthalmoscopes A61B3/13 takes precedence
• CPC Class Codes: A61B 3/135 Slit-lamp microscopes
• CPC Class Codes: A61B 3/145 by video means
• CPC Class Codes: G06T 2207/20016 Hierarchical, coarse-to-fine, multiscale or multiresolution image processing; Pyramid transform
• CPC Class Codes: G06T 2207/30041 Eye; Retina; Ophthalmic
• CPC Class Codes: G06T 3/14 Transformations for image registration, e.g. adjusting or mapping for alignment of images
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/30 Determination of transform parameters for the alignment of images, i.e. image registration

Abstract

The invention is a method and system for mosaicing images of the eye to create high resolution, wide-field ophthalmic images for the diagnosis and treatment of eye diseases. The inventive method acquires a first image of the eye, acquires a second image of the eye, and processes the images to produce a mosaic representation. The second image includes a portion of the first image. To guide in acquiring the second image, the method may view the first image while acquiring the second image. The images typically are acquired using either a direct ophthalmoscope or a slitlamp biomicroscope. The method further may convert the images to a digital format. The step of processing includes aligning and merging the images, and conducting real-time processing and non-real-time processing. Real-time processing may include eliminating non-overlapping images, image registration, topology inference, local-to-global alignment, image fusion, signal estimation, and multiresolution blending, while the non-real-time processing may include local alignment, global alignment, image registration, intra-alignment, inter-alignment, signal averaging, and photometric blending.

Claims

1. A method of mosaicing images of the eye, comprising the steps of:

acquiring a first image of the eye from a narrow field of view image acquisition device;

acquiring a second image of the eye from said image acquisition device, wherein said second image includes a portion of said first image; and

processing said images to produce a mosaic representation.

2. The method of claim 1, further comprising the step of providing a direct ophthalmoscope to conduct said steps of acquiring.

3. The method of claim 1, further comprising the step of providing a slitlamp biomicroscope to conduct said steps of acquiring.

4. The method of claim 1, wherein said processing step comprises the step of converting said images to a digital format.

5. The method of claim 1, further comprising the step of viewing said first image as a guide during the step of acquiring said second image.

6. The method of claim 1, wherein said processing step comprises the steps of aligning and merging said first and second images.

7. The method of claim 1, wherein said processing step comprises real-time processing and non-real-time processing steps.

8. The method of claim 7, wherein said real-time processing step includes at least one of the following steps:

eliminating non-overlapping images, image registration, topology inference, local-to-global alignment, image fusion, signal estimation, and multiresolution blending.

9. The method of claim 7, wherein said non-real-time processing step includes at least one of the following steps:

local alignment, global alignment, image registration, intra-alignment, inter-alignment, signal averaging, and photometric blending.

10. The method of claim 1, wherein said acquiring steps comprise receiving said images from said image acquisition device to a processor.

11. The method of claim 1, further comprising the step of receiving a selection of the first image prior to acquiring the first image.

12. The method of claim 11, further comprising the step of automatically acquiring subsequent images from said image acquisition device after receiving the selection of the first image.

13. The method of claim 12, further comprising the step of creating a working region from said acquired images.

14. The method of claim 13, further comprising the step of oversampling said working region to enhance the quality of said working region.

15. A system for mosaicing images of the eye, comprising:

a narrow field of view image acquisition device adapted to provide first and second adjustment images of the eye;

a camera coupled to said image acquisition device to capture said images;

a data processor coupled to said camera to process said images into a mosaic representation; and

a monitor coupled to said data processor to guide a user in acquiring images using said image acquisition device.

16. The system of claim 15, further comprising a data storage device coupled to said data processor for storing said images.

17. The system of claim 16, wherein said selection unit includes at least one of the following:

a keyboard, a mouse, and a microphone.

18. The system of claim 15, further comprising a selection unit coupled to said data processor.

19. The system of claim 15, further comprising an analog-to-digital converter coupled to said image acquisition device and said data storage device.

20. The system of claim 15, wherein said image acquisition device is a direct opthalmoscope.

21. The system of claim 15, wherein said image acquisition device is a slitlamp biomicroscope.

22. The system of claim 15, wherein said processor processes said images in real-time using software adapted to do at least one of the following steps:

eliminate non-overlapping images, provide image registration, provide topology inference, provide local-to-global alignment, provide image fusion, provide signal estimation, and provide multiresolution blending.

23. The system of claim 15, wherein said processor processes said images in non-real-time using software adapted to do at least one of the following:

provide local alignment, provide global alignment, provide image registration, provide intra-alignment, provide inter-alignment, provide signal averaging, and provide photometric blending.

24. A method of detecting eye diseases, comprising the steps of:

examining the eye using a narrow field of view image acquisition device;

capturing images of the eye, wherein each of said images includes a portion of another image;

aligning and merging said images to create a mosaic representation;

viewing said mosaic representation; and

detecting eye disease from said mosaic representation.

25. The method of claim 24, wherein said step of examining is conducted using a direct ophthalmoscope.

26. The method of claim 24, wherein said step of examining is conducted using a slitlamp biomicroscope.

27. The method of claim 24 wherein said steps of capturing images, and aligning and merging said images occur during said step of examining the eye.

28. The method of claim 24 wherein said step of capturing images comprises receiving said images from said image acquisition device to a processor.

29. A method of creating a mosaic representation of the eye, comprising the steps of:

providing a narrow field of view image acquisition device adapted to acquire images of the eye;

acquiring a first image of a first portion of the eye;

moving said image acquisition device to a second portion of the eye, wherein said second portion overlaps said first portion;

acquiring a second image of said second portion of the eye; and

processing said images to create a mosaic representation of the eye.

30. The method of claim 29, wherein said steps of acquiring comprise the step of automatically capturing images from a stream of images.

31. The method of claim 29, wherein said step of providing an image acquisition device comprises the step of providing a direct ophthalmoscope.

32. The method of claim 29, wherein said step of providing an image acquisition device comprises the step of providing a slitlamp biomicroscope.

33. The method of claim 29, further comprising viewing said first image while conducting said step of moving.

34. The method of claim 29, wherein said step of processing comprises merging and aligning said first and second images.

35. A method for mosaicing direct ophthalmoscope images, comprising the steps of:

acquiring a first direct ophthalmoscope image;

converting said first image into a digital format;

acquiring at least one subsequent direct ophthalmoscope image, wherein said subsequent images reproduce a portion of a preceding image;

converting said images into a digital format;

processing said images during said steps of acquiring said images to guide acquisition of said subsequent images;

processing said images to increase resolution and quality of said images;

merging said images to produce a mosaic representation of said images; and

processing said mosaic representation such that said mosaic representation provides a seamless representation of said images.

36. A method for mosaicing slitlamp images, comprising the steps of:

acquiring a first slitlamp image;

converting said first image into a digital format;

acquiring subsequent slitlamp images, wherein said subsequent images reproduce a portion of a preceding image;

converting said images into a digital format;

processing said images during said steps of acquiring said images to guide acquisition of additional images;

processing said images to increase resolution and quality of said images;

merging said images to produce;

a mosaic representation of said images; and

processing said mosaic representation such that said mosaic provides a seamless representation of said images.