System and method for using three dimensional infrared imaging to provide detailed anatomical structure maps

• US 8,463,006 B2
• Filed: 04/15/2008
• Issued: 06/11/2013
• Est. Priority Date: 04/17/2007
• Status: Expired due to Fees

• Current Assignee: Francine J. Prokoski
• Original Assignee: Francine J. Prokoski
• Inventors: Prokoski, Francine J.
• Primary Examiner(s): RANGREJ, SHEETAL
• Application Number:
  US12/596,629
• Publication Number:
  US 20100172567A1
• Time in Patent Office: 1,883 Days
• Field of Search: 382/128
• US Class Current: 382/128
• CPC Class Codes: A61B 5/0064 Body surface scanning
• CPC Class Codes: A61B 5/0086 using infrared radiation
• CPC Class Codes: A61B 5/015 By temperature mapping of body part
• CPC Class Codes: A61B 5/1077 Measuring of profiles
• CPC Class Codes: A61B 5/411 Detecting or monitoring allergy or intolerance reactions to an allergenic agent or substance
• CPC Class Codes: A61B 5/415 the glands, e.g. tonsils, adenoids or thymus
• CPC Class Codes: A61B 5/418 lymph vessels, ducts or nodes
• CPC Class Codes: A61B 5/445 Evaluating skin irritation or skin trauma, e.g. rash, eczema, wound, bed sore
• CPC Class Codes: A61B 5/4519 Muscles A61B5/389, A61B5/224 take precedence
• CPC Class Codes: A61B 5/7264 Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems neural networks per se G06N3/00; expert systems per se G06N5/00
• CPC Class Codes: G01K 13/20 Clinical contact thermometers for use with humans or animals
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G16H 30/20 for handling medical images, e.g. DICOM, HL7 or PACS
• CPC Class Codes: G16H 30/40 for processing medical images, e.g. editing
• CPC Class Codes: Y02A 90/10 Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Abstract

Calibrated infrared and range imaging sensors are used to produce a true-metric three-dimensional (3D) surface model of any body region within the fields of view of both sensors. Curvilinear surface features in both modalities are caused by internal and external anatomical elements. They are extracted to form 3D Feature Maps that are projected onto the skin surface. Skeletonized Feature Maps define subpixel intersections that serve as anatomical landmarks to aggregate multiple images for models of larger regions of the body, and to transform images into precise standard poses. Features are classified by origin, location, and characteristics to produce annotations that are recorded with the images and feature maps in reference image libraries. The system provides an enabling technology for searchable medical image libraries.

Claims

1. A computer implemented method for generating an accurate model in three dimensions (3D) of internal and external and external anatomical features, comprising:

collecting simultaneous images of a body segment using a plurality of imaging devices;

calibrating the model by using a computer processor to process the simultaneous images from the plurality of imaging devices,

wherein the plurality of imaging devices comprises an infrared imager, a range imager, and a visual imager, and the calibration step further comprises;

performing intrinsic calibration of each imager to compute its intrinsic correction transform for lens aberrations, radial distortion, detector array nonregularities, and other biases in its output image;

performing extrinsic calibrations of a selected two of the imagers, the selected imagers each having a lower (x,y,) sample density than any non-selected imager, said extrinsic calibration obtaining an extrinsic overlay transform for each respective selected imager;

combining corrected and overlaid images of the infrared camera and the range imager to produce a 3D infrared model with a body-centric coordinate system;

processing the range image to extract a curvilinear feature map of external anatomy;

processing the infrared image to extract a curvilinear feature map of internal anatomy;

skeletonizing the respective curvilinear feature maps;

producing skeleton node maps containing intersection and branch locations of said curvilinear features;

labeling each node according to a standard directory description of intersecting or branching anatomical features;

forming a layered composite image of the infrared, range, and visual images, plus their feature maps, plus their skeletonized feature maps, plus their node maps;

selecting nodes corresponding to three reference points designated for said body segment;

rotating the composite image in three-dimensional space such that the three reference points define a two dimensional (2D) image plane, said 2D image plane being a standard pose for said body segment;

storing said rotated standardized composite image as a labeled facet of a total 3D body map;

repeating said further calibration steps for additional body segments.

2. A method as in claim 1, where a second infrared imager is substituted for the visual imager.

3. A method as in claim 1, where imagers of higher spatial resolution are substituted and used to refine said body map by obtaining images over smaller body segments for at least a portion of said body map, said portion being increased until a desired level of refinement has been reached, said refined body map segments being linked to their corresponding body map segments.

4. A method to produce a 2D body map by producing, for a given body segment, a composite image in standardized pose in accordance with claim 1, then for each pixel of a curvilinear feature assign grayscale or color value equal to the range value at that pixel, then disregard range values now encoded into grey or color presentation, thereby leaving a 2D body map as a residual.

5. A method as in claim 4, further comprising annotating visual, x-ray, or other 2D images with anatomical features by overlaying and merging said 2D images with the 2D body map using common features for alignment and registration.

6. A method as in claim 1, further comprising

generating sequential body maps over time, where at each time interval body-centric body map coordinates are translated into an external coordinate system with reference points outside the body;

tracking movement of the body relative to the external coordinate system;

predicting body map coordinates in the external coordinate system at the next time interval;

adjusting body map coordinates to the predicted values; and

projecting the adjusted body map onto a body surface.

7. A method as in claim 6, further comprising synchronizing imaging, processing and projection cycles to respiration of the human subject.

8. A method as in claim 6, further comprising synchronizing imaging, processing and projection cycles to the cardiac pulse cycles of the human subject.

9. A method as in claim 1, further comprising projecting the body map onto a surface or model of a human body.

10. A method as in claim 2, further comprising projecting the body map onto a surface or model of a human body.