Nonlinear blind demixing of single pixel underlying radiation sources and digital spectrum local thermometer

• US 20040181375A1
• Filed: 08/22/2003
• Published: 09/16/2004
• Est. Priority Date: 08/23/2002
• Status: Active Grant

• Current Assignee: The United States of America As Represented By The Secretary of Agriculture
• Original Assignee: The United States of America As Represented By The Secretary of Agriculture
• Inventors: Szu, Harold, Kopriva, Ivica, Buss, James R.
• Granted Patent: US 7,366,564 B2
• Time in Patent Office: Days
• US Class Current: 703/2
• CPC Class Codes: A61B 5/0084 for introduction into the body, e.g. by catheters A61B5/1459 takes precedence
• CPC Class Codes: A61B 5/0086 using infrared radiation
• CPC Class Codes: A61B 5/0091 for mammography
• CPC Class Codes: A61B 5/015 By temperature mapping of body part
• CPC Class Codes: A61B 5/7267 involving training the classification device
• CPC Class Codes: G06F 2218/22 Source localisation; Inverse modelling
• CPC Class Codes: G06T 7/0012 Biomedical image inspection
• CPC Class Codes: G16H 50/70 for mining of medical data, e.g. analysing previous cases of other patients

Abstract

Changes, increase or decrease, in the body fluid flow are passively detected by using a single pixel, non-linear blind de-mixing procedure, which can be extended to general biomedical measurement and diagnoses instruments. More specifically the single pixel, non-linear blind de-mixing procedure is applied on the hot spots of rheumatic arthritis or breast cancer detection problem using passive two-color infrared imaging as well as to passively detect blockages in the body fluid circulatory system that might be of importance for coronary artery bypass surgery, diabetes and deep vein thrombosis. Other applications of the mentioned algorithm include a pair of cameras for video, a pair of antennas for cell phones, and in situ data gathering or imaging using multiple mode fiberoptical sensing as well as selective amplification hearing aids through two-ear binaural processing for de-noise echo cancellation and signal classification.

Claims

1. A method of determining uniquely underlying sources forming a source vector S=(S₁, S₂, . . . ) propagating through a nonlinear mixing medium of a constant temperature, open equilibrium system by measuring multiple radiation components forming a data vector per single pixel X=(X₁, X₂, . . . ), comprising:

measuring the data vector X;

applying a constraint to the equilibrium system such that the thermal diffusion of the equilibrium system is constrained isothermally at the equilibrium free energy, wherein the equilibrium free energy is the Helmholtz free energy H=E−

TS, wherein E is the energy, T is the equilibrium reservoir temperature, and S is the classical Shannon information theory entropy;

defining a state of the open equilibrium system by a feed-forward first order error energy E(X/S)=μ

{g([W]X)−

S), wherein g is the Lagrange constraint vector and [W] is the feed-foward matrix;

reducing the feed-forward first order energy E(X/S) to a second order Least Mean Square (LMS) error energy for a specific Lagrange constraint vector μ

; and

determining from among all possible vector sources S=(S₁, S₂, . . . ) the one vector source that satisfies the minimum H for an arbitrary mixing matrix [A] and smooth nonlinearity g;

X=g<sup>−

1</sup>{[A]S}, wherein [A] is the heat transport mixing matrix and is the inverse of [W].

2. A method of a physical demixing system for early malign tumor detection including taking two pictures of a source using two spectral cameras non-intrusively, each pair of an input pixel data vector having two components, a long wavelength and a middle wavelength, the method comprising:

defining, by an unsupervised learning artificial neural network, a pair per pixel as a vector X(pixel, t)=(X_long(pixel, t), X_short(pixel, t)), where t indicates a repetition time that can reveal the growth of a malign tumor which is mixed with a benign tumor in terms of an unknown percentage source vector S(pixel, t)=(S_benign(pixel, t), S_malign(pixel, t)); and

determining, by the unsupervised learning artificial neural network, a quantitative value of the malign tumor source s_malign/(s_malign+S_benign) by imposing generalized information theory at a local equilibrium in terms of the Helmholtz free energy of the system.

3. An apparatus determining uniquely underlying sources forming a source vector S=(S₁, S₂, . . . ) propagating through a nonlinear mixing medium of a constant temperature, open equilibrium system by measuring multiple radiation components forming a data vector per single pixel X=(X₁, X₂, . . . ), comprising:

a processor measuring the data vector X, applying a constraint to the equilibrium system such that the thermal diffusion of the equilibrium system is constrained isothermally at the equilibrium free energy, wherein the equilibrium free energy is the Helmholtz free energy H=E−

TS, wherein E is the energy, T is the equilibrium reservoir temperature, and S is the classical Shannon information theory entropy, defining a state of the open equilibrium system by a feed-forward first order error energy E(X/S)=μ

{g([W]X)−

S}, wherein i is the Lagrange constraint vector and [W] is the feedfoward matrix, reducing the feed-forward first order energy E(X/S) to a second order Least Mean Square (LMS) error energy for a specific Lagrange constraint vector A, and determining from among all possible vector sources S=(S₁, S₂, . . . ) the one vector source that satisfies the minimum H for an arbitrary mixing matrix [A] and smooth nonlinearity g;

X=g<sup>−

1</sup>{[A]S}, wherein [A] is the heat transport mixing matrix and is the inverse of [W].

4. A computer-readable medium storing a program determining uniquely underlying sources forming a source vector S=(S₁, S₂, . . . ) propagating through a nonlinear mixing medium of a constant temperature, open equilibrium system by measuring multiple radiation components forming a data vector per single pixel X=(X₁, X₂, . . . ), the program when executed by the computer causes the computer to execute the functions comprising:

measuring the data vector X;

applying a constraint to the equilibrium system such that the thermal diffusion of the equilibrium system is constrained isothermally at the equilibrium free energy, wherein the equilibrium free energy is the Helmholtz free energy H=E−

TS, wherein E is the energy, T is the equilibrium reservoir temperature, and S is the classical Shannon information theory entropy;

defining a state of the open equilibrium system by a feed-forward first order error energy E(X/S)=μ

{g([W]X)−

S}, wherein μ

is the Lagrange constraint vector and [W] is the feedfoward matrix;

reducing the feed-forward first order energy E(X/S) to a second order Least Mean Square (LMS) error energy for a specific Lagrange constraint vector g; and

determining from among all possible vector sources S=(S₁, S₂, . . . ) the one vector source that satisfies the minimum H for an arbitrary mixing matrix [A] and smooth nonlinearity g;

X=g<sup>−

1</sup>{[A]S}, wherein [A] is the heat transport mixing matrix and is the inverse of [W].

5. The apparatus of claim 4, wherein the processor comprises a digital signal processor.

6. The apparatus of claim 4, wherein the processor comprises field programmable gate arrays.

7. The apparatus of claim 4, wherein the processor comprises mixed analog and digital VLSI circuits.

8. The apparatus of claim 4, wherein the apparatus comprising a digital heat spectrum local thermometer.

9. A two-camera single optical axis based infrared (IR) imaging system determining uniquely underlying sources forming a source vector S=(S₁, S₂, . . . ) propagating through a nonlinear mixing medium of a constant temperature, open equilibrium system by measuring multiple radiation components forming a data vector per single pixel X=(X₁, X₂, . . . ), comprising:

a multispectral IR camera comprising;

a single optical axis receiving a beam of the data vector X including data corresponding to the source;

a beam splitter splitting the beam into a long IR wavelength beam and a mid IR wavelength beam;

a long IR CCD receiving the long IR wavelength beam, converting the IR wavelength beam into a first image and outputting the first image; and

a mid IR CCD receiving the mid IR wavelength beam, converting the IR wavelength beam into a second image, and outputting the second image; and

a computer receiving the first image and the second image and measuring the data vector X, applying a constraint to the equilibrium system such that the thermal diffusion of the equilibrium system is constrained isothermally at the equilibrium free energy, wherein the equilibrium free energy is the Helmholtz free energy H=E−

TS, wherein E is the energy, T is the equilibrium reservoir temperature, and S is the classical Shannon information theory entropy, defining a state of the open equilibrium system by a feed-forward first order error energy E(X/S)={g([W]X)−

S}, wherein g is the Lagrange constraint vector and [W] is the feedfoward matrix, reducing the feed-forward first order energy E(X/S) to a second order Least Mean Square (LMS) error energy for a specific Lagrange constraint vector A, and determining from among all possible vector sources S=(S₁, S₂, . . . ) the one vector source that satisfies the minimum H for an arbitrary mixing matrix [A] and smooth nonlinearity g;

X=g<sup>−

1</sup>{[A]S}, wherein [A] is the heat transport mixing matrix and is the inverse of [W].

10. A method of early breast pre-cancer ductal carcinoma in situ tumor classification, diagnosis and tracking using a digital heat spectrum local thermometer, by determining uniquely underlying sources forming a source vector S=(S₁, S₂, . . . ) propagating through a nonlinear mixing medium of a constant temperature, open equilibrium system by measuring multiple radiation components forming a data vector per single pixel X=(X₁, X₂, . . . ) comprising:

receiving, by a multispectral infrared (IR) camera of the digital heat spectrum local thermometer, a beam of the data vector X of spectral data including data corresponding to a heat source, splitting the beam into a mid IR beam and a long IR beam, converting by a first charge-coupled device the long IR beam into a first image, and converting by a second charge-coupled device the mid IR beam into a second image;

receiving, by a computer of the digital heat spectrum local thermometer, the first image and the second image,

cooling the first charge-coupled device and the second charge-coupled device;

receiving, by the multispectral infrared (IR) camera, a second beam of data vector of spectral data including data corresponding to the heat source, splitting the second beam into a second mid IR beam and a second long IR beam, converting by the first charge-coupled device the second long IR beam into a third image, and converting by the second charge-coupled device the mid IR beam into a fourth image; and

applying, by the computer, a constraint to the equilibrium system such that the thermal diffusion of the equilibrium system is constrained isothermally at the equilibrium free energy, wherein the equilibrium free energy is the Helmholtz free energy H=E−

TS, wherein E is the energy, T is the equilibrium reservoir temperature, and S is the classical Shannon information theory entropy, defining a state of the open equilibrium system by a feed-forward first order error energy E(X/S)=μ

{g([W]X)−

S}, wherein p is the Lagrange constraint vector and [W] is the feedfoward matrix, reducing the feed-forward first order energy E(X/S) to a second order Least Mean Square (LMS) error energy for a specific Lagrange constraint vector μ

, and determining from among all possible vector sources S=(S₁, S₂, . . . ) the one vector source that satisfies the minimum H for an arbitrary mixing matrix [A] and smooth nonlinearity g;

X=g<sup>−

1</sup>{[A]S}, wherein [A] is the heat transport mixing matrix and is the inverse of [W].

11. A method of tracking infra-red imaging history or histology by determining uniquely underlying sources forming a source vector S=(S₁, S₂,. . . ) propagating through a nonlinear mixing medium of a constant temperature, open equilibrium system by measuring multiple radiation components forming a data vector per single pixel X=(X₁, X₂, . . . ), comprising:

measuring the data vector X;

applying a constraint to the equilibrium system such that the thermal diffusion of the equilibrium system is constrained isothermally at the equilibrium free energy, wherein the equilibrium free energy is the Helmholtz free energy H=E−

TS, wherein E is the energy, T is the equilibrium reservoir temperature, and S is the classical Shannon information theory entropy;

defining a state of the open equilibrium system by a feed-forward first order error energy E(X/S)=μ

{g([W]X)−

S}, wherein μ

is the Lagrange constraint vector and [W] is the feedfoward matrix;

reducing the feed-forward first order energy E(X/S) to a second order Least Mean Square (LMS) error energy for a specific Lagrange constraint vector μ

;

determining from among all possible vector sources S=(S₁, S₂, . . . ) the one vector source that satisfies the minimum H for an arbitrary mixing matrix [A] and smooth nonlinearity g;

X =g<sup>−

1</sup>{[A]S}, wherein [A] is the heat transport mixing matrix and is the inverse of [W]; and

tracking the one vector source.

12. A two-camera single optical axis based infrared (IR) imaging system comprising:

a multispectral IR camera comprising;

a single optical axis receiving a beam of data vector of spectral data including data corresponding to a heat source;

a beam splitter splitting the beam into a long IR wavelength beam and a mid IR wavelength beam;

a long IR CCD receiving the long IR wavelength beam, converting the IR wavelength beam into a first image and outputting the first image; and

a mid IR CCD receiving the mid IR wavelength beam, converting the IR wavelength beam into a second image, and outputting the second image;

a computer receiving the first image and the second image and a beam of data vector of spectral data including data corresponding to the heat source, and executing a single pixel, non-linear blind de-mixing procedure as a non linear Lagrange Constraint Neural Network unsupervised two-spectral IR breast image classification on the first image and the second image to determine the heat source.

13. A method of early breast pre-cancer ductal carcinoma in situ tumor classification, diagnosis and tracking comprising:

receiving, by a multispectral infrared (IR) camera on a single optical axis, a beam of data vector of spectral data including data corresponding to a heat source, splitting the beam into a mid IR beam and a long IR beam, converting the long IR beam into a first image, and converting the mid IR beam into a second image; and

receiving, by a computer, the first image and the second image, and executing a single pixel, non-linear blind de-mixing procedure as a nonlinear Lagrange Constraint Neural Network unsupervised two-spectral IR breast image classification on the first image and the second image to determine the heat source.

14. A method of early breast pre-cancer ductal carcinoma in situ tumor classification, diagnosis and tracking using a digital heat spectrum local thermometer comprising:

receiving, by a multispectral infrared (IR) camera of the digital heat spectrum local thermometer, a beam of data vector of spectral data including data corresponding to a heat source, splitting the beam into a mid IR beam and a long IR beam, converting by a first charge-coupled device the long IR beam into a first image, and converting by a second charge-coupled device the mid IR beam into a second image;

receiving, by a computer of the digital heat spectrum local thermometer, the first image and the second image,

cooling the first charge-coupled device and the second charge-coupled device;

receiving, by the multispectral infrared (IR) camera, a second beam of data vector of spectral data including data corresponding to the heat source, splitting the second beam into a second mid IR beam and a second long IR beam, converting by the first charge-coupled device the second long IR beam into a third image, and converting by the second charge-coupled device the mid IR beam into a fourth image; and

executing, by the computer, a single pixel, non-linear blind de-mixing procedure as a nonlinear Lagrange Constraint Neural Network unsupervised two-spectral IR breast image classification on the first image, the second image, the third image, and the fourth image to determine the heat source.

15. A method of tracking infrared imaging history or histology, comprising:

receiving, by a multispectral infrared (IR) camera on a single optical axis, a beam of data vector of spectral data including data corresponding to a heat source, splitting the beam into a mid IR beam and a long IR beam, converting the long IR beam into a first image, and converting the mid IR beam into a second image; and

receiving, by a computer, the first image and the second image, and executing a single pixel, non-linear blind de-mixing procedure as a nonlinear Lagrange Constraint Neural Network unsupervised two-spectral IR breast image classification on the first image and the second image to determine the heat source.

16. A method of in situ data gathering by a multiple mode fiberoptic recording, comprising:

receiving, by an objective lens, a beam of multiple wavelengths and including a data vector of spectral data including data corresponding to a heat source of a tissue image;

transmitting, by the objective lens, the beam to a dual-mode fiber relay;

receiving, by an ocular lens, the beam passing through the dual-mode fiber relay;

focusing, by the ocular lens, the beam on a scanning mirror;

reflecting, by the scanning mirror, a first wavelength of the beam onto a first detector array, and a second wavelength of the beam onto a second detector array;

de-mixing a first tissue image and a second tissue image by a computer executing a single pixel, non-linear blind de-mixing procedure as a nonlinear Lagrange Constraint Neural Network unsupervised two-spectral image classification procedure, and

de-mixing a tissues image corresponding with virtual tumor artificially introduced by molecular tagging of fluorescence proteins.

17. A method of passive detection of blockages in the body fluid circulatory system comprising:

receiving, by a multispectral infrared (IR) camera on a single optical axis, a beam of data vector of spectral data including data corresponding to a heat source, splitting the beam into a mid IR beam and a long IR beam, converting the long IR beam into a first image, and converting the mid IR beam into a second image; and

receiving, by a computer, the first image and the second image, and executing a single pixel, non-linear blind de-mixing procedure as a nonlinear Lagrange Constraint Neural Network unsupervised two-spectral IR image classification on the first image and the second image to determine the heat source.

18. The method of claim 17, wherein the detected blockages are in the coronary arteries.

19. The method of claim 17, wherein the detected blockages are deep vein thrombosis.

20. The method of claim 17, wherein the detected blockages relate to diabetes.

21. A method of passive detection and diagnosis of rheumatic arthritis comprising:

receiving, by a multispectral infrared (IR) camera on a single optical axis, a beam of data vector of spectral data including data corresponding to a heat source, splitting the beam into a mid IR beam and a long IR beam, converting the long IR beam into a first image, and converting the mid IR beam into a second image; and

receiving, by a computer, the first image and the second image, and executing a single pixel, non-linear blind de-mixing procedure as a nonlinear Lagrange Constraint Neural Network unsupervised two-spectral IR image classification on the first image and the second image to determine the heat source.

22. An apparatus including selective amplification hearing aids selectively amplifying a received signal through de-noise echo cancellation and signal classification simultaneously by two-ear binaural processing, comprising:

a two-microphone sensory system;

a processor executing a single pixel, non-linear blind de-mixing procedure as a nonlinear unsupervised feedforward Lagrange Constraint Neural Network signal processing procedure;

a hand-held device comprising a digital signal processor executing a single pixel, non-linear blind de-mixing procedure as a nonlinear feedforward Lagrange Constraint Neural Network extracting a clear signal and noise from the received signal and wirelessly transmitting the clear signal; and

a selective hearing amplifier wirelessly receiving the transmitted clear signal.