Hardware implemented pixel level digital filter and processing of electromagnetic signals

• US 20100245634A1
• Filed: 02/01/2010
• Published: 09/30/2010
• Est. Priority Date: 03/26/2009
• Status: Active Grant

• Current Assignee: Ned M. Ahdoot
• Original Assignee: Ned M. Ahdoot
• Inventors: Ahdoot, Ned M.
• Granted Patent: US 8,159,568 B2
• Time in Patent Office: Days
• US Class Current: 348/250
• CPC Class Codes: H04N 2209/045 using mosaic colour filter
• CPC Class Codes: H04N 23/84 for processing colour signals
• CPC Class Codes: H04N 25/60 Noise processing, e.g. detecting, correcting, reducing or removing noise
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

An apparatus for performing digital video processing of electromagnetic waves in IR or other electromagnetic waves spectra; derived from adjacent or separate bands (or sub-bands) representing a channel of electromagnetic wave intensities; that is processed for equipment radiometric corrections (calibration), adverse environmental elements or substance calibrations, channel filtering in real time and dynamic pixel to pixel time.

Claims

1. An apparatus for performing digital video processing of electromagnetic waves in IR spectra;

a sequence of raster input pixel signals, each respective raster input pixel having, an n-tuple set of values of electromagnetic waves intensities from a video camera or sensors to signify n-tuple intensity identification of a pixel, the n-tuple (hereafter called channels) set is derived from adjacent or separate bands (or sub-bands) representing a channel of electromagnetic wave intensities;

that is processed for equipment radiometric corrections (calibration), adverse environmental (element or substance) calibrations, and detection;

wherein data intensities representing a set of addresses input to plurality of “

n”

memories;

each memory address corresponding to a set of pre-loaded memory data to represent equipment radiometric corrected data and calibrated data due to adverse environment conditions.

2. The apparatus of claim 1 wherein system related imperfections such as variations in FPA temperature, optical imperfections and individual pixel imperfections affecting sensor data are corrected by altering the received sensor data with a pre-recorded or empirically derived data;

corrections to be made with a cascading memories in which each stage in the cascading (series) of stages corrects data due to a source of the imperfection as shown in FIGS. 2, 2B, and 2C as follows;

a) a controller for sequencing activity of each stage and to direct the output from any precious stage to the next stage;

radiometric pixel by pixel corrections for the imperfections of equipment are corrected by setting the CCD frame address such as rows and columns to plurality of Radiometric Correction Memory address generator, wherein the data contents are set to address to the Radiometric Data Correction (substitution), in which the data correction for each pixel is already pre-loaded;

Memory identify each pixel and data corrections;

b) inhibit (prevent) corrections along any one of the cascading stages;

c) the sequence timing to be done with a fastest system clock synchronized to pixel timing or sequencing without timing in which the final (data) settlement of each stage and all the stages, is valid upon electronic propagation settlement of stages;

d) controller having individual control lines for each stage to transfer data from any previous stage to this stage;

e) initial stages are allocated to the system imperfections affecting all pixels, and secondary stages will be pixel location related;

f) each stage consisting of a correction memory in which data contents is the correction a pixel sensor and the address to the memory is sensor data, previous stage sensor data, quantized source of imperfection for each stage or pixel coordinates;

each correcting memories data preloaded with a set of data to replace the affected sensor data according to FIGS. 2, 2B and 2C;

g) each stage is armed with correct no correct signal from the controller with additional signal to enable or disable outputs from registers or memories;

h) the controller initially receiving control information from the main CPU or dynamically receiving changes based upon the health of the system.

3. The apparatus of claims 1 and 2 further providing a group of Calibration Memories each having a relevant Calibration Function Generator receiving calibration sensor (data) in any spectral electromagnetic range to have a known emissions or abortions close to the target gas or solid substance;

each calibration memory comparing the received calibration data with a known calibration data;

each Calibration Function Generator providing a calibration factor in which regular channels are calibrated accordingly as follows;

calibrations are needed due to presence of different gaseous elements or solid substances primarily in the atmosphere ideally altering regular channels having the target intensity in different spectral field;

a) detecting the highest or lowest energy densities of an environmental element (such as cloud, fog, C02, rain, or sand storms), and to provide calibration corrections information for different channels of different elements;

b) Calibration Factors are amended to the regular data channels and set as an address to Calibrating Memories to adjust the received regular channel data, as depicted in FIG. 3B;

Calibration Factors are derived empirically or other calculation means for any different calculation equation linear exponential, and others;

Calibration Factors are the most significant bits and calibration data least significant bit;

c) Calibration Memories are preloaded with the data in which upon addressing of Calibration Factor and channel intensity, it provides the calibrated data FIG. 3B;

d) regular channels are calibrated for variety of atmospheric substances illumination, brightness, target temperature, rain, fog and distances;

e) a set of visible light filters, detectors and visible light Calibration Memories for detecting specific color energy densities of an element in the field of view to provide visible light calibration characterization for any one of the Calibration memories elements such as cloud, fog, C02, rain, or sand storms;

f) gray code (scale) in visible light field is used for illumination and brightness.

4. The apparatus of claim 1, 2, 3 and 4 for performing digital video target detection of a sequence of raster input pixel signals, each respective raster input pixel having, an ordered n-tuple set of values from a video camera or sensors to signify n-tuple sub-bands (channels) of IR electromagnetic wavelengths, the apparatus consisting of the following:

a) a digital video synchronization and timing that synchronizes to the header of the incoming digital video data bit stream and provides digital timing and control signal required to identify each one of the “

n”

tuple set of prime colors, and position of each prime color within a pixel timing;

b) digital filter memories, coupled to receive the intensities of the n-topple sequence of raster input sub-band signal intensities and map each respective n-topple intensity into a respective sub-band memory address each memory address having;

a predetermined memory address range,

a sub-band identity number of,

a group of sub bands spectra and prime colors belonging to an object within as;

a multi sub-band object'"'"'s group identity number;

identification of a sub-band of an object within a group of sub-bands belonging to an object, and within different groups of sub-bands;

preloaded with a single bit markers at predetermined addresses;

single bit markers addresses representing a band pass, low pass or high pass filters to be addressed by intensities intended to be detected as a band of filters (“

0”

not detect, and “

1”

for detect);

c) sub-band identification marks are input to an n-input AND gate to represent simultaneous detection of all the sub-bands within a pixel timing to declare detections of targets in diffident spectra.

5. The apparatus of claims 1 and 5 further providing statistical accumulations of detected pixels of each channel in a multi frame memory in which each channel is dedicated to one of the bits within memory data bit patterns to represent total number of detections of that channel within an arbitrary “

f”

number of frames and each memory address corresponds to the pixel coordinates in a frame as follows;

a) a set of registers, initialized by a frame header synchronization logic, that is maintained to represent the row and column coordinates of pixel address within a frame of video data that is used to address memories;

b) a set of dual registers A, and B to hold channels detect no detect signals from one pixel timing to the next, to be written into a memory;

c) a set of dual memory frames of A, and B representing the statistical detection occurrences of detected intensities of pixels in one frame;

d) a set of dual memory frames of A, and B representing the statistical detection occurrences of detected intensities of pixels in “

f”

number of frames;

e) a set of timing generators signifying different timing within a pixel data bit stream period to provide necessary timing to read the contents (partial) of the memory increment the data and write the incremented data back to the memory for “

f”

number of frames;

f) a set of FIFO'"'"'s to write the detect no detect of a multiform accumulations of each channel along with the pixel addresses for further processing.

6. The apparatus of claims 1, 4 and 5 further providing statistical detection of targets stored in a frame memory addressed by a frame coordinate;

each address pointing to a memory location in which each data bit in the memory data bit pattern is assigned to a specific target as follows;

a) a memory addressed by the detect no detect data bits (claim 4) and preloaded with data associating expected targets to be detected in plurality of channels, each target being assigned to a specific data bit in the memory data bit pattern on pixel by pixel level basis;

b) a set of dual memory frames of A, and B representing the statistical detection of intensities of targets in one frame;

c) a set of dual memory frames of A, and B representing the statistical detection of detected intensities of targets in “

f”

number of frames;

d) a set of timing generators signifying different timing within a pixel data bit stream period to provide necessary timing to read the contents (partial) of the memory increment the data and write the incremented data back to the memory for “

f”

number of frames;

e) a set of FIFO'"'"'s to write the detect no detect of targets in a multiform accumulations along with the frame coordinated in which a target was detected.

7. Provisions of claims 1, 2, 3 and 5 wherein concentration of individual gases within CWAs, mustards and TNT are measured in the relevant electromagnetic spectra (such as 9 to 13.5 um, and visible light spectra) using plurality of tunable filter memories as follows;

a) digital filter tunable memories, coupled to receive the intensities of the n-topple sequence of raster input sub-band signal intensities and map each respective n-topple intensity into a respective sub-band memory address each memory address having;

a predetermined memory address range,

a group of sub bands spectra belonging to certain gas within as;

a multi sub-band object'"'"'s group identity number;

identification of a sub-band of a gaseous element within a group of sub-bands belonging to a compound;

memories preloaded with a single bit markers at predetermined addresses;

single bit markers addresses representing a band pass, low pass or high pass filters to be addressed by intensities intended for detection within a sub band of filters (“

0″

not detect, and “

1”

for detect);

sub-band identification marks are input to an n-input AND gate to represent simultaneous detection of all the sub-bands within a pixel timing to declare detections of targets in diffident spectra;

b) a controller having interfaces to the tunable memories;

read the one bit detect no detect signal;

read the relevant intensities from the intensity registers, and compare received intensities with expected intensities;

adjust the band pass filtering of the memories, rearranging “

1'"'"'s” and

“

0'"'"'s”

having the pre stored absorbance coefficients at certain wavelength, for a specific substance (species) for measurements of concentrations;

having necessary capability to adjust the absorption confident for each substance and to provide calculations for measurements of concentrations of substances;

having necessary capability to provide algorithms and calculations to measure individual concentrations of compounds.

8. Provisions of claims 1 and 7 for remote concentration measurement of individual gases within a group of CWAs, mustards and TNT within electromagnetic spectra (such as 9 to 13.5 um, and visible light);

by further dividing the spectral band to much smaller sub-bands, each sub-band having its own absorption coefficients belonging to one of the target element within the group of elements;

using plurality of filters, sensors, dispersing elements, CCDs, amplifiers to subdivide the spectrum into sub-bands;

sub-bands spaced to achieve concentration measurements of compounds within the band;

a controller having calculating capability and absorption coefficients of each target compound for measurements of harmful chemicals even in the presence of other gases as follows;

a) a plurality of optical filters placed along the path of the electromagnetic signals;

each filter dividing the said band into groups of smaller bands;

b) a plurality of dispersing elements placed along the path of each group of smaller band spectra to further subdivide the small groups into smaller sub-bands;

c) a plurality of CCD substrates placed along the path of sub-bands to absorb photon energy and convert it to electricity;

d) a plurality of analog amplifiers;

each to amplify the voltages generated by CCD charges of each sub-band;

e) a plurality of multiplexers guiding and distributing sub-band analog voltages into the inputs of A/D converters that quantizes amplifier analog voltages to digital bit streams;

f) the CPU having absorption coefficient for each channel, and calculation processing capability to measure individual concentrations of each compound within the group of compounds.

9. Provisions of claims 1, 5, and 8 wherein each detected target (solid, liquid or gas within a pixel) is represented into a visible light video frame format, for each substance to represented by a pre-assigned color as follows:

a) along with the mark “

1”

bit, in the tunable filter memories, a set of data representing a color for each substance, each set of data consisting of three prime colors selected for a detected target;

for the combination of prime colors to be displayed on a monitor to represent a pixel for the choice color, to be viewed by an operator.

11. The apparatus of claim 1 for performing digital filtering and identification of targets in software for electromagnetic waves received from a video camera for IR and visible light source in a sequence of raster input pixel signals, each respective raster input having, an ordered n-tuple (channel) set of values from a camera or sensors representing n-tuple wavelength (prime colors or channels) identifying bands of electromagnetic waves, wherein a band of electromagnetic waves is any band within so called MWIR (or other similar name bands);

the process begins with (1) providing a bandwidth of the prime colors or channel of a target for recognition of a color in visible and recognition of target in IR within a, (2) storing the detected colors of different targets in a frame file (in the relevant x, and y coordinate) including the color number. (3) filtering and recognition of the remaining colors belonging to the same target within same pixel as follows;

(for convince the flow chart of FIG. 10 discusses visible light, in which the same applies for IR and the word “

prime color”

is substituted with the word channel);

a) wait for the frame start initialization signal from frame synchronization logic and go to the next step;

b) initialize “

n”

c) read the intensity of one of the prime colors such as red, green, or blue (or different channels of IR established during initialization) that appear in the video digital bit stream and proceed to the next step;

d) check if the intensities of three prime colors falls within the selected limits;

if it does transition to next step, otherwise transition to step e);

e) keep the target number, color number, pixel coordinates and increment “

n”

;

f) store the substituting prime colors (of a pixel) for this object in relevant coordinates as a background for visible light;

g) check for the end of prime colors (or channels), if it is not the end, transition back to block step c), otherwise transition to next step;

h) store color number and its associate targets in the same pixel coordinates in a frame;

store the same color number of a substitute color in the same coordinates;

increment coordinate and transition to next step;

i) check for the end of pixels in a row and column of a frame, if it is not the end of all pixels in a frame and transition to step b), otherwise transition to next stage of concurrent processing.

12. Provisions of claims 5, and 6 to provide a single file to identify objects based upon associated color identification performed in c) in a number of frames, the process includes the following steps:

a) set x, and y coordinates to zero and select;

b) at block 110, read the object'"'"'s number and number of colors associated with that object and increment coordinate number;

c) at decision block 115, check the number of colors detected during c) is larger or equal to the value “

r”

, if it is transition to block 120, otherwise transition to block 125;

d) At block 120 add the detected number of colors of the pixel and the object number to the new single frame, and transition back to block 110;

e) at block 125 look for the end of x, and y coordinates, if it not the end of one frame coordinates transition to bloc 110, otherwise transition to block 130;

f) at block 130 look for the next object supposed to have been detected in step c) and transition to block 135;

g) at block 135, check for the end of the objects set during initialization to be detected, if more objects go to block 100, otherwise transition to block 140;

h) at block 140, increment the frame number part of consecutive number of frames set during initialization, then transition to block 145;

i) at block 145, check for the end of frames, if it is not the end transition to block 100, otherwise exit.

13. Provisions of claim 12. wherein a single multi frame file with coordinates of a single file is generated to store colors with their associate targets in a single multi frame “

f”

, storing relating colors (Channels) to different targets, with the addition that if the number of colors detected for each color entry is less than a number “

a”

it is discarded as follows;

a) initialize the multi frame “

f”

coordinates;

b) initialize the possible color entries of a coordinate;

c) read the next number of color of the of the color entries;

d) check if the number of the color entries is less than a value “

b”

if it is less, discard the color entry and go back to block 210, otherwise go to e);

e) increment the color entry and transition to f);

f) check for the end of possible entries, if it is not go back to block 210, if it is go to g);

g) increment the coordinate number and go to block h);

h) check for the end of coordinates, if it is not go back to b), otherwise end this process.

10. An apparatus to generate memory address to a large memory system and memory banks in which the source of address generation is plurality of “

n”

registers, each register is set to different variable values of n(1), n(2), n(3), to n(x), wherein x denotes the length of each register, the apparatus consisting of the following;

a) plurality of “

n”

registers, each register receiving its data from different sources, of sensors or CPU;

b) a memory based address lookup table, coupled to the registers, to translate variable data into a fixed memory address;

the memory based lookup table, is either a PROM or a Random Access Memory, wherein the contents of the RAM is set by a register or CPU, to address lookup table generates memory bank address, and memory address;

the address lookup table provides a translation from all the various levels of strength (quotient), to an absolute memory bank address value, within a memory bank;

an address lookup table provides a translation from a set of pre assigned variable levels of strength and direct inputs from the computer address latch, to an absolute memory bank address value, for selecting a memory bank;

c) a set of memory banks addressed by the selected registers or memory address lookup table;

each bank consisting of a sub-bank memory selector, Data Memory Selector and instruction memories;

instruction memories preloaded with application instruction relevant to the register settings and feedback instruction registers;

Instructions fed to the CPU through a FIFO to execute specific instructions related to register settings;

each instruction containing executable instructions, a feedback (to the registers) data, and a number relating to the instruction bytes to be followed;

d) A CPU receiving information specified in the above through a FIFO for necessary executions with an option to load feedback instruction either before or after the execution of instructions.

e) a feedback generator