Position sensing detector focal plane array (PSD-FPA) event detection and classification system

• US 8,421,015 B1
• Filed: 09/15/2008
• Issued: 04/16/2013
• Est. Priority Date: 09/13/2007
• Status: Active Grant

• Current Assignee: Oceanit Laboratories, Inc.
• Original Assignee: Oceanit Laboratories, Inc.
• Inventors: Swanson, Robert E., Scott, Basil H., Wolfshagen, Randy, Eiler, Justin
• Primary Examiner(s): Porta, David
• Assistant Examiner(s): BOOSALIS, FANI POLYZOS
• Application Number:
  US12/283,708
• Time in Patent Office: 1,674 Days
• Field of Search: 250/338.4, 250/330, 250/332, 250/353, 356/5.01, 257/432
• US Class Current: 250/338.4
• CPC Class Codes: F41G 3/147 based on detection of a firing weapon
• CPC Class Codes: G01S 3/781 Details
• CPC Class Codes: G01S 3/783 using amplitude comparison of signals derived from static detectors or detector systems

Abstract

An event detection and classification system uses a new type of optical sensing component, a Position Sensing Detector Focal Plane Array (PSD-FPA). The PSD-FPA provides for high-speed operation that allows for accurate sensing of fast artifacts that are unique to weapons fire and enables precise location of optical phenomenon. The system detects and classifies events, particularly weapons fire, and rejects false alarms. An optical lens sub-system focuses light onto a PSD-FPA, which senses the photons and generates electrical signals associated with individual elements of the PSD-FPA. These signals are processed to identify and classify weapons-related or other events. Background subtraction, variable gain, time-intensity and time-location correlation, digital filtering, Fourier analysis, and wavelet analysis are all used to successfully classify the events while rejecting false alarms.

Claims

1. An optical event detection system, comprising

an optical sensor module and

an electronic control module electrically connected to the optical sensor module,

wherein the optical sensor module comprises an optical lens sub-system, wherein the optical lens sub-system has a lens, one or more optical sensors having focal plane arrays comprising one or more position sensing detector focal plane arrays (PSD-FPA), and electronics;

and wherein each of the one or more PSD-FPA include detector elements that are lateral effect diode elements, wherein each detector element is comprised of a sensing surface and four signal sensing connections.

2. The system of claim 1, wherein each of the one or more PSD-FPA further comprise a read-out integrated circuit (ROIC).

3. The system of claim 2, wherein the one or more optical sensors further comprises an imaging focal plane array.

4. The system of claim 2, wherein the one or more PSD-FPA further comprises more than one PSD-FPA.

5. The system of claim 2, wherein the electronic control module further comprises human and machine interfaces and a computer and software for system control and system interfaces.

6. The system of claim 5, further comprising interconnect cabling, wherein the optical sensor module further comprises a control and input/output (I/O) component and the electronic control module further comprises an I/O interface, wherein the control and I/O component and I/O interface communicated via the interconnect cabling, and wherein the computer is a signal processing and control computer.

7. The system of claim 6, wherein the optical lens sub-system further comprises one or more minors or light split elements.

8. The system of claim 6, wherein the detector elements of the one or more PSD-FPA comprise a number of lateral effect diode elements, each comprising a sensing surface and four signal sensing connections, wherein each sensing surface comprises a P-N diode structure with a covering of resistive P-layer and illumination photons interact with the sensing surface to generate charges that move through the resistive layer to the signal sensing connections, which connect to the ROIC using indium bump interconnects.

9. The system of claim 8, wherein the ROIC comprises a controllable current source prior to an amplifier corresponding to each signal sensing connection, wherein the controllable current source performs background subtraction by removing a portion of the signals generated from the lateral effect diode elements before they reach the corresponding amplifier.

10. The system of claim 9, wherein the ROIC further comprises a comparator located at the output of each amplifier for adjusting the background subtraction by comparison.

11. The system of claim 10, wherein the comparator comprises an adjustable reference voltage used to establish a controlled uniform background light level, which in turn established controllable contrast levels.

12. The system of claim 10, wherein the ROIC further comprises a variable capacitor across each amplifier controlled by a gain control logic, a sample and hold capacitor that stores the resulting output and is connected dynamically to the amplifier using a sample and hold switch, and an output multiplexer, which allow for variable gain.

13. The system of claim 12, wherein the optical lens sub-system further comprises one or more minors or light split elements, wherein the control and I/O component further comprises a memory buffer for storing output signals from the ROIC, an electronic data-to-electronic control module interface for sending data to the electronic control module when the memory buffer is full, and a low noise level shifting and voltage generator for generating precision control clocks/voltages for the ROIC, further comprising data frames organized by the I/O interface of the electronic control module from data received from the optical sensor module and data windows formed from stacks of the data frames, wherein each data window contains signals from a single element of the one or more PSD-FPA organized in time sequence.

14. The system of claim 6, wherein the control and I/O component further comprises a memory buffer for storing output signals from the ROIC, an electronic data-to-electronic control module interface for sending data to the electronic control module when the memory buffer is full, and a low noise level shifting and voltage generator for generating precision control clocks/voltages for the ROIC.

15. The system of claim 6, further comprising data frames organized by the I/O interface of the electronic control module from data received from the optical sensor module and data windows formed from stacks of the data frames, wherein each data window contains signals from a single element of the one or more PSD-FPA organized in time sequence.

16. An optical event detection and classification method, comprising

providing an optical sensor module comprising one or more position sensing detector focal plane arrays (PSD-FPA), which each comprise a detector and a read out integrated circuit (ROIC), an optical lens sub-system that comprises one or more lenses, and a control and input/output (I/O) component,

providing an electronic control module,

focusing light onto the one or more PSD-FPA with the optical lens sub-system,

sensing photons and generating electrical currents with the detector,

amplifying and sampling the electrical currents with the ROIC,

transmitting the amplified and sampled electrical currents to the control and I/O component,

digitizing and multiplexing the amplified and sampled electrical currents with the control and I/O component,

transmitting the digitized and multiplexed electrical currents to the electronic control module,

receiving the transmitted electrical currents in the electronic control module, and

interpreting the received electrical currents to detect and characterize events.

17. The method of claim 16, wherein the one or more PSD-FPA comprises more than one PSD-FPA, further comprising multiple spectral band sensing using the multiple PSD-FPA.

18. The method of claim 17, further comprising detecting the temperature of an observed object or event using the multiple spectral band sensing.

19. The method of claim 17, further comprising generating control signals with the computer, transmitting the control signals back to the optical sensor module via the I/O interface, receiving the control signals with the control and I/O component, and generating timing pulses and voltages with the control and I/O component to control the PSD-FPA.

20. The method of claim 19, wherein the control and I/O component comprises a memory buffer, data-to-electronic control module interface, and low noise level shifting and voltage generator, wherein the digitizing and multiplexing the amplified and sampled electrical currents with the control and I/O component comprises performing analog-to-digital conversion on the amplified and sampled electrical currents and storing digitized data samples in the memory buffer, wherein the transmitting the digitized and multiplexed electrical currents to the electronic control module comprises sending digitized data samples in the memory buffer to the electronic control module via the data-to-electronic control module interface using a common electrical format when the memory buffer is full, and wherein the generating timing pulses and voltages with the control and I/O component to control the PSD-FPA comprises using received control signals to generate timing and commands and using the low noise level shifting and voltage generator to generate precision control clocks or voltages for the ROIC.

21. The method of claim 19, wherein the receiving the transmitted electrical currents with the I/O interface comprises organizing the transmitted electrical currents into rows and frames of data and the interpreting the received electrical currents carried out in the computer comprises stacking the frames of data to form data windows, each containing signals from a single PSD-FPA diode element organized in time sequence.

22. The method of claim 21, wherein the interpreting the received electrical currents in the computer further comprises multiple processing of two or more data windows simultaneously.

23. The method of claim 22, wherein the interpreting the received electrical currents in the computer further comprises shifting data in a data windows that was not discarded so that the average background value is zero, forming a zeroed signal, determining the start point of a signal rise or pulse that exceeds the event detection threshold, the maximum point of that signal rise or pulse, and the end point of that rise or pulse, if any, using the start point, maximum point, and end point to determine if the signal rise or pulse is a signal rise or a signal pulse, and processing signal rises and signal pulses separately.

24. The method of claim 23, wherein the processing signal pulses comprises calculating line fit equations for the rising and falling edge of each signal pulse as defined by the start point, maximum point, and end point of the pulse and comparing the ratio of the rising and falling slope of each pulse to determine if it falls within the range typical for any of various types of optical events and determining the error between the lines defined by the line fit equations and the actual data.

25. The method of claim 23, wherein the processing signal pulses further comprises performing a time-location correlation analysis to determine if a real location can be associated with an observed event, in the absence of which the source of the observed event is unlikely to be weapons fire.

26. The method of claim 24, wherein the processing signal pulses further comprises fitting two non-linear functions to each signal pulse, determining parameters of the functions, comparing the parameters of the functions to the parameters typical of various types of optical events, and calculating an error metric for each non-linear function measuring how much the actual data deviates from the curve defined by the fit non-linear function.

27. The method of claim 26, wherein the non-linear functions have the form F(J)=K*J*exp(1−

L*J), where J is the time index for the function and is zero for the start point of the signal pulse, and L and K are estimated parameters, and Xj, TRANSFORM=Ln[Xj/J] is the transform function used for F(J), wherein only the points from the maximum point of a pulse to the end point are used to estimate the F(J) parameters K and L, where L can be used to categorize weapons producing muzzle blasts.

28. The method of claim 26, wherein the processing signal pulses further comprises digitally filtering window data and using filter responses and ratios as parameters for comparison and testing, conducting Fourier analysis of window data and using Fourier coefficients and frequency power components and ratios as parameters for comparison and testing, and conducting wavelet analysis of windows data using function parameters, their ratios, and error metrics as parameters for comparison and testing.

29. The method of claim 21, wherein the interpreting the received electrical currents in the computer further comprises autonomously determining an event detection threshold for a data window based on signal variation within that data window and discarding the data window if it does not contain a signal rise or pulse that exceeds the event detection threshold.

30. The method of claim 29, wherein the processing signal rises comprises initial fast processing, maintaining state information, storing records of detected signal rises, and using time and position correlation to determine when a long duration event or a multi-part event has occurred.

31. The method of claim 30, wherein the initial fast processing comprises calculating line fit equations for the signal rise and comparing slope parameters against typical values for weapons-related events and fitting non-linear function to the signal rise.

32. The method of claim 30, wherein the interpreting the received electrical currents in the computer further comprises confirming a rocket propelled grenade by identifying a signal pulse initiation event matching RPG characteristics, followed by a continuous event that changes in intensity or moves in angular location occurring within a set time limit, wherein the processing signal rises further comprises testing to ensure that the pulse initiation event and continuous event occur within the set time limit and are correlated in location.

33. The method of claim 30, wherein the processing signal rises further comprises tracking data in two dimensions to confirm a missile or rocket by identifying a continuous event that changes in intensity or moves in angular location.

34. The method of claim 28, wherein the processing signal pulses further comprises performing a time-location correlation analysis and determining if a signal pulse is event-related and if so what class of event it is related to, using a weighted decision process that integrates each analysis and test performed, wherein each test or analysis is assigned a weighting factor that determines its relevance for each event type.

35. The method of claim 28, wherein the processing signal pulses further comprises performing a time-location correlation analysis and determining if a signal pulse is weapons-related and if so what class of weapon it is related to, using a weighted decision process that integrates each analysis and test performed, wherein each test or analysis is assigned a weighting factor that determines its relevance for each weapon type.

36. The method of claim 25, wherein the performing a time-location correlation analysis comprises determining the start and end points of a signal pulse in a data window and using data from all signal leads to calculate a position for each point in the data window, determining an average background center point location, calculating the distance of each point in the data window from the average background center point location, determining a minimum location shift threshold based on the average background center point location, an average distance deviation from the average background center point location, and a standard deviation of this average distance deviation, and determining whether several points in a row break the minimum location shift threshold.

37. The method of claim 16, wherein the one or more PSD-FPA comprises more than one PSD-FPA, further comprising using one PSD-FPA for passive sensing and the other for laser illuminated sensing.

38. The method of claim 16, wherein the electronic control module comprises an I/O interface and a computer, wherein the receiving the transmitted electrical current in the electronic control module comprises receiving the transmitted electrical currents with the I/O interface, further comprising sending the received electrical currents to the computer, wherein the interpreting the received electrical currents is carried out in the computer.

39. The method of claim 16, further comprising operating the one or more PSD-FPA at data rates of 9,000 to 12,000 frames per second.

40. The method of claim 16, wherein the providing an optical sensor module further comprises configuring the optical lens sub-system into a desired arrangement including one or more mirrors or light split elements in addition to the one or more lenses.

41. The method of claim 16, wherein the detector comprises an array of lateral effect diode elements, each of which has signal leads that connect to the ROIC, wherein the sensing photons and generating electrical currents with the detector comprises generating charges from interaction of photons with the detector material that move through a resistive layer to the signal leads, so that the charges are split between the signal leads according to the resistance encountered, which varies according to the distance between the incident photons giving rise to the charges and each signal lead.

42. The method of claim 41, further comprising performing background subtraction on the electrical currents to remove background clutter.

43. The method of claim 42, wherein the performing the background subtraction comprises performing background subtraction separately and independently for each diode element and for each signal lead on the PSD-FPA by connecting each signal lead to a controllable current source prior to the input of an amplifier in the ROIC and removing a portion of the electrical current from the detector with the controllable current source before it reaches the amplifier.

44. The method of claim 43, further comprising performing background calibration.

45. The method of claim 44, further comprising performing background calibration at a set frequency, to implement dynamic background subtraction and mitigate signals that change more slowly than the background calibration frequency, or based on observed background changes, to ensure that contrast levels remain within a specified range.

46. The method of claim 43, wherein the ROIC further comprises an adjustable comparison reference voltage in a comparator at the output of each amplifier, a compare bit and comparator output switch, and a current source control logic, wherein the performing the background subtraction further comprises performing background calibration by comparing the output voltage of each amplifier with the adjustable comparison reference voltage in the comparator and sending a compare bit value to the current source control logic that sets the value of the controllable current source, wherein the compare bit output is connected to the control logic by the comparator output switch, which is closed, forming the connection only when background calibration is commanded.

47. The method of claim 43, wherein the ROIC further comprises a variable capacitor across each amplifier controlled by gain control logic, further comprising controlling the gain control logic directly with the control and I/O component and indirectly with the electronic control module in order to vary the gain of the electrical currents.

48. The method of claim 47, further comprising implementing a dual-gain, fast switchover tripwire function that changes the gain of an individual PSD-FPA diode element from high to low within a single frame read time when the output of the amplifier rises towards a saturation voltage to prevent saturation from extremely bright events.

49. The method of claim 48, wherein the ROIC further comprises an adjustable comparison reference voltage in a comparator at the output of each amplifier, a compare bit and comparator output switch, and a current source control logic, wherein the performing the background subtraction further comprises performing background calibration by comparing the output voltage of each amplifier with the adjustable comparison reference voltage in the comparator and sending a compare bit value to the current source control logic that sets the value of the controllable current source, wherein the compare bit output is connected to the control logic by the comparator output switch, which is closed, forming the connection only when background calibration is commanded, wherein implementing the dual-gain, fast switchover tripwire function comprises setting the adjustable comparison reference voltage to a tripwire voltage when background calibration is not necessary and sending the compare bit to the current source control logic, which sets all four ROIC circuits for a single PSD-FPA diode element to a tripwire gain setting depending on the compare bit.

50. A method of signal processing for detecting events of interest, comprising

receiving data from a sensor with one or more sensor elements,

organizing the data into one or more data windows that each contain data from one of the one or more sensor elements organized in time sequence,

calculating an average sample-to-sample signal change of the data in one of the one or more data windows,

autonomously determining an event detection threshold for the one data window based on the average sample-to-sample signal change of the data,

determining whether the one data window contains a pulse or signal rise that exceeds the event detection threshold, and

discarding the one data window if it does not contain a pulse or signal rise that exceeds the event detection threshold, thereby minimizing the amount of signal processing required.

51. An event detection and classification method, comprising:

providing one or more sensors, each having one or more sensor elements,

sensing and generating sensor data with the one or more sensors, and

processing the sensor data to identify and categorize events of interest,

wherein the processing the sensor data comprises performing background subtraction, time-intensity and time-location correlation, and wavelet analysis.

52. The method of claim 51, wherein the providing one or more sensors comprises providing one or more position sensing detector focal plane arrays (PSD-FPA), providing an optical lens sub-system that focuses incoming light on the PSD-FPA, wherein the sensing and generating sensor data with the one or more sensors comprises sensing photons and generating electrical signals with the PSD-FPA, wherein the processing the sensor data to identify and categorize events of interest comprises processing the electrical signals to identify and categorize optical events of interest.

53. The method of claim 51, wherein the processing the sensor data further comprises performing Fourier analysis.

54. The method of claim 51, wherein the processing the sensor data further comprises organizing the sensor data into one or more data windows that each contain data from one of the one or more sensor elements organized in time sequence, calculating an average sample-to-sample signal change of the data in one of the one or more data windows, autonomously determining an event detection threshold for the one data window based on the average sample-to-sample signal change of the data, determining whether the one data window contains a pulse or signal rise that exceeds the event detection threshold, and discarding the one data window if it does not contain a pulse or signal rise that exceeds the event detection threshold, thereby minimizing the amount of signal processing required.