System and method of automatically determining material reaction or sensitivity using images

• US 9,654,742 B2
• Filed: 11/30/2015
• Issued: 05/16/2017
• Est. Priority Date: 11/30/2012
• Status: Expired due to Fees

• Current Assignee: Safety Management Services, Inc.
• Original Assignee: Safety Management Services, Inc.
• Inventors: Guymon, Clint
• Primary Examiner(s): Yentrapati, Avinash
• Application Number:
  US14/954,986
• Publication Number:
  US 20160112682A1
• Time in Patent Office: 533 Days
• CPC Class Codes: G01N 21/88 Investigating the presence of flaws or contamination
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06V 10/10 Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

The disclosure extends to systems, methods and computer program products for automatically determining whether an energetic substance or a material has experienced a reaction (“go”) or a non-reaction (“no-go”) during an insult from an impact, friction, ESD or other small-scale sensitivity testing device. The systems, methods, and computer program products of the disclosure use a video capturing device, a CPU or computer, sensitivity test equipment, and a set of rules or instructions to be followed for quantifying and determining whether a reaction has occurred or not.

Claims

1. A system for evaluating a reaction event of a material comprising:

an image capturing device that captures one or more image frames of the reaction event;

a storage device coupled to the imaging capturing device for storing the one or more image frames in memory;

at least one sensitivity testing device for conducting a test on the material; and

a computing device comprising one or more processors;

wherein the storage device stores instructions that, when executed by one or more processors, cause the processors to,

quantify one or more reaction event characteristics based on the image frames, wherein quantifying the one or more reaction event characteristics comprises calculating one or more identifiers comprising brightness, buoyancy, shape, or uniformity of the reaction event; and

wherein the instructions further cause the one or more processors to determine and store a characterization of the reaction event as a go or no-go reaction based the one or more reaction event characteristics;

wherein calculating the one or more identifiers further comprises calculating color;

wherein the image capture device is configured to capture the one or more image frames are captured based on a triggering event that begins a capture session and an expected length of a baseline reaction event or the reaction event that ends the capture session, wherein the one or more frames are recorded and identified as one or more frames of interest;

wherein the instructions further cause the one or more processors to identify one or more significant frames from among the frames of interest, wherein the instructions cause the one or more processors to identify the one or more significant frames by analyzing each of the frames of interest for a change in an image property and identifying a frame as a significant frame when there is a change in the image property;

wherein the instructions further cause the one or more processors to convert a color image for each of the significant frames to a grayscale image; and

wherein the instructions further cause the one or more processors to filter the grayscale image by giving each pixel a grayscale brightness value, wherein instructions cause the one or more processors to attenuate the grayscale brightness value as a distance from a center of a brightest point increases.

2. The system of claim 1, wherein the triggering event comprises an image based trigger, sound based trigger, current based trigger, proximity based trigger, or interrupt based trigger.

3. The system of claim 1, wherein the one or more frames of interest are analyzed for an indication of a significant difference from a previous frame of interest and are then identified as one or more significant frames.

4. The system of claim 3, wherein the instructions further cause the one or more processors to apply a background map to each of the significant frames that are identified, wherein the background map eliminates ambient information from the quantization of the significant frames.

5. The system of claim 4, wherein the instructions cause the one or more processors to create the background map by standardizing all pixels in each of the significant frames to a baseline brightness value to thereby eliminate all visual effects of the ambient surroundings, such that the analysis is focused directly on an analysis site.

6. The system of claim 1, wherein the image property of one or more of the frames of interest comprises an average brightness and wherein the instructions cause the one or more processors to identify one or more of the frames of interest as the significant frame when its brightness is more than 1-10 standard deviations greater than a mean brightness of the one or more frames of interest.

7. The system of claim 1, wherein instructions cause the one or more processors to attenuate using a non-linear attenuation following a hyperbolic tangent distribution.

8. The system of claim 1, wherein the instructions cause the processors to calculate a brightness by assigning a brightness value to each pixel, and a brightness value for a given frame that comprises a sum of all pixel values or multiple of all pixel values in the given frame, wherein the pixel values comprise one or more of red pixel values, blue pixel values, or green pixel values.

9. The system of claim 1, wherein the instructions cause the processors to calculate a buoyancy based on a difference between an average center of mass, based on a grayscale or color pixel value, and a value of a center of mass for a plurality of image frames.

10. The system of claim 1, wherein the instructions cause the one or more processors to calculate the uniformity based on a reshaped pixel matrix where a number of peaks in the reshaped pixel matrix is determined, summed and normalized.

11. The system of claim 1, wherein the instructions cause the one or more processors to calculate shape by determining an effective aspect ratio of a frame by performing matrix multiplication with the pixel matrix and a weighting matrix minus the product of the brightness and center of mass in the horizontal and vertical directions, and normalizing the resulting value and summing those resulting values over all frames of interest.

12. The system of claim 1, wherein the color of each frame is quantified by the sum of the product of the red, green, blue, or combination thereof of pixel values and a weighting factor.

13. The system of claim 1, wherein the storage device stores a mean baseline value and a baseline standard deviation for one or more of the identifiers of brightness, buoyancy, shape, uniformity, or color for the baseline reaction event.

14. The system of claim 13, wherein the instructions further cause the one or more processors to normalize the identifiers of brightness, buoyancy, shape, uniformity, or color for the reaction event and assigns a value to each identifier, wherein normalization is based on the mean baseline value and the standard deviation for brightness, buoyancy, shape, uniformity, or color of the baseline reaction event.

15. The system of claim 14, wherein the instructions cause the processors to:

subtract the mean baseline value of the baseline identifiers of brightness, buoyancy, shape, uniformity, or color from the values of each respective trial identifiers of brightness, buoyancy, shape, uniformity, or color; and

divide by the baseline standard deviation for the respective identifier.

16. The system of claim 15, wherein the instructions further cause the processors to combine the normalized value of each identifier to yield a single identifying value that reflects a difference between the trial reaction event and the baseline reaction event.

17. The system of claim 16, wherein the instructions cause the one or more processors to determine the identifying value from the summation of the trial identifiers of brightness, buoyancy, shape, uniformity, or color and wherein the identifying value is proportional to the number of standard deviations from the baseline values.

18. The system of claim 16, wherein the instructions further cause the one or more processors to determine a threshold value for characterizing the reaction event as a go or no-go reaction, such that if the identifying value of the trial identifiers of the reaction event is greater than the threshold, the trial is then designated as a go reaction;

and if the identifying value of the trial identifiers of the reaction event is less than the threshold, the trial is then designated as a no-go reaction.

19. The system of claim 3, wherein the instructions further cause the one or more processors to calculate a Type I (false positive) error and Type II (false negative) error by identifying the number of falsely identified Go'"'"'s and No-Go'"'"'s based on the quantified reaction characteristics of brightness, buoyancy, shape, uniformity, or color.

20. The system of claim 19, wherein the instructions cause the one or more processors to calculate the Type I error and Type II error in a recursive manner, wherein a distribution of the identifying value of the trial identifiers for the go reactions and a distribution of the identifying value of the trial identifiers for the no-go reactions are located in a continuous probability distribution.

21. The system of claim 20, wherein the instructions cause the one or more processors to optimize a value of the Type I error or the Type II error by changing the method or methods to distinguish a reaction from a non-reaction to reflect the appropriate amount of risk obtained in sensitivity test results.

22. The system of claim 21, wherein the instructions further cause the one or more processors to:

reclassify a portion of the reactions identified as go reactions based on the estimated probabilities of go reactions and no-go reactions at a given identifying value or collection of identifying values, and reclassify a portion of the reactions identified as no-go reactions based on the estimated probabilities of go reactions and no-go reactions at a given identifying value or collection of identifying values;

update the values for go reactions and no-go reactions; and

re-estimate the distribution of go reactions and no-go reactions based on the updated values for go reactions and no-go reactions at each identifying value or collection of identifying values;

wherein the proportion of go reactions and no-go reactions at each identifying value or collection of identifying values are re-estimated based on the updated distribution of go reactions and no-go reactions;

wherein the instructions cause the processors to repeatedly reclassify, update, and re-estimate until convergence.

23. The system of claim 22, wherein the proportion of go reactions to no-go reactions at a given identifying value or collection of identifying values is estimated from a ratio of the probability of a go reaction to a no-go reaction at a given identifying value or collection of identifying values, wherein the probability of a go reaction is found from the estimated distribution of go reactions and the probability of a no-go reaction is found from the estimated distribution of no-go reactions.

24. The system of claim 19, wherein the Type I error is minimized to reduce the risk to personnel during operations, storage, and transportation.

25. Non-transitory computer readable storage media storing instructions for evaluating a reaction event of a material that, when executed by one or more processors, cause the one or more processors to:

capture one or more image frames of a baseline reaction event using an image capturing device, wherein a baseline data set is obtained from the baseline reaction event and wherein the baseline data set does not comprise all information of the material;

store the one or more image frames of the baseline reaction event in computer readable memory;

capture one or more image frames of a trial reaction event using an image capturing device, wherein a trial data set is obtained from the trial reaction event and wherein the trial data set comprises information relating to the material;

store the one or more image frames of the trial reaction event in computer readable memory;

quantify the baseline data set and the trial data set, wherein the quantifying process further comprises calculating one or more of the following identifiers;

brightness from the one or more image frames of the baseline reaction event and the trial reaction event;

buoyancy from the plurality of image frames of the baseline reaction event and the trial reaction event;

shape from the one or more image frames of the baseline reaction event and the trial reaction event; and

uniformity from the one or more image frames of the baseline reaction event and the trial reaction event;

compare the quantified trial data set to the quantified baseline data set; and

determine whether a reaction event occurred based on the comparison of the quantified trial data set to the quantified baseline data set

wherein calculating the one or more identifiers comprises color from the one or more image frames of the baseline reaction event and the trial reaction event;

wherein the one or more image frames are captured based on a triggering event that begins a capture session and an expected length of the baseline reaction event and the trial reaction event that ends the capture session, wherein the frame or frames are recorded and identified as frame(s) of interest;

wherein the instructions further cause the one or more processors to identify significant frames from among the frames of interest, wherein each of the frames of interest are analyzed for a change in an image property and a frame is identified as a significant frame when there is a change in the image property;

wherein the instructions further cause the one or more processors to convert a color image for each of the significant frames identified to a grayscale image, and filter each of the grayscale images; and

wherein the instructions further cause the one or more processors to weight the filter by giving each pixel a grayscale brightness value, wherein the filter is centered on the brightest part of the grayscale image and then the grayscale brightness value is attenuated as a distance from a center of a brightest point increases.

26. The readable storage media of claim 25, wherein the triggering event is an image based trigger, or a sound based trigger, or a current based trigger.

27. The readable storage media of claim 25, wherein the frame or frames of interest are analyzed for an indication of a change obtained during the baseline reaction event and the trial reaction event, wherein the frame or frames of interest that indicate a change are separated from the frame or frames of interest where there is no indication of a change and the separated frames are identified as significant frame or frames.

28. The readable storage media of claim 27, wherein the instructions further cause the one or more processors to create a background map for each of the significant frames.

29. The readable storage media of claim 28, wherein the background map is created by standardizing all pixels in each of the significant frames to a baseline brightness value to thereby eliminate all visual effects of the ambient surroundings, such that the analysis is focused directly on an analysis site.

30. The readable storage media of claim 25, wherein each of the frames of interest are analyzed and compared to an average brightness and a frame is identified as a significant frame when its brightness is more than 5 standard deviations greater than a mean brightness.

31. The readable storage media of claim 25, wherein the attenuation is non-linear and follows a hyperbolic tangent distribution.

32. The readable storage media of claim 25, wherein the instructions further cause the one or more processors to assign a brightness value to each pixel in each of the significant frames, wherein the brightness value for a given significant frame is the sum of all pixel values in that significant frame normalized by resolution and the number of significant frames identified in the baseline reaction event or the trial reaction event.

33. The readable storage media of claim 32, wherein the instructions further cause the one or more processors to assign a buoyancy value, wherein buoyancy is calculated from the difference between an average center of mass, based on a grayscale pixel value, and the value of the center of mass for the first of the one or more image frames.

34. The readable storage media of claim 33, wherein uniformity is found from a reshaped pixel matrix where the number of peaks in the reshaped pixel matrix is found, summed and then normalized.

35. The readable storage media of claim 34, wherein the instructions further cause the one or more processors to identify the shape of each of the significant frames, wherein shape is calculated by weighting each significant frame'"'"'s shape by performing matrix multiplication with the pixel matrix and the weighting matrix minus the product of the brightness and center of mass in the horizontal and vertical directions, and normalize the resulting value and summing those resulting values over all significant frames.

36. The readable storage media of claim 33, wherein color is quantified by the sum of the product of the red, green, blue, or combination thereof of pixel values and a weighting factor.

37. The readable storage media of claim 25, wherein the instructions further cause the one or more processors to calculate a mean baseline value and a baseline standard deviation for each of the identifiers of brightness, buoyancy, shape, uniformity, or color for the baseline reaction event and storing the results in computer readable memory.

38. The readable storage media of claim 36, wherein the instructions further cause the one or more processors to normalize the identifiers of brightness, buoyancy, shape, uniformity, or color for the trial reaction event and assigning a value to each identifier, wherein normalizing is based on the mean baseline value and the standard deviation for brightness, buoyancy, shape and uniformity of the baseline reaction event.

39. The readable storage media of claim 37, wherein normalizing each of the identifiers of brightness, buoyancy, shape, uniformity, or color for the trial reaction event further comprises subtracting the mean baseline values of the baseline identifiers of brightness, buoyancy, shape, uniformity, or color from the values of each of the trial identifiers of brightness, buoyancy, shape, uniformity or color, and then dividing the result by the baseline standard deviation for that identifier.

40. The readable storage media of claim 38, wherein the normalized value of each identifier is combined to yield a single identifying value that reflects a difference between the trial reaction event and the baseline reaction event.

41. The readable storage media of claim 39, wherein the identifying value is determined from the summation of the trial identifiers of brightness, buoyancy, shape, uniformity, or color and the identifying value is proportional to the number of standard deviations from the baseline values.

42. The readable storage media of claim 40, wherein the instructions further cause the one or more processors to set a specified threshold value to determine whether a reaction event falls within predetermined parameters characterizing the reaction event as a go or no-go reaction, such that if the identifying value of the trial identifiers of the reaction event is greater than the specified threshold, the trial is then designated as a go reaction;

and if the identifying value of the trial identifiers of the reaction event is less than the specified threshold, the trial is then designated as a no-go reaction.

43. The readable storage media of claim 41, wherein the instructions further cause the one or more processors to calculate Type I error and Type II error for the specified threshold.

44. The readable storage media of claim 42, wherein a recursive method is used to calculate the Type I error and Type II error, wherein the distribution of the identifying value of the trial identifiers for the go reactions and the distribution of the identifying value of the trial identifiers for the no-go reactions are located in a continuous probability distribution.

45. The readable storage media of claim 43, wherein the value of the Type I error or the Type II error is optimized by changing the threshold value to reflect the appropriate amount of risk obtained by the sensitivity test results.

46. The readable storage media of claim 44, wherein the instructions further cause the one or more processors to:

reclassify a portion of the reactions identified as go reactions to no-go reactions based on the estimated probabilities of go reactions and no-go reactions at a given identifying value or collection of identifying values, and reclassifying a portion of the reactions identified as no-go reactions to go reactions based on the estimated probabilities of go reactions and no-go reactions at a given identifying value or collection of identifying values;

update the values for go reactions and no-go reactions;

re-estimate the distribution of go reactions and no-go reactions based on the updated values for go reactions and no-go reactions at each identifying value or collection of identifying values;

re-estimate the proportion of go reactions and no-go reactions at each identifying value or collection of identifying values based on the updated distribution of go reactions and no-go reactions;

wherein this process is recursively completed to convergence.

47. The readable storage media of claim 45, wherein the instructions further cause the one or more processors to:

estimate the proportion of go reactions to no-go reactions at a given identifying value or collection of identifying values from a ratio of the probability of a go reaction to a no-go reaction at a given identifying value or collection of identifying values, wherein the probability of a go reaction is found from the estimated distribution of go reactions and the probability of a no-go reaction is found from the estimated distribution of no-go reactions.

48. The readable storage media of claim 42, wherein the instructions further cause the one or more processors to adjust the threshold to optimize the Type I error and the Type II error to minimize the risk to personnel during operations, storage, and transportation.