SYSTEM AND METHOD OF AUTOMATICALLY DETERMINING MATERIAL REACTION OR SENSITIVITY USING IMAGES

• US 20160112682A1
• Filed: 11/30/2015
• Published: 04/21/2016
• Est. Priority Date: 11/30/2012
• Status: Active Grant

• Current Assignee: Safety Management Services, Inc.
• Original Assignee: Safety Management Services, Inc.
• Inventors: Guymon, Clint
• Granted Patent: US 9,654,742 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G01N 21/88 Investigating the presence of flaws or contamination
• CPC Class Codes: G06F 18/2415 based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 7/001 using an image reference approach
• CPC Class Codes: G06V 10/10 Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: H04N 23/71 Circuitry for evaluating the brightness variation
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

The disclosure extends to systems, methods and computer program products for automatically determining whether an energetic substance or a material has experienced a reaction (“go”) or a non-reaction (“no-go”) during an insult from an impact, friction, ESD or other small-scale sensitivity testing device. The systems, methods, and computer program products of the disclosure use a video capturing device, a CPU or computer, sensitivity test equipment, and a set of rules or instructions to be followed for quantifying and determining whether a reaction has occurred or not.

Claims

1-28. -28. (canceled)

29. A system for evaluating a reaction event of a material comprising:

an image capturing device that captures one or more image frames of the reaction event;

a storage device coupled to the imaging capturing device for storing the one or more image frames in memory;

at least one sensitivity testing device for conducting a test on the material; and

a computing device comprising one or more processors;

wherein the storage device stores instructions that, when executed by one or more processors, cause the processors to,

quantify one or more reaction event characteristics, wherein the instructions further cause the one or more processors to calculate one or more identifiers comprising brightness, buoyancy, shape, and uniformity of the reaction event.

30. The system of claim 29, wherein the one or more identifiers further comprises color;

wherein the one or more image frames are captured based on a triggering event that begins a capture session and an expected length of the baseline reaction event and the trial reaction event that ends the capture session, wherein the one or more frames are recorded and identified as one or more frames of interest.

31. The system of claim 30, wherein the triggering event is an image based trigger, sound based trigger, current based trigger, proximity based trigger, or interrupt based trigger.

32. The system of claim 30, wherein the one or more frames of interest are analyzed for an indication of a significant difference above any other frames of interest and are then identified as one or more significant frames.

33. The system of claim 32, wherein the system further comprises a background map for each of the significant frames that are identified, wherein the background map eliminates ambient information from the quantization of the significant frames.

34. The system of claim 33, wherein the background map is created by standardizing all pixels in each of the significant frames to a baseline brightness value to thereby eliminate all visual effects of the ambient surroundings, such that the analysis is focused directly on an analysis site.

35. The system of claim 30, wherein the system further comprises one or more significant frames that are identified from among the frames of interest, wherein each of the frames of interest are analyzed for a change in an image property and a frame is identified as a significant frame when there is a change in the image property.

36. The system of claim 30, wherein each of the frames of interest are analyzed and compared to an average brightness and a frame is identified as a significant frame when its brightness is more than 1-10 standard deviations greater than a mean brightness.

37. The system of claim 35, wherein the system further comprises a color image for each of the significant frames identified that is converted to a grayscale image, wherein each of the grayscale images is filtered.

38. The system of claim 37, wherein the system further comprises a filter that includes weighting by giving each pixel a grayscale brightness value, wherein the filter is centered on the brightest part of the grayscale image and then the grayscale brightness value is attenuated as a distance from a center of a brightest point increases.

39. The system of claim 38, wherein the attenuation is non-linear and follows a hyperbolic tangent distribution.

40. The system of claim 30, wherein the system assigns a brightness value to each pixel, wherein the brightness value for a given frame is the sum of all pixel values or multiple thereof in that frame, wherein the pixel values comprise red pixel values, blue pixel values, green pixel values, or a combination thereof.

41. The system of claim 30, wherein the system assigns a buoyancy value, wherein buoyancy is calculated from the difference between the average center of mass, based on a grayscale or color pixel value, and the value of the center of mass for the first of the plurality of image frames.

42. The system of claim 30, wherein uniformity is found from a reshaped pixel matrix where the number of peaks in the reshaped pixel matrix is found, summed and then normalized.

43. The system of claim 30, wherein the shape of each of the frames of interest is identified, wherein shape may be calculated by determining the effective aspect ratio;

wherein the shape is calculated by performing matrix multiplication with the pixel matrix and a weighting matrix minus the product of the brightness and center of mass in the horizontal and vertical directions, and normalizing the resulting value and summing those resulting values over all frames of interest.

44. The system of claim 30, wherein the color of each frame is quantified by the sum of the product of the red, green, blue, or combination thereof of pixel values and a weighting factor.

45. The system of claim 30, wherein the system further comprises a mean baseline value and a baseline standard deviation that is calculated for each of the identifiers of brightness, buoyancy, shape, uniformity, or color for the baseline reaction event, wherein the results are stored in computer readable memory.

46. The system of claim 45, wherein the system normalizes the identifiers of brightness, buoyancy, shape, uniformity, or color for the trial reaction event and assigns a value to each identifier, wherein normalization is based on the mean baseline value and the standard deviation for brightness, buoyancy, shape, uniformity, or color of the baseline reaction event.

47. The system of claim 46, wherein the mean baseline values of the baseline identifiers of brightness, buoyancy, shape, uniformity, or color is subtracted from the values of each respective trial identifiers of brightness, buoyancy, shape, uniformity, or color, and is then divided by the baseline standard deviation for the respective identifier.

48. The system of claim 47, wherein the normalized value of each identifier is combined to yield a single identifying value that reflects a difference between the trial reaction event and the baseline reaction event.

49. The system of claim 48, wherein the identifying value is determined from the summation of the trial identifiers of brightness, buoyancy, shape, uniformity, or color and the identifying value is proportional to the number of standard deviations from the baseline values.

50. The system of claim 48, wherein the system further comprises a specified threshold value to determine whether a reaction event falls within that threshold to characterize the reaction event as a go or no-go reaction, such that if the identifying value of the trial identifiers of the reaction event is greater than the specified threshold, the trial is then designated as a go reaction;

and if the identifying value of the trial identifiers of the reaction event is less than the specified threshold, the trial is then designated as a no-go reaction.

51. The system of claim 32, wherein the system comprises a Type I (false positive) error and Type II (false negative) error that is calculated by identifying the number of falsely identified Go'"'"'s and No-Go'"'"'s based on the quantified reaction characteristics of brightness, buoyancy, shape, uniformity, or color.

52. The system of claim 51, wherein a recursive method is used to calculate the Type I error and Type II error, wherein the distribution of the identifying value of the trial identifiers for the go reactions and the distribution of the identifying value of the trial identifiers for the no-go reactions are located in a continuous probability distribution.

53. The system of claim 52, wherein the value of the Type I error or the Type II error is optimized by changing the method or methods to distinguish a reaction from a non-reaction to reflect the appropriate amount of risk obtained by the sensitivity test results.

54. The system of claim 53, wherein a portion of the reactions identified as go reactions to no-go reactions are reclassified based on the estimated probabilities of go reactions and no-go reactions at a given identifying value or collection of identifying values, and a portion of the reactions identified as no-go reactions to go reactions are classified based on the estimated probabilities of go reactions and no-go reactions at a given identifying value or collection of identifying values;

wherein the values for go reactions and no-go reactions are updated;

wherein the distribution of go reactions and no-go reactions are re-estimated based on the updated values for go reactions and no-go reactions at each identifying value or collection of identifying values;

wherein the proportion of go reactions and no-go reactions at each identifying value or collection of identifying values are re-estimated based on the updated distribution of go reactions and no-go reactions;

wherein this process is recursively completed to convergence.

55. The system of claim 54, wherein the proportion of go reactions to no-go reactions at a given identifying value or collection of identifying values is estimated from a ratio of the probability of a go reaction to a no-go reaction at a given identifying value or collection of identifying values, wherein the probability of a go reaction is found from the estimated distribution of go reactions and the probability of a no-go reaction is found from the estimated distribution of no-go reactions.

56. The system of claim 51, wherein the Type I error is minimized to reduce the risk to personnel during operations, storage, and transportation.

57. Computer readable storage media storing instructions for evaluating a reaction event of a material that, when executed by one or more processors, cause the one or more processors to:

capture one or more image frames of a baseline reaction event using an image capturing device, wherein a baseline data set is obtained from the baseline reaction event and wherein the baseline data set does not comprise all information of the material;

store the one or more image frames of the baseline reaction event in computer readable memory;

capture one or more image frames of a trial reaction event using an image capturing device, wherein a trial data set is obtained from the trial reaction event and wherein the trial data set comprises information relating to the material;

store the one or more image frames of the trial reaction event in computer readable memory;

quantify the baseline data set and the trial data set, wherein the quantifying process further comprises one or more of the following identifiers;

calculate brightness from the one or more image frames of the baseline reaction event and the trial reaction event;

calculate the buoyancy from the plurality of image frames of the baseline reaction event and the trial reaction event;

calculate the shape from the one or more image frames of the baseline reaction event and the trial reaction event; and

calculate the uniformity from the one or more image frames of the baseline reaction event and the trial reaction event;

compare the quantified trial data set to the quantified baseline data set; and

determine whether a reaction event occurred based on the comparison of the quantified trial data set to the quantified baseline data set.

58. The readable storage media of claim 57, wherein the one or more identifiers comprises calculating the color from the one or more image frames of the baseline reaction event and the trial reaction event;

wherein the one or more image frames are captured based on a triggering event that begins a capture session and an expected length of the baseline reaction event and the trial reaction event that ends the capture session, wherein the frame or frames are recorded and identified as frame(s) of interest.

59. The readable storage media of claim 58, wherein the triggering event is an image based trigger, or a sound based trigger, or a current based trigger.

60. The readable storage media of claim 58, wherein the frame or frames of interest are analyzed for an indication of a change obtained during the baseline reaction event and the trial reaction event, wherein the frame or frames of interest that indicate a change are separated from the frame or frames of interest where there is no indication of a change and the separated frames are identified as significant frame or frames.

61. The readable storage media of claim 60, wherein the instructions further cause the one or more processors to create a background map for each of the significant frames.

62. The readable storage media of claim 61, wherein the background map is created by standardizing all pixels in each of the significant frames to a baseline brightness value to thereby eliminate all visual effects of the ambient surroundings, such that the analysis is focused directly on an analysis site.

63. The readable storage media of claim 58, wherein the instructions further cause the one or more processors to identify significant frames from among the frames of interest, wherein each of the frames of interest are analyzed for a change in an image property and a frame is identified as a significant frame when there is a change in the image property.

64. The readable storage media of claim 63, wherein each of the frames of interest are analyzed and compared to an average brightness and a frame is identified as a significant frame when its brightness is more than 5 standard deviations greater than a mean brightness.

65. The readable storage media of claim 63, wherein the instructions further cause the one or more processors to convert a color image for each of the significant frames identified to a grayscale image, and filter each of the grayscale images.

66. The readable storage media of claim 65, wherein the instructions further cause the one or more processors to weight the filter by giving each pixel a grayscale brightness value, wherein the filter is centered on the brightest part of the grayscale image and then the grayscale brightness value is attenuated as a distance from a center of a brightest point increases.

67. The readable storage media of claim 66, wherein the attenuation is non-linear and follows a hyperbolic tangent distribution.

68. The readable storage media of claim 63, wherein the instructions further cause the one or more processors to assign a brightness value to each pixel in each of the significant frames, wherein the brightness value for a given significant frame is the sum of all pixel values in that significant frame normalized by resolution and the number of significant frames identified in the baseline reaction event or the trial reaction event.

69. The readable storage media of claim 68, wherein the instructions further cause the one or more processors to assign a buoyancy value, wherein buoyancy is calculated from the difference between the average center of mass, based on a grayscale pixel value, and the value of the center of mass for the first of the one or more image frames.

70. The readable storage media of claim 69, wherein uniformity is found from a reshaped pixel matrix where the number of peaks in the reshaped pixel matrix is found, summed and then normalized.

71. The readable storage media of claim 70, wherein the instructions further cause the one or more processors to identify the shape of each of the significant frames, wherein shape is calculated by weighting each significant frame'"'"'s shape by performing matrix multiplication with the pixel matrix and the weighting matrix minus the product of the brightness and center of mass in the horizontal and vertical directions, and normalize the resulting value and summing those resulting values over all significant frames.

72. The readable storage media of claim 69, wherein color is quantified by the sum of the product of the red, green, blue, or combination thereof of pixel values and a weighting factor.

73. The readable storage media of claim 63, wherein the instructions further cause the one or more processors to calculate a mean baseline value and a baseline standard deviation for each of the identifiers of brightness, buoyancy, shape, uniformity, or color for the baseline reaction event and storing the results in computer readable memory.

74. The readable storage media of claim 72, wherein the instructions further cause the one or more processors to normalize the identifiers of brightness, buoyancy, shape, uniformity, or color for the trial reaction event and assigning a value to each identifier, wherein normalizing is based on the mean baseline value and the standard deviation for brightness, buoyancy, shape and uniformity of the baseline reaction event.

75. The readable storage media of claim 73, wherein normalizing each of the identifiers of brightness, buoyancy, shape, uniformity, or color for the trial reaction event further comprises subtracting the mean baseline values of the baseline identifiers of brightness, buoyancy, shape, uniformity, or color from the values of each of the trial identifiers of brightness, buoyancy, shape, uniformity or color, and then dividing the result by the baseline standard deviation for that identifier.

76. The readable storage media of claim 74, wherein the normalized value of each identifier is combined to yield a single identifying value that reflects a difference between the trial reaction event and the baseline reaction event.

77. The readable storage media of claim 75, wherein the identifying value is determined from the summation of the trial identifiers of brightness, buoyancy, shape, uniformity, or color and the identifying value is proportional to the number of standard deviations from the baseline values.

78. The readable storage media of claim 76, wherein the instructions further cause the one or more processors to set a specified threshold value to determine whether a reaction event falls within predetermined parameters characterizing the reaction event as a go or no-go reaction, such that if the identifying value of the trial identifiers of the reaction event is greater than the specified threshold, the trial is then designated as a go reaction;

and if the identifying value of the trial identifiers of the reaction event is less than the specified threshold, the trial is then designated as a no-go reaction.

79. The readable storage media of claim 77, wherein the instructions further cause the one or more processors to calculate Type I error and Type II error for the specified threshold.

80. The readable storage media of claim 78, wherein a recursive method is used to calculate the Type I error and Type II error, wherein the distribution of the identifying value of the trial identifiers for the go reactions and the distribution of the identifying value of the trial identifiers for the no-go reactions are located in a continuous probability distribution.

81. The readable storage media of claim 79, wherein the value of the Type I error or the Type II error is optimized by changing the threshold value to reflect the appropriate amount of risk obtained by the sensitivity test results.

82. The readable storage media of claim 80, wherein the instructions further cause the one or more processors to:

reclassify a portion of the reactions identified as go reactions to no-go reactions based on the estimated probabilities of go reactions and no-go reactions at a given identifying value or collection of identifying values, and reclassifying a portion of the reactions identified as no-go reactions to go reactions based on the estimated probabilities of go reactions and no-go reactions at a given identifying value or collection of identifying values;

update the values for go reactions and no-go reactions;

re-estimate the distribution of go reactions and no-go reactions based on the updated values for go reactions and no-go reactions at each identifying value or collection of identifying values;

re-estimate the proportion of go reactions and no-go reactions at each identifying value or collection of identifying values based on the updated distribution of go reactions and no-go reactions;

wherein this process is recursively completed to convergence.

83. The readable storage media of claim 81, wherein the instructions further cause the one or more processors to:

estimate the proportion of go reactions to no-go reactions at a given identifying value or collection of identifying values from a ratio of the probability of a go reaction to a no-go reaction at a given identifying value or collection of identifying values, wherein the probability of a go reaction is found from the estimated distribution of go reactions and the probability of a no-go reaction is found from the estimated distribution of no-go reactions.

84. The readable storage media of claim 78, wherein the instructions further cause the one or more processors to adjust the threshold to optimize the Type I error and the Type II error to minimize the risk to personnel during operations, storage, and transportation.

85. Computer readable storage media storing instructions for evaluating a reaction event of a material that, when executed by one or more processors, cause the one or more processors to:

capture one or more image frames of a baseline reaction event using an image capturing device, wherein a baseline data set is obtained from the baseline reaction event and wherein the baseline data set does not comprise all information of the material;

store the one or more image frames of the baseline reaction event in computer readable memory;

capture one or more image frames of a trial reaction event using an image capturing device, wherein a trial data set is obtained from the trial reaction event and wherein the trial data set comprises information relating to the material;

store the one or more image frames of the trial reaction event in computer readable memory;

quantify the baseline data set and the trial data set, wherein the quantifying process further comprises one or more of the following identifiers;

calculate brightness from the one or more image frames of the baseline reaction event and the trial reaction event;

calculate the buoyancy from the plurality of image frames of the baseline reaction event and the trial reaction event;

calculate the shape from the one or more image frames of the baseline reaction event and the trial reaction event;

calculate the uniformity from the one or more image frames of the baseline reaction event and the trial reaction event; and

calculate the color from the one or more image frames of the baseline reaction event and the trial reaction event;

compare the quantified trial data set to the quantified baseline data set; and

determine whether a reaction event occurred based on the comparison of the quantified trial data set to the quantified baseline data set.