Method and apparatus for wave analysis and event recognition

• US 5,276,629 A
• Filed: 08/14/1992
• Issued: 01/04/1994
• Est. Priority Date: 06/21/1990
• Status: Expired due to Fees

• Current Assignee: Morgan J. Cowles, Sarah A. Cowles
• Original Assignee: Reynolds Software, Inc.
• Inventors: Reynolds, Kentyn
• Primary Examiner(s): Black, Thomas G.
• Assistant Examiner(s): ZANELLI, MICHAEL J
• Application Number:
  US07/930,476
• Time in Patent Office: 508 Days
• Field of Search: 364/487, 364/484, 364/485, 364/481, 364/483, 381/41, 381/42, 381/43, 381/45, 381/48, 381/49, 381/50, 382/16, 382/43, 84/601, 84/608, 84/615, 324/77 R, 324/77 A, 324/76.13, 324/76.18, 324/76.21, 324/76.33, 324/76.41
• US Class Current: 702/71
• CPC Class Codes: G06F 2218/12 Classification; Matching
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 25/87 Detection of discrete points within a voice signal

Abstract

A method and apparatus for acquiring, recording, synchronizing, analyzing, and interpreting continuous wave data. A process for isolating separate events from wave data composed of multiple events and for determining the identification and characteristics of an event'"'"'s wave source, its frequency components, amplitude, duration, and timing. A procedure for defining the parameters of the source wave and for adapting these parameters to the analysis requirements (e.g., correct frequency, amplitude, and timing divisions). A procedure for verifying interpretation results, for correcting interpretation errors, and for retaining successful results. Products of the processes, such as databases and recordings, can be made.

Claims

1. A computer implemented method for determining a similarity between discrete portions of a continuous stream of wave data and a predetermined set of frequency ratios, the method comprising the steps of:

(a) generating a spectral time strip for a discrete portion of the continuous stream of wave data, the spectral time strip including at least a first partial and a second partial;

(b) determining a first frequency position for a prevalent amplitude peak of the first partial;

(c) determining a second frequency position for a prevalent amplitude peak of the second partial;

(d) determining a first frequency ratio according to the expression ##EQU1## (e) comparing the first frequency ratio with at least one frequency ratio from the predetermined set of frequency ratios to obtain a similarity score; and

(f) generating a signal according to a predetermined scale that is indicative of the degree of similarity between the discrete portion of the continuous stream of wave data and at least one frequency ratio from the predetermined set of frequency ratios.

2. The method as recited in claim 1, wherein step (e) further comprises comparing the first frequency ratio with a time related frequency ratio from the predetermined set of frequency ratios to obtain a similarity score.

3. The method as recited in claim 1, wherein step (e) further comprises comparing the first frequency ratio with a plurality of frequency ratios from the predetermined set of frequency ratios to obtain a similarity scores.

4. The method as recited in claim 1, 2, or 3, wherein each of the frequency ratios from the predetermined set of frequency ratios includes a frequency peak for a fourth partial time related to the second partial and a frequency peak for a third partial time related to the first partial.

5. The method as recited in claim 1, wherein step (b) further comprises the step of scanning the spectral time strip at the first frequency position.

6. The method as recited in claim 1, wherein step (c) further comprises the step of scanning the spectral time strip at the second frequency position.

7. The method as recited in claim 1, wherein step (b) further comprises searching the spectral time strip within a first predetermined frequency range for an amplitude peak that is most prevalent.

8. The method as recited in claim 1, wherein step (c) further comprises searching the spectral time strip within a second predetermined frequency range for an amplitude peak that is most prevalent.

9. The method as recited in claim 1, wherein the continuous stream of wave data includes a continuous stream of complex wave data.

10. The method as recited in claim 1 or 9, wherein the continuous stream of wave data includes a continuous stream of digital wave data.

60. The method as recited in claim 1, wherein step (f) further comprises generating signals according to a predetermined scale that are indicative of the degree of similarity between the discrete portion of the continuous stream of wave data and the plurality of frequency ratios from the predetermined set of frequency ratios.

61. The method as recited in claim 1, 11, 14, 18, 22, 26, 29, 34, 40, 45, or 48 wherein the continuous stream of wave data includes a continuous stream of analog wave data.

11. A computer implemented method for determining a similarity between predetermined wave data and discrete portions of a continuous stream of wave data, the method comprising the steps of:

(a) inputting as a first input predetermined spectral wave data, the predetermined spectral wave data further including a plurality of time strips, with each time strip including at least a first partial and a second partial;

(b) generating a frequency ratio between the first partial and the second partial for each time strip of the predetermined spectral wave data according to the expression ##EQU2## (c) storing the frequency ratios for each time strip of the predetermined spectral wave data;

(d) inputting as a second input a continuous stream of wave data;

(e) generating from the second input spectral wave data that includes a plurality of time strips, with each time strip further including at least a third partial and a fourth partial, with the third partial being time related to the first partial, and the fourth partial being time related to the second partial;

(f) generating a frequency ratio between the third partial and the fourth partial for each time strip of the spectral wave data according to the expression ##EQU3## (g) comparing each frequency ratio of spectral wave data associated with the second input with the stored frequency ratio of a predetermined time strip of the predetermined spectral wave data to obtain a similarity score for each time strip; and

(h) generating a signal for each time strip according to a predetermined scale that is indicative of the degree of similarity between the continuous stream of wave data and the predetermined spectral wave data.

12. The method as recited in claim 11, wherein the continuous stream of wave data includes a continuous stream of complex wave data.

13. The method as recited in claim 11 or 12, wherein the continuous stream of wave data includes a continuous stream of digital wave data.

14. A computer implemented method for determining the similarity between predetermined wave data and discrete portions of a continuous stream of wave data, the method comprising the steps of:

(a) inputting as a first input predetermined first spectral wave data, the spectral wave data further including a first plurality of time strips, with each time strip including at least a first partial;

(b) inputting as a second input the continuous stream of wave data;

(c) generating from the second input second spectral wave data that includes a second plurality of time strips, with each time strip including at least a second partial;

(d) comparing an amplitude for each of the plurality of time strips of the first partial with an amplitude for each of the plurality of time related time strips of the second partial;

(e) subtracting the amplitudes of the time strips of the second partial from the amplitudes of the time strips of the first partial, respectively, for each comparison at step (d) to obtain an amplitude difference for each of the respective time strips;

(f) summing each amplitude difference to obtain a total difference to determine if said difference is less than a predetermined value to indicate similarity; and

(g) generating a signal indicative of the degree of similarity between the discrete portions of the continuous stream of wave data and the predetermined wave data based on the total difference determined at step (f).

15. The method as recited in claim 14, wherein the method further comprises the steps of:

(h) determining an offset percentage for each comparison at step (d);

(i) comparing the offset percentages to determining if there is a substantially consistent offset percentage for each comparison at step (d); and

(j) normalizing the time strip from the second spectral wave data if the offset percentages are within a predetermined range.

16. The method as recited in claim 14, wherein the continuous stream of wave data is a continuous stream of complex wave data.

17. The method as recited in claim 14 or 16, wherein the continuous stream of wave data is a continuous stream of digital wave data.

18. A computer implemented method for determining a start time of a new event in a continuous stream of wave data, the method comprising the steps of:

(a) inputting first spectral wave data that defines a predetermined event occurring at a first predetermined time location;

(b) determining for the first spectral wave data a first frequency-distribution width and a first frequency-distribution width form at the first predetermined time location;

(c) generating second spectral wave data for discrete portions of the continuous stream of wave data;

(d) determining for the second spectral wave data a plurality of frequency-distribution widths and a plurality of frequency-distribution width forms at a plurality of time locations;

(e) comparing the first frequency-distribution width/first frequency-distribution width form determined at step (b) with each of the plurality of frequency-distribution widths/frequency-distribution width forms determined at step (d) to determine whether the continuous stream of wave data includes a new event as defined by the first frequency-distribution width/first frequency-distribution width form; and

(f) generating a signal representative of whether the continuous stream of wave data includes a new event as defined by the first frequency-distribution width/first frequency-distribution width form.

19. The method as recited in claim 18, wherein step (e) includes identifying a time location of a new event in the continuous stream of wave data.

20. The method as recited in claim 18, wherein the continuous stream of wave data includes a continuous stream of complex wave data.

21. The method as recited in claim 18 or 20, wherein the continuous stream of wave data includes a continuous stream of digital wave data.

22. A computer implemented method for generating a spectral time strip from a continuous stream of wave data, the wave data including frequency, amplitude, and time information, the method comprising the steps of:

(a) inputting as a first input the continuous stream of wave data;

(b) inputting as a second input a frequency scale that includes a plurality of frequency scale points;

(c) selecting from the continuous stream of wave data a time strip of a predetermined time interval;

(d) Fast Fourier transforming the time strip to produce a Fast Fourier Transform ("FFT") time strip that includes a plurality of amplitude values at a plurality of FFT frequency points;

(e) selecting a frequency scale point from the frequency scale;

(f) determining a series of FFT frequency points on the FFT time strip that includes the frequency scale point selected at step (e);

(g) interpolating, using a polynomial interpolation, a curve from the FFT frequency points determined at step (f);

(h) determining from the curve interpolated at step (g) an amplitude value at the frequency scale point selected at step (e);

(i) storing the amplitude value determined at step (h) and frequency scale point selected at step (e) as an amplitude - frequency pair;

(j) repeating steps (e) to (i) for each of the plurality of frequency scale points; and

(k) generating a spectral time strip based on the stored amplitude - frequency pairs.

23. The method as recited in claim 22, wherein step (g) further comprises interpolating using a Chi-squared polynomial.

24. The method as recited in claim 22, wherein the continuous stream of wave data includes a continuous stream of complex wave data.

25. The method as recited in claim 22, or 23, or 24, wherein the continuous stream of wave data includes a continuous stream of digital wave data.

26. A computer implemented method for generating a spectral time strip from a continuous stream of wave data, the wave data including frequency, amplitude, and time information, the method comprising the steps of:

(a) inputting as a first input the continuous stream of wave data;

(b) inputting as a second input a frequency scale that includes a plurality of frequency scale points;

(c) selecting from the continuous stream of wave data a time strip of a predetermined time interval;

(d) Fast Fourier transforming the time strip to produce a Fast Fourier Transform ("FFT") time strip that includes a plurality of amplitude values at a plurality of FFT frequency points;

(e) selecting a frequency scale point from the frequency scale;

(f) determining a frequency range of frequency points for the frequency scale point selected at step (e), with the frequency scale point being located in the frequency range;

(g) searching the frequency range of frequency points for a highest amplitude value of the plurality of amplitude vales of the FFT frequency points within the frequency range;

(h) storing the highest amplitude value found at step (g) and frequency scale point selected at step (e) as an amplitude - frequency pair;

(i) repeating steps (e) to (h) for each of the plurality frequency scale points of the frequency scale; and

(j) generating a spectral time strip from the stored amplitude - frequency pairs.

27. The method as recited in claim 26, wherein the continuous stream of wave data includes a continuous stream of complex wave data.

28. The method as recited in claim 26 or 27, wherein the continuous stream of wave data includes a continuous stream of digital wave data.

29. A computer implemented method for generating a spectral time strip from a continuous stream of wave data, the wave data including frequency, amplitude, and time information, the method comprising the steps of:

(a) inputting as a first input the continuous stream of wave data;

(b) inputting as a second input a frequency scale that includes a plurality of frequency scale points;

(c) selecting from the continuous stream of wave data a time strip of a predetermined time interval;

(d) Fast Fourier transforming the time strip to produce a Fast Fourier Transform ("FFT") time strip that includes a plurality of amplitude values at a plurality of FFT frequency points;

(e) selecting a frequency scale point from the frequency scale;

(f) performing steps (g) and (h) if the selected frequency scale point is at or above a first predetermined frequency, and performing steps (i), (j), (k), and (l), if the selected frequency scale point is below a second predetermined frequency;

(g) determining a frequency amplitude in the FFT time strip in a frequency range that includes the frequency scale point selected at step (e);

(h) storing the highest amplitude determined a step (g) and frequency scale point selected at step (e) as an amplitude - frequency pair and then performing step (m);

(i) determining a series of FFT frequency points on the FFT time strip that includes the frequency scale point selected at step (e);

(j) interpolating, using a polynomial interpolation, a curve from the FFT frequency points determined at step (i);

(k) determining from the curve interpolated at step (j) an amplitude value at the frequency scale point selected at step (e);

(l) storing the amplitude value determined at step (j) and frequency scale point selected at step (e) as an amplitude - frequency pair;

(m) repeating steps (e) to (l) for each of the plurality of frequency scale points of the frequency scale; and

(n) generating a spectral time strip from the amplitude - frequency pairs stored at steps (h) or (l).

30. The method as recited in claim 29, wherein step (g) further comprises the steps of:

determining a frequency range of frequency points for the frequency scale point selected at step (e); and

searching the frequency range of frequency points for a highest amplitude value of the plurality of amplitude values of the FFT frequency points within the frequency range.

31. The method as recited in claim 29, wherein step (j) further comprises interpolating using a Chi-squared polynomial.

32. The method as recited in claim 29, wherein the continuous stream of wave data includes a continuous stream of complex wave data.

33. The method as recited in claim 29, or 30, 31, or 32, wherein the continuous stream of wave data includes a continuous stream of digital wave data.

34. A computer implemented method for generating a spectral time strip from a continuous stream of wave data, the wave data including frequency, amplitude, and time information, the method comprising the steps of:

(a) inputting as a first input the continuous stream of wave data;

(b) inputting as a second input a frequency scale that includes a plurality of frequency scale points;

(c) selecting from the continuous stream of wave data a plurality of time strips having independent time intervals;

(d) Fast Fourier transforming the plurality of time strips to produce N Fast Fourier Transform ("FFT") time strips an each of the N FFT time strips includes a plurality of amplitude values at a plurality of FFT frequency points;

(e) selecting one of the N FFT time strips produced at step (d);

(f) selecting a frequency scale point from the frequency scale;

(g) determining a series of FFT frequency points on the FFT time strip selected at step (e) that includes the frequency scale point selected at step (f);

(h) interpolating, using a polynomial interpolation, a curve for the FFT time strip selected at step (e) from the FFT frequency points determined at step (g);

(i) determining from the curve interpolated at step (h) an amplitude value at the frequency scale point selected at step (f);

(j) storing the amplitude value determined at step (i) and frequency scale point selected at step (f) as an amplitude - frequency pair;

(k) repeating steps (f) to (j) for each of the plurality of frequency scale points;

(l) generating a spectral time strip based on the stored amplitude - frequency pairs for the FFT time strip selected at step (e);

(m) repeating steps (e) and (l) for each of the N FFT time strips; and

(n) combining the spectral time strips generated at step (l) to produce a resulting spectral time strip.

35. The method as recited in claim 34, wherein step (n) further comprises combining the spectral time strips using amplitude averaging and amplitude clipping.

36. The method as recited in claim 34, wherein step (n) further comprises combining the spectral time strips using frequency limiting.

37. The method as recited in claim 34, wherein step (h) further comprises interpolating using a Chi-squared polynomial.

38. The method as recited in claim 34, wherein the continuous stream of wave data includes a continuous stream of complex wave data.

39. The method as recited in claim 34, or 35, 37, or 38, wherein the continuous stream of wave data includes a continuous stream of digital wave data.

40. A computer implemented method for generating a spectral time strip from a continuous stream of wave data, the wave data including frequency, amplitude, and time information, the method comprising the steps of:

(a) inputting as a first input the continuous stream of wave data;

(b) inputting as a second input a frequency scale that includes a plurality of frequency scale points;

(c) selecting from the continuous stream of wave data a plurality of time strips having independent time intervals;

(d) Fast Fourier transforming the plurality of time strips to produce N Fast Fourier Transform ("FFT") time strips an each of the N FFT time strips includes a plurality of amplitude values at a plurality of FFT frequency points;

(e) selecting one of the N FFT time strips produced at step (d);

(f) selecting a frequency scale point from the frequency scale;

(g) determining a frequency range of frequency points for the frequency scale point selected at a step (f), with the frequency scale point being located in the frequency range;

(h) searching the frequency range of frequency points for a highest amplitude value of the plurality of amplitude values of the FFT frequency points of the FFT time strip selected at step (e);

(i) storing the highest amplitude value found at step (h) and frequency scale point selected at step (f) as an amplitude - frequency pair;

(j) repeating steps (f) to (i) for each of the plurality frequency scale points of the frequency scale;

(k) generating a spectral time strip from the stored amplitude - frequency pairs for the FFT time strip selected at step (e);

(l) repeating steps (e) and (j) for each of the N FFT time strips; and

(m) combining the spectral time strips generated at step (k) to produce a resulting spectral time strip.

41. The method as recited in claim 40, wherein step (m) further comprises combining the spectral time strips using amplitude averaging and amplitude clipping.

42. The method a recited in claim 40, wherein step (m) further comprises combining the spectral time strips using frequency limiting.

43. The method a recited in claim 40, wherein the continuous stream of wave data includes a continuous stream of complex wave data.

44. The method a recited in claim 40 or 43, wherein the continuous stream of wave data includes a continuous stream of digital wave data.

45. A computer implemented method for determining a similarity between a predetermined frequency drift envelope and a frequency drift envelope for a continuous stream of wave data, the method comprising the steps of:

(a) generating a spectral time strip for a discrete portion of the continuous stream of wave data, the spectral time strip including at least one partial;

(b) scanning the spectral time strip within a predetermined frequency range for an amplitude peak that is most prevalent;

(c) determining a frequency position for the most prevalent amplitude peak;

(d) determining a frequency envelope for the amplitude peaks with a frequency envelope including the prevalent amplitude peak;

(e) determining a relative drift between the prevalent amplitude peak and each of the amplitude peaks in the envelope;

(f) determining a drift center based on an average of the plurality of relative drifts determined at step (e);

(g) comparing the drift center determined at step (f) with the drift center of the predetermined drift envelope to obtain a similarity score; and

(h) generating a signal according to a predetermined scale that is indicative of the degree of similarity between the predetermined frequency drift envelope and the frequency drift envelope for the continuous stream of wave data.

46. The method as recited in claim 45, wherein the continuous stream of wave data includes a continuous stream of complex wave data.

47. The method as recited in claim 45 or 46, wherein the continuous stream of wave data includes a continuous stream of digital wave data.

48. A computer implemented method for determining a similarity between predetermined wave data and discrete portions of a continuous stream of wave data, the method comprising the steps of:

(a) inputting as a first input predetermined spectral wave data, the predetermined spectral wave data further including a plurality of time strips, with each time strip including at least a first partial and a second partial;

(b) generating a first signal that includes first distortion information for each time strip of the predetermined spectral wave data;

(c) storing the first distortion information for each time strip of the predetermined spectral wave data;

(d) inputting as a second input a continuous stream of wave data;

(e) generating from the second input spectral wave data that includes a plurality of time strips, with each time strip further including at least a third partial and a fourth partial;

(f) generating a second signal that includes second distortion information for each time strip of the spectral wave data;

(g) comparing the second distortion information with the stored first distortion information to obtain a similarity score for each time strip; and

(h) generating signals according to a predetermined scale that are indicative of the degree of similarity between the predetermined spectral wave data and the continuous stream of wave data.

49. The method as recited in claim 48, wherein the first distortion information includes amplitude-independent, frequency-distribution widths for the first and second partials, and the second distortion information includes amplitude-independent, frequency-distribution widths for the third and fourth partials.

50. The method as recited in claim 48, wherein the first distortion information includes amplitude-independent, frequency-distribution widths settle-times for the first and second partials, and the second distortion information includes amplitude-independent, frequency-distribution widths settle-times for the third and fourth partials.

51. The method as recited in claim 48, wherein the first distortion information includes amplitude-independent, frequency-distribution width forms for the first and second partials, and second distortion information includes amplitude-independent, frequency-distribution width forms for the third and fourth partials.

52. The method as recited in claim 48, wherein step (c) further comprises the step of storing a first amplitude-independent slope for the first and second partials, step (f) further comprises the step of generating a second amplitude-independent slope for the third and fourth partials, and step (g) further comprises the step of comparing the stored first amplitude-independent slope with the second amplitude-independent slope.

53. The method as recited in claim 48, wherein step (c) further comprises the step of storing a first amplitude-independent slope for the first and second partials if the first partial is coterminous with the second partial, step (f) further comprises the step of generating a second amplitude-independent slope for the third and fourth partials if the third partial is coterminous with the fourth partial, and step (g) further comprises the step of comparing the stored first amplitude-independent slope with the second amplitude-independent slope.

54. The method as recited in claim 48, wherein the continuous stream of wave data includes a continuous stream of complex wave data.

55. The method as recited in claim 48 or 54, wherein the continuous stream of wave data includes a continuous stream of digital wave data.

56. An apparatus for determining a similarity between predetermined wave data and discrete portions of a continuous stream of wave data, the apparatus comprising:

(a) storage device for storing a predetermined set of frequency ratios;

(b) input device that receives discrete portions of the continuous stream of wave data;

(c) first generator coupled to the output of the input device for generating a spectral time strip with the spectral time strip including at least a first and a second partial;

(d) a processor for determining a first frequency position for a prevalent amplitude peak of the first partial, a second frequency position for a prevalent amplitude peak of a second partial, and generating a first frequency ratio according to the expression ##EQU4## (e) comparator for comparing the first frequency ratio with at least one stored frequency ratio to obtain a similarity score.

57. An apparatus for determining a similarity between predetermined wave data and discrete portions of a continuous stream of wave data, the apparatus comprising:

(a) input device for receiving as the first input predetermined spectral wave data, the predetermined spectral wave data further including a plurality of time strips, with each time strip including at least a first partial and a second partial, and as a second input the continuous stream of wave data;

(b) first generator coupled to the output of the input device for generating a frequency ratio between the first and second partials for each time strip of the predetermined spectral wave data according to the expression ##EQU5## (c) storage device for storing the frequency ratios for each time strip of the predetermined spectral wave data;

(d) second generator coupled to the output of the input device for generating from the second input spectral wave data a plurality of time strips, with each time strip further including at least a third partial and a fourth partial;

(e) third generator coupled to the output of the second generator for generating a frequency ratio between the third partial and the fourth partial for each time strip of the spectral wave data according to the expression ##EQU6## (f) comparator for comparing each frequency ratio of spectral wave data associated with the second input with the stored frequency ratio of a predetermined time strip of the predetermined spectral wave data to obtain a similarity score for each time strip.

58. An apparatus for generating a spectral time strip form a continuous stream of wave data, the wave data including frequency, amplitude, and time information, the apparatus comprising:

(a) input device for receiving as a first input the continuous stream of wave data and as a second input a frequency scale that includes a plurality of frequency scale points;

(b) processor for selecting from the continuous stream of wave data a time strip having a predetermined time interval, Fast Fourier transforming the time strip to produce a Fast Fourier Transform ("FFT") time strip that includes a plurality of amplitude values at a plurality of FFT frequency points, selecting a frequency scale point from the frequency scale, and if the selected frequency scale point is at or above a first predetermined frequency, determining a highest amplitude in the FFT time strip in a frequency range that includes the selected frequency scale point, and if the selected frequency scale point is below a second predetermined frequency, determining a series of FFT frequency points on the FFT time strip that includes the selected frequency scale point, interpolating, using a polynomial interpolation, a curve from the determined FFT frequency points, and determining from the interpolated curve an amplitude value at the selected frequency scale point;

(c) storage device for storing the highest amplitude and frequency scale point based on at least one Fast Fourier transformed time strip as a first amplitude - frequency pair and storing the amplitude value and scale point based on at least one interpolated curve as a second amplitude - frequency pair; and

(d) generator for generating a spectral time strip from a stored first or second amplitude - frequency pair.

59. An apparatus for determining a similarity between predetermined wave data stored in a memory device and discrete portions of a continuous stream of wave data, the apparatus comprising:

(a) input device to receive as input a continuous stream of wave data;

(b) converter coupled to the output of the input device for generating a spectral time strip from the continuous stream of wave data using polynomial interpolation;

(c) extractor for extracting from the spectral time strip a plurality of identifying characteristics;

(d) storage device for storing predetermined wave data; and

(e) comparator coupled to the output of the extractor and the storage device for comparing the plurality of identifying characteristics extracted by extractor with the stored predetermined wave data to obtain a similarity score.