Video-based identification of operational mode shapes

• US 10,037,609 B2
• Filed: 02/01/2016
• Issued: 07/31/2018
• Est. Priority Date: 02/01/2016
• Status: Active Grant

• Current Assignee: Massachusetts Institute of Technology
• Original Assignee: Massachusetts Institute of Technology
• Inventors: Chen, Justin Gejune, Buyukozturk, Oral, Freeman, William T., Durand, Frederic Pierre, Davis, Myers Abraham, Wadhwa, Neal
• Primary Examiner(s): Entezari, Michelle M
• Application Number:
  US15/012,835
• Publication Number:
  US 20170221216A1
• Time in Patent Office: 911 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments

Abstract

A method and corresponding device for identifying operational mode shapes of an object in a video stream includes extracting pixel-wise Eulerian motion signals of an object from an undercomplete representation of frames within a video stream. Pixel-wise Eulerian motion signals are downselected to produce a representative set of Eulerian motion signals of the object. Operational mode shapes of the object are identified based on the representative set. Resonant frequencies can also be identified. Embodiments enable vibrational characteristics of objects to be determined using video in near real time.

Claims

1. A method of identifying operational mode shapes of an object in a video stream, the method comprising:

extracting pixel-wise Eulerian motion signals of an object from an undercomplete representation of frames within a video stream;

downselecting signals from the pixel-wise Eulerian motion signals to produce a representative set of pixel-wise Eulerian motion signals of the object; and

identifying operational mode shapes of the object based on the representative set of pixel-wise Eulerian motion signals from the undercomplete representation of the frames within the video stream.

2. The method of claim 1, further comprising identifying motion frequencies of the object based on the representative set of motion signals from the undercomplete representation of the frames within the video stream.

3. The method of claim 1, wherein the pixel-wise Eulerian motion signals are equal in number to the pixels within the frames of the video stream.

4. The method of claim 1, wherein downselecting the signals includes choosing signals on the basis of local contrast in the frames within the video stream.

5. The method of claim 1, wherein end-to-end processing of the video stream to identify the operational mode shapes of the object is performed at a rate at least one order of magnitude faster than Ncorr digital image correlation (DIC).

6. The method of claim 1, further including downsampling signals prior to extracting the pixel-wise Eulerian motion signals.

7. The method of claim 1, further including performing an outlier analysis to remove erroneous motion signals from the pixel-wise Eulerian motion signals.

8. The method of claim 1, further comprising averaging motion signals of the representative set of motion signals, and wherein identifying operational mode shapes of the object includes using the averaged motion signals.

9. The method of claim 1, further comprising generating a visual representation of the operational mode shapes or motion signals of the object.

10. The method of claim 1, further comprising uploading the frames within the video stream to a remote server or downloading the operational mode shapes from the remote server.

11. The method of claim 1, wherein the extracting, downselecting, and identifying occur at a network server and operate on the frames received via a network path.

12. The method of claim 1, wherein identifying the operational mode shapes includes determining a frequency spectrum for each motion signal of the representative set of motion signals.

31. The method of claim 4, wherein choosing signals on the basis of local contrast includes at least one of applying a signal energy filter and applying a local contrast thresholding filter.

13. A device for identifying operational mode shapes of an object in a video stream, the device comprising:

memory configured to store frames from a video stream of an object; and

a processor configured to;

extract pixel-wise Eulerian motion signals of the object from an undercomplete representation of the frames from the video stream,

downselect signals from the pixel-wise Eulerian motion signals to produce a representative set of pixel-wise Eulerian motion signals of the object, and

identify operational mode shapes of the object based on the representative set of pixel-wise Eulerian motion signals from the undercomplete representation of the frames within the video stream.

14. The device of claim 13, wherein the processor is further configured to identify motion frequencies of the object based on the representative set of motion signals from the undercomplete representation of the frames within the video stream.

15. The device of claim 13, wherein the pixel-wise Eulerian motion signals are equal in number to the pixels within the frames of the video stream.

16. The device of claim 13, wherein the processor is further configured to downselect the signals on the basis of local contrast in the frames within the video stream.

17. The device of claim 13, wherein the processor is further configured to perform end-to-end processing of the video stream to obtain the operational mode shapes of the object at least one order of magnitude faster than Ncorr digital image correlation (DIC).

18. The device of claim 13, wherein the processor is further configured to downsample signals prior to extracting the pixel-wise Eulerian motion signals.

19. The device of claim 13, wherein the processor is further configured to perform an outlier analysis to remove erroneous motion signals from the pixel-wise Eulerian motion signals.

20. The device of claim 13, wherein the processor is further configured to average two or more motion signals of the representative set of motion signals and to identify the operational mode shapes of the object based on the averaged representative set of motion signals.

21. The device of claim 13, further comprising a visual interface configured to display a representation of the operational mode shapes or motion signals of the object.

22. The device of claim 13, further comprising a communications interface configured to upload the video stream of the object to a remote server or to download the operational mode shapes of the object from the remote server.

23. The device of claim 13, wherein the memory and processor form part of a network server configured to receive the frames from the video stream via a network path.

24. The device of claim 13, wherein the processor is further configured to determine a frequency spectrum for each motion signal of the representative set of motion signals.

25. The device of claim 13, wherein the memory is operational mode shape video storage memory configured to store a number of video frames spanning at least one motion period corresponding to a resonant frequency of the object.

26. The device of claim 13, wherein the processor is an operational mode shape data processor configured to receive the video frames of the object and to extract the pixel-wise Eulerian signals by applying oriented complex spatial bandpass filters to data representing individual pixels of the frames within the video stream.

27. The device of claim 26, wherein the operational mode shape data processor is further configured to determine local phase and local amplitude to extract the pixel-wise Eulerian motion signals and determine edge strength of pixels in the frames within the video stream simultaneously.

28. A quality control system comprising the device of claim 13, wherein the object is a product being manufactured or tested, the system further including:

a vibration transducer configured to cause vibration of the product; and

an operational mode shape video camera configured to capture images of the product during vibration.

29. An equipment monitoring system comprising the device of claim 13, wherein the object comprises a component of a machine, the component in motion during machine operation, the system further comprising an operational mode shape video camera configured to capture images of the one or more components in motion, and wherein the processor is further configured to identify an unwanted motion frequency in the component in motion.

32. The device of claim 16, wherein the processor is further configured to downselect the signals on the basis of local contrast by applying at least one of a signal energy filter and a local contrast thresholding filter.

30. A device for identifying operational mode shapes of an object in a video stream, the device comprising:

means for extracting pixel-wise Eulerian motion signals of an object from an undercomplete representation of frames within a video stream;

means for downselecting signals from the pixel-wise Eulerian motion signals to produce a representative set of pixel-wise Eulerian motion signals of the object; and

means for identifying operational mode shapes of the object based on the representative set of pixel-wise Eulerian motion signals from the undercomplete representation of the frames within the video stream.