Apparatus and method for visualizing periodic motions in mechanical components

• US 10,062,411 B2
• Filed: 06/09/2016
• Issued: 08/28/2018
• Est. Priority Date: 12/11/2014
• Status: Active Grant

• Current Assignee: RDI Technologies, Inc.
• Original Assignee: RDI Technologies, Inc.
• Inventors: Hay, Jeffrey R., Slemp, Mark William
• Primary Examiner(s): Woldemariam, Aklilu
• Application Number:
  US14/999,660
• Publication Number:
  US 20160300341A1
• Time in Patent Office: 810 Days
• Field of Search: 382103, 382107, 382134, 382274, 348143, 348241, 348383, 348696, 348730, 362276
• CPC Class Codes: G01N 2291/0289 Internal structure, e.g. defects, grain size, texture
• CPC Class Codes: G01N 2291/2693 Rotor or turbine parts
• CPC Class Codes: G01N 29/12 by measuring frequency or resonance of acoustic waves measuring frequency or resonant frequency of mechanical vibrations or acoustic waves in general G01H1/06, G01H3/04, G01H13/00; acoustic resonators G10K11/04; vibration or shock testing of structures G01M7/00
• CPC Class Codes: G06F 16/7335 Graphical querying, e.g. query-by-region, query-by-sketch, query-by-trajectory, GUIs for designating a person/face/object as a query predicate end-user interface involving hot spots associated with the video H04N21/4725; end-user interface for selecting a Region of Interest H04N21/4728
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 2218/10 by analysing the shape of a waveform, e.g. extracting parameters relating to peaks
• CPC Class Codes: G06F 3/005 Input arrangements through a video camera
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20056 Discrete and fast Fourier transform, [DFT, FFT]
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/262 using transform domain methods, e.g. Fourier domain methods
• CPC Class Codes: G06V 20/46 Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G11B 27/022 Electronic editing of analogue information signals, e.g. audio or video signals
• CPC Class Codes: H04N 5/148 : Video amplifiers amplifiers in general H03F

Abstract

An apparatus for visualizing physical movements includes:

• • a device for acquiring video image files;
  • a data analysis system including processor and memory;
  • a computer program operating in the processor to identify an area in the images where periodic motions associated with physical movement of an object may be detected and quantified, and compute a new image sequence in which the motions are visually amplified; and,
  • a user interface that displays the motion-amplified video image of the mechanical component.

An associated method for using the apparatus is also disclosed.

Claims

1. An apparatus for visualizing physical movements comprising:

a processor and a memory for storage;

a device for acquiring video image files, each video file having a sequence of video images comprising at least two individual still image frames, which are stored in memory and divided into pixels defined by two or more spatial coordinates within the frame, wherein pixel intensity is determined by brightness of light reflected or thermal energy emitted at that pixel with such intensity values expressed as amplitude;

a computer program operating in said processor to;

compare the amplitude of at least one set of corresponding pixels in a first image frame to that in a second image frame;

determine an amplitude that corresponds with movement in an object in an area of the frame defined by one or more pixels that includes the at least one corresponding pixel;

create motion amplification by the steps of;

identifying one of the image frames as a reference frame;

combining frames by identifying another of said image frames as a difference frame and overlaying the difference frame over the reference frame;

wherein the combining step comprises increasing the amplitude of the difference frame by an amplification factor that increases the intensity value of the corresponding pixel in the difference frame as representation of motion;

and compute a new image sequence in which said motions are visually amplified which the apparatus is configured to display via a user interface that displays a motion-amplified video image of said object.

2. The apparatus of claim 1 wherein said device for acquiring video files is selected from the group consisting of:

video cameras, webcams, digital cameras integral in a mobile device, and memory devices containing prerecorded video files.

3. The apparatus of claim 1 wherein said user interface further displays the raw video images of said object for visual comparison to said motion-amplified video of said object.

4. The apparatus of claim 1 wherein said computer program detects a triggering event associated with exceeding a threshold level of movement the system determines based on an amplitude of motion level versus time during which video files are acquired, wherein the data analysis system creates motion amplification upon detecting the triggering event.

5. The apparatus of claim 4 wherein said user interface allows a user to vary said selected amplification factor over a selected range.

6. The apparatus of claim 1 wherein said user interface allows a user to select a frame rate for displaying said motion-amplified video that is less than the frame rate of said acquired video image file.

7. The apparatus of claim 1 wherein said user interface allows a user to display a single frame of video corresponding to a selected time.

8. The apparatus of claim 1 wherein said user interface displays the instant value of at least one parameter selected from the group consisting of:

motion amplification factor;

frame rate of said motion-amplified video display; and

the time corresponding to a selected single video frame.

9. The apparatus of claim 1 wherein said user interface allows a user to export said motion-amplified video file.

10. The apparatus of claim 9 wherein said user interface allows the user to export said motion-amplified video file along with data selected from the group consisting of:

the corresponding raw video file, and, metadata associated with said motion-amplified video file.

11. A method for visualizing physical movements comprising the steps of:

acquiring video image files, each video file having a sequence of video images comprising at least two individual still image frames, which are stored in memory and divided into pixels defined by two or more spatial coordinates within the frame, wherein pixel intensity is determined by brightness of light reflected or thermal energy emitted at that pixel with such intensity values expressed as amplitude;

configuring a memory to store two or more image frames;

in the operation of a computer program, comparing the amplitude of at least one set of corresponding pixels in a first image frame to that in a second image frame;

determining an amplitude that corresponds with movement in an object in an area of the frame defined by one or more pixels that includes the at least one corresponding pixel;

creating motion amplification by the steps of;

identifying one of the image frames as a reference frame;

combining frames by identifying another of said image frames as a difference frame and overlaying the difference frame over the reference frame;

wherein the combining step comprises increasing the amplitude of the difference frame by an amplification factor that increases the intensity value of the corresponding pixel in the difference frame as representation of motion; and

,

computing a new image sequence file in which said motions are visually amplified; and

displaying a motion-amplified video image of said object.

12. The method of claim 11 wherein said GUI allows a user to select a motion amplification factor.

13. The method of claim 11 wherein said GUI allows a user to select a frame rate for displaying said motion-amplified images.

14. The method of claim 11 wherein said GUI further displays corresponding raw images of said component for visual comparison.

15. The method of claim 11 further including the step of:

exporting said motion-amplified video images of said object along with any user-selected metadata.

16. The method of claim 11 wherein said object is selected from the group consisting of:

bridges, tracks, beams, cables and guy wires, structural members, motors and pumps, linkages, shafts, and airfoils.

17. A system for visualizing physical movements in an object, comprising:

a device for acquiring video image files, each video file having a sequence of video images comprising at least two individual still image frames, which are stored in memory and divided into pixels defined by two or more spatial coordinates within the frame, wherein pixel intensity is determined by brightness of light reflected or thermal energy emitted at that pixel with such intensity values expressed as amplitude;

a data analysis system including processor and memory for storage;

a computer program operating in said processor to;

compare the amplitude of at least one set of corresponding pixels in a first image frame to that in a second image frame;

determine an amplitude that corresponds with movement in an object in an area of the frame defined by one or more pixels that includes the at least one corresponding pixel;

create motion amplification by the steps of;

identifying one of the image frames as a reference frame;

combining frames by identifying another of said image frames as a difference frame and overlaying the difference frame over the reference frame;

wherein the combining step comprises increasing the amplitude of the difference frame by an amplification factor that increases the intensity value of the corresponding pixel in the difference frame as representation of motion; and

,

compute a new image sequence in which said motions are visually amplified; and

further comprising a user interface configured to allow display of a motion-amplified video file or an image frame of said object.

18. The system of claim 17 wherein said user interface further displays the raw video images of said object for visual comparison to said motion-amplified video of said object.

19. The system of claim 17 wherein said user interface allows a user to vary said selected amplification factor over a selected range.

20. The system of claim 17 wherein said user interface allows a user to select a frame rate for displaying said motion-amplified video that is less than the frame rate of said acquired video image file.

21. The system of claim 17 wherein said user interface displays the instant value of at least one parameter selected from the group consisting of:

motion amplification factor;

frame rate of said motion-amplified video display; and

the time corresponding to a selected single video frame.