Image processing of apparatus condition

• US 8,824,731 B2
• Filed: 10/31/2007
• Issued: 09/02/2014
• Est. Priority Date: 10/31/2007
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Maggiore, Jeanne C., Andresen, Gerrod E.
• Primary Examiner(s): Bitar, Nancy
• Application Number:
  US11/932,326
• Publication Number:
  US 20090110289A1
• Time in Patent Office: 2,498 Days
• Field of Search: 382/195, 382/181, 382/135, 382/190, 382/141
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 7/001 using an image reference approach

Abstract

In one embodiment, a method of continually monitoring and detecting in real-time a condition of an apparatus is detailed. In one step, an apparatus is continually monitored in real-time using continual real-time images of the apparatus taken by at least one camera. In another step, the continual real-time images of the apparatus from the at least one camera are communicated to at least one computer processing unit. In still another step, the continual real-time images of the apparatus are processed using at least one software program embedded in the at least one computer processing unit in order to monitor and detect in real-time a condition of the apparatus.

Claims

1. A method of monitoring and detecting in real-time a fault of a monitored apparatus comprising:

collecting a reference image fault database by taking fault images of a reference apparatus when the reference apparatus experiences a fault, the reference apparatus being substantially similar to the monitored apparatus;

monitoring the monitored apparatus by taking a real-time image of the monitored apparatus with at least one camera;

communicating the real-time image of the monitored apparatus from the at least one camera to at least one computer processing unit;

detecting a real-time fault of the monitored apparatus using said at least one computer processing unit using a software program by comparing the real-time image of the monitored apparatus to said fault images of said reference apparatus to determine whether the monitored apparatus has a fault; and

communicating via the computer processing unit the detected fault of the monitored apparatus to a user interface and alerting a user as to what corrective action is required.

2. The method of claim 1 wherein the monitored apparatus comprises at least one of an equipment, a tool, a vehicle, a satellite, an aircraft, an underwater device, or an in-space device.

3. The method of claim 1 wherein the monitoring of the monitored apparatus comprises the at least one camera taking the real-time image of a pre-determined portion of the monitored apparatus.

4. The method of claim 1 wherein the monitoring of the monitored apparatus comprises the at least one camera taking the real-time image of at least one of a setting, a position, a reading, a color change, or a surface change of said monitored apparatus.

5. The method of claim 1 wherein the at least one camera comprises at least one of high-resolution, infrared, high-definition, laser, night-vision, optical, light-illumination, or laser-illumination.

6. The method of claim 1 wherein the monitoring of the monitored apparatus comprises the at least one camera taking the real-time image of an aircraft during at least one of pre-flight, in-flight, or post-flight.

7. The method of claim 1 wherein the at least one camera is mounted on one or more robotic arms.

8. The method of claim 1 further comprising at least one of storing the real-time image of the monitored apparatus into a storage database or trending a condition of the monitored apparatus over time based on the real-time image of the monitored apparatus using the at least one computer processing unit.

9. The method of claim 1 further comprising the at least one computer processing unit determining whether the monitored apparatus is outside of a desired state, determining whether the monitored apparatus is being used improperly, determining remaining life of the monitored apparatus, or determining predicted failure of the monitored apparatus.

10. The method of claim 1 further comprising at least one of examining the monitored apparatus, repairing the monitored apparatus, or replacing the monitored apparatus based on the detected real-time fault of the monitored apparatus.

11. The method of claim 1 further comprising performing the method automatically.

19. The method of claim 1 wherein the monitoring the monitored apparatus comprises taking continuous real-time images of the monitored apparatus with the at least one camera.

20. The method of claim 9 further comprising the at least one computer processing unit determining whether the monitored apparatus is being used improperly.

12. A system for monitoring and detecting in real-time a fault of a monitored apparatus comprising:

at least one camera for taking a real-time image of the monitored apparatus;

at least one computer processing unit for processing the real-time image of the monitored apparatus taken by the at least one camera;

a reference image fault database in communication with the at least one computer processing unit, said reference image fault database containing fault images of a reference apparatus which were taken when the reference apparatus experienced a fault, the reference apparatus being substantially similar to the monitored apparatus;

programming code in communication with the at least one computer processing unit configured to detect a real-time fault of the monitored apparatus by comparing the real-time image of the monitored apparatus taken by the at least one camera with the fault images of the reference apparatus contained in the reference image fault database; and

a user interface in communication with the computer processing unit, wherein the computer processing unit communicates the detected fault of the monitored apparatus to the user interface and alerts a user as to what corrective action is required.

13. The system of claim 12 wherein the system is attached to at least one of equipment, a tool, a vehicle, a satellite, an aircraft, an underwater device, or an in-space device.

14. The system of claim 12 wherein the at least one camera is at least one of high-resolution, infrared, high-definition, laser, night-vision, optical, light-illuminated, or laser-illuminated.

15. The system of claim 12 wherein the at least one camera is mounted on one or more robotic arms.

16. The system of claim 12 further comprising at least one storage database for storing the real-time fault of the monitored apparatus or trending a condition of the monitored apparatus over time.

17. The system of claim 12 wherein the system is configured to automatically monitor and detect in real-time the fault of the monitored apparatus.

18. The system of claim 12 wherein the programming code is configured to determine whether the monitored apparatus is outside of a desired state, whether the monitored apparatus is being used improperly, a remaining life of the monitored apparatus, or a predicted failure of the monitored apparatus.