Integration of manufacturing control functions using a multi-functional vision system

• US 9,377,778 B2
• Filed: 08/20/2010
• Issued: 06/28/2016
• Est. Priority Date: 02/17/2010
• Status: Active Grant

• Current Assignee: The Boeing Co.
• Original Assignee: The Boeing Co.
• Inventors: Crothers, Phillip John
• Primary Examiner(s): YU, ARIEL J
• Application Number:
  US12/859,926
• Publication Number:
  US 20110200245A1
• Time in Patent Office: 2,139 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: B64F 5/10 Manufacturing or assembling aircraft, e.g. jigs therefor
• CPC Class Codes: G05B 19/4183 characterised by data acquisition, e.g. workpiece identification
• CPC Class Codes: Y02P 90/02 Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Abstract

A manufacturing control system comprises a vision system for viewing operations within a manufacturing area and for producing vision data representing the viewed operations. A systems control analyzes the vision data, and controls at least two control functions related to the manufacturing operations based on the analyzed vision data.

Claims

1. A manufacturing control system, comprising:

quality control, safety control, logistics, and automation control functions for controlling aircraft manufacturing operations within a manufacturing area, wherein the logistics function monitors a position and a status of aircraft components, to track the aircraft components against an expected location and time within the manufacturing area, wherein the safety function predicts a collision between vehicles within the manufacturing area, and wherein the aircraft components comprise components for assembly of aircraft;

a vision system for viewing the aircraft components within the manufacturing area and for producing images of the viewed aircraft components, wherein the vision system includes digital cameras each having pan, tilt and zoom functions, wherein the digital cameras are directed to view the aircraft components through use of one of the pan, tilt, and zoom functions to point and zoom the digital cameras at aircraft components that are subject of the logistics function not within a field of view of the digital cameras when used to perform the safety function; and

a systems control for extracting vision data from the images and for providing the extracted vision data to the quality control, safety control, logistics, and automation control functions.

2. The manufacturing control system of claim 1, further comprising:

software for simulating, analyzing, and intervening with operations within the manufacturing area based on the extracted vision data.

3. The manufacturing control system of claim 1, further comprising:

laser projection means for projecting nominal data onto the aircraft components within the manufacturing area that may be viewed by the vision system, and

wherein the systems control is operative to use the nominal data to locate features on the aircraft components within the manufacturing area using images of the aircraft components recorded by the vision system.

4. The manufacturing control system of claim 1, wherein the automation control function is configured to control automated equipment within the manufacturing area, the logistic control function is configured to control logistics of the operations within the manufacturing area,

the safety control function is configured to control safety of the operations within the manufacturing area;

and

the quality control function is configured to control the quality of the aircraft components manufactured in the manufacturing area.

5. The manufacturing control system of claim 4, wherein each of the manufacturing control functions includes software for interrogating the vision system and for analyzing the extracted vision data.

6. The manufacturing control system of claim 1, wherein the logistics function uses a live tracking model and a live 3-D simulation environment to track the aircraft components against the expected times and locations.

7. The manufacturing control system of claim 6, wherein the aircraft components includes a part and the logistics function determines whether assembly of the part has been completed.

8. The manufacturing control system of claim 1, wherein the aircraft components include at least one of, parts, tools, fixtures, or consumables.

9. The manufacturing control system of claim 1, wherein the logistics function provides real time monitoring of work completion including detection of incomplete work or out of sequence work.

10. A manufacturing control system, comprising:

a vision system including a plurality of vision sensors for viewing aircraft components within a manufacturing area and for recording images of the viewed aircraft components, wherein the vision sensors comprise digital cameras each having pan, tilt and zoom functions, wherein the digital cameras are directed to view the aircraft components through use of one of the pan, tilt, and zoom functions to point and zoom the vision sensors at aircraft components that are subject of a logistic function not within a field of view of the vision sensors used to perform a safety function, wherein the logistics function monitors a position and a status of the aircraft components, to track the aircraft components against an expected location and time within the manufacturing area, wherein the safety function predicts a collision between vehicles within the manufacturing area, and wherein the aircraft components comprise components for assembly of aircraft;

a plurality of manufacturing control functions for controlling manufacturing operations within the manufacturing area, each of the manufacturing control functions including means for analyzing vision data and for controlling a manufacturing operation based on results of the analysis; and

a systems control for controlling the vision system and for providing the vision data to each of the manufacturing control functions at differing preselected times using the images recorded by the vision sensors.

11. The manufacturing control system of claim 10, wherein the plurality of manufacturing control functions includes:

an automation control function configured to control automated equipment within the manufacturing area,

the logistic control function being configured to control logistics of the operations within the manufacturing area,

the safety control function being configured to control safety of the operations within the manufacturing area, and

a quality control function configured to control the quality of items manufactured in the manufacturing area.

12. The manufacturing control system of claim 10, wherein:

means for analyzing vision data includes a software program for interrogating the vision system and for analyzing the vision data.

13. The manufacturing control system of claim 10, further comprising:

software used by the systems control for simulating, analyzing, and intervening with operations within the manufacturing area based on the extracted vision data.

14. The manufacturing control system of claim 10, further comprising:

laser projection means for projecting nominal data onto aircraft components within the manufacturing area that may be viewed by the vision system, and

wherein the systems control is operative to use the nominal data to locate and measure features on the aircraft components within the manufacturing area using images of the aircraft components recorded by the vision sensors.

15. A computer implemented method comprising:

concurrently controlling a combination of safety and logistics operations within a manufacturing area, comprising;

viewing the manufacturing area with a vision system, including using vision sensors to view aircraft components within the manufacturing area and record images of the viewed aircraft components, wherein the vision sensors comprise digital cameras each having pan, tilt and zoom functions to point and zoom the vision sensors at aircraft components that are subject of a logistics function that is not within a field of view of the vision sensors used to perform a safety control function, wherein the logistics function monitors a position and a status of the aircraft components to track the aircraft components against an expected location and time within the manufacturing area, and wherein the safety function predicts a collision between vehicles within the manufacturing area;

extracting vision data from the recorded images using a processor, creating extracted vision data;

sending the extracted vision data to manufacturing control functions at differing preselected times, wherein the manufacturing control functions include the safety control and logistics functions; and

using the extracted vision data to perform the safety control and logistics functions in the manufacturing area.

16. The method of claim 15, wherein the control functions include at least one of:

determining whether work in a manufacturing operation has been completed, and

determining whether work in a manufacturing operation has been completed out-of-sequence.

17. The method of claim 15, further comprising:

simulating, analyzing, and intervening with operations in the manufacturing area based on the extracted vision data.

18. The method of claim 17, wherein:

viewing the manufacturing area with a vision system, including using vision sensors to view aircraft components within the manufacturing area, further includes viewing and recording images in real time; and

generating a 3-D simulation of operations in the manufacturing area based on the extracted vision data, further comprises generating the 3-D simulation in real time measurement feedback.

19. The method of claim 15, wherein using vision sensors to view aircraft components within the manufacturing area includes directing the vision sensors to view and record images of the aircraft components, wherein directing the vision sensors comprises directing the digital cameras to point to the aircraft components through one of panning, tilting, or zooming, wherein the aircraft components comprise at least one of:

parts,

vehicles,

machines,

assembly equipment,

tools, and

an inventory of consumables.

20. The method of claim 15, further comprising:

selecting and locating the aircraft components within the manufacturing area to form selected aircraft components, and

wherein using vision sensors to view the selected aircraft components within the manufacturing area and record images of the viewed selected aircraft components comprises directing the digital cameras each having pan, tilt and zoom functions to point to the selected aircraft components through one of panning, tilting, or zooming.

21. The method of claim 15, wherein the manufacturing area is an aircraft manufacturing environment.

22. A computer implemented method comprising:

directing a vision system having a digital camera to view aircraft components in an aircraft manufacturing area using pan, tilt, or zoom functions of the camera, wherein the aircraft components comprise a first aircraft component and a second aircraft component that is different than the first aircraft component;

recording images of the first aircraft component;

extracting vision data from the recorded images of the first aircraft component using a processor to create extracted vision data for the first aircraft component;

using the extracted vision data for the first aircraft component to perform a safety function, wherein the safety function predicts a collision between vehicles within the manufacturing area;

directing the vision system to view a second aircraft component in a different field of view using the pan, tilt, or zoom functions of the digital camera;

recording images of the second aircraft component;

extracting vision data from the recorded images of the second aircraft component using the processor to create extracted vision data for the second aircraft component; and

using the extracted vision data for the second aircraft component to perform a logistics function to track the second aircraft component against an expected location and time in the aircraft manufacturing area.