IMAGE-BASED FEEDBACK FOR ASSEMBLY INSTRUCTIONS

• US 20170352282A1
• Filed: 06/03/2016
• Published: 12/07/2017
• Est. Priority Date: 06/03/2016
• Status: Abandoned Application

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Anderson, Evelyn R., Bender, Michael, Childress, Rhonda L.
• Application Number:
  US15/172,235
• Publication Number:
  US 20170352282A1
• Time in Patent Office: Days
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G09B 5/02 with visual presentation of the material to be studied, e.g. using film strip
• CPC Class Codes: H04N 23/10 for generating image signals from different wavelengths
• CPC Class Codes: H04N 25/60 Noise processing, e.g. detecting, correcting, reducing or removing noise
• CPC Class Codes: H04N 25/61 the noise originating only from the lens unit, e.g. flare, shading, vignetting or "cos4"

Abstract

A set of images of a product are obtained from a camera. The set of images are of a plurality of parts of the product being assembled. Based on the set of obtained images, an assembly step that is being performed is detected. One or more assembly metrics for assembling the product are retrieved. A potential assembly issue is determined. The determination is based on the detected assembly step and also based on the retrieved assembly metrics. An assistance feedback is provided based on the determined potential assembly issue.

Claims

1. A method for aiding assembly of a product, the product comprising a plurality of parts, the method comprising:

obtaining a set of images of the product as the plurality of parts are being assembled, the set of images obtained from at least one camera;

detecting, based on the set of obtained images, an assembly step being performed;

retrieving one or more assembly metrics for assembling the product;

determining, based on the detected assembly step and based on the retrieved assembly metrics, a potential assembly issue; and

providing, based on the determined potential assembly issue, an assistance feedback.

2. The method of claim 1, wherein the set of images are a second set of images, and wherein the assembly step is a second assembly step and wherein the determining is further based on a first assembly step, the method further comprises:

obtaining a first set of images of the product as the plurality of parts are being assembled, the first set of images obtained before the second set of images are obtained; and

detecting, based on the first set of obtained images, a first assembly being completed, the first assembly step completed before the second assembly step.

3. The method of claim 2, wherein the one or more assembly metrics include the average time to perform the second assembly step, and wherein the potential assembly issue is performing the second assembly step in a time longer than the average time.

4. The method of claim 2, wherein the one or more assembly metrics indicate that the second assembly step should be completed before performing the first assembly step, and wherein the potential assembly issue is an out-of-order assembly.

5. The method of claim 1, wherein the one or more assembly metrics indicate the orientation of a first part in relation to a second part of the plurality of parts, and wherein the determined potential assembly issue is an improper orientation of the first part.

6. The method of claim 1, wherein the camera is communicatively coupled to a computing device.

7. The method of claim 6, wherein the computing device is selected from the group consisting of a smartphone, a headset, and a tablet.

8. The method of claim 6, wherein the computing device provides the assistance feedback.

9. The method of claim 1, wherein the set of images are obtained from at least two cameras.

10. The method of claim 1, wherein the assistance feedback is additional instructions.

11. The method of claim 1, wherein the assistance feedback is a link to a community-based online resource.

12. The method of claim 1, wherein the assistance feedback is an audio-visual demonstration.

13. The method of claim 1, wherein the assistance feedback is a communication session with an experienced assembler of the product.

14. The method of claim 1, wherein the method further comprises:

updating, based on the determined potential assembly issue and based on the detected assembly step, an assembly data store;

identifying, based on the determined potential assembly issue and based on the updated assembly data store, a deficient assembly instruction; and

performing, based on the identified deficient assembly instruction, a corrective action.

15. The method of claim 14, wherein the corrective action is updating the assembly instructions for the assembly step.

16. The method of claim 14, wherein the corrective action is notifying an assembly instruction preparer regarding the identified deficient assembly instruction.

17. A computing device for aiding assembly of a product, the product comprising a plurality of parts, the computing device comprising:

a memory;

a camera, the camera for capturing one or more images; and

a processor, the processor in communication with the memory and in communication with the camera, wherein the processor is configured to perform a method comprising;

obtaining, from the camera, a set of images of the product as the plurality of parts are being assembled;

detecting, based on the set of obtained images, an assembly step being performed;

retrieving one or more assembly metrics for assembling the product;

determining, based on the detected assembly step and based on the retrieved assembly metrics, a potential assembly issue; and

providing, based on the determined potential assembly issue, an assistance feedback.

18. The computing device of claim 17, wherein the assistance feedback is an audio-visual demonstration.

19. A computer program product for aiding assembly of a product, the product comprising a plurality of parts, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a computer to perform a method comprising:

obtaining a set of images of the product as the plurality of parts are being assembled, the set of images obtained from at least one camera;

detecting, based on the set of obtained images, an assembly step being performed;

retrieving one or more assembly metrics for assembling the product;

determining, based on the detected assembly step and based on the retrieved assembly metrics, a potential assembly issue; and

providing, based on the determined potential assembly issue, an assistance feedback.

20. The computer program product of claim 19, wherein the one or more assembly metrics indicate the orientation of a first part in relation to a second part of the plurality of parts, and wherein the determined potential assembly issue is an improper orientation of the first part.