Guided inspection of an installed component using a handheld inspection device

• US 9,852,500 B2
• Filed: 07/15/2015
• Issued: 12/26/2017
• Est. Priority Date: 07/15/2015
• Status: Active Grant

• Current Assignee: GM Global Technology Operations LLC (General Motors Company)
• Original Assignee: GM Global Technology Operations LLC (General Motors Company)
• Inventors: Rourke, Rick F., Scheuerman, Robert J.
• Primary Examiner(s): Rahaman, Mohammed
• Assistant Examiner(s): Adrovel, William
• Application Number:
  US14/800,141
• Publication Number:
  US 20170018067A1
• Time in Patent Office: 895 Days
• Field of Search: None
• CPC Class Codes: G01B 11/022 by means of tv-camera scanning
• CPC Class Codes: G01B 11/24 for measuring contours or curvatures
• CPC Class Codes: G01B 11/2513 with several lines being projected in more than one direction, e.g. grids, patterns
• CPC Class Codes: G06F 16/00 Information retrieval; Database structures therefor; File system structures therefor
• CPC Class Codes: G06F 17/00 Digital computing or data processing equipment or methods, specially adapted for specific functions information retrieval, database structures or file system structures therefor G06F16/00
• CPC Class Codes: G06K 7/10881 constructional details of hand-held scanners
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 2207/30164 Workpiece; Machine component
• CPC Class Codes: G06T 2207/30204 Marker
• CPC Class Codes: G06T 7/0004 Industrial image inspection
• CPC Class Codes: G06T 7/0006 using a design-rule based approach
• CPC Class Codes: G06T 7/13 Edge detection
• CPC Class Codes: G06T 7/60 Analysis of geometric attributes
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 20/647 by matching two-dimensional images to three-dimensional objects
• CPC Class Codes: G06V 20/653 by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces
• CPC Class Codes: G06V 30/142 using hand-held instruments; Constructional details of the instruments
• CPC Class Codes: H04N 23/633 for displaying additional information relating to control or operation of the camera
• CPC Class Codes: H04N 23/64 : Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image

Abstract

A method for inspecting an installed component includes receiving an identity and selected location of the component as an input signal via a handheld inspection device having a controller, digital camera, and display screen, and collecting a dynamic pixel image of the selected location in real-time using the digital camera. The method includes displaying the image in real time via the display screen, projecting virtual guidance lines onto the image corresponding to edges of the installed component, and identifying the component via the controller when the image is aligned with the projected acquisition lines. A predetermined area of the installed component is identified after identifying the installed component, a predetermined feature dimension is measured within the identified predetermined area, and an output signal is generated with a status indicative of whether the measured feature dimension falls within a calibrated range.

Claims

1. A method for inspecting an installed component, the installed component including first and second mated pieces, the method comprising:

receiving an identity and selected location of the installed component as an input signal via a handheld inspection device, the handheld inspection device having a controller, a digital camera, and a display screen;

collecting a dynamic pixel image of the selected location in real-time using the digital camera;

displaying the collected dynamic pixel image in real-time via the display screen;

determining, via the controller from the identity and selected location, a set of virtual acquisition guidance lines calibrated to align with designated edges of the installed component in the displayed dynamic pixel image;

superimposing the virtual acquisition guidance lines onto the displayed dynamic pixel image via the display screen such that the virtual acquisition guidance lines correspond to the designated edges of the installed component within the selected location;

confirming the identity of the installed component in the selected location via the controller responsive to the edges of the installed component in the displayed dynamic pixel image aligning with the projected acquisition lines;

identifying a predetermined area of the installed component via the controller after confirming the identity of the installed component;

measuring, via the controller using image processing instructions, a predetermined feature dimension between the first and second mated pieces of the installed component within the identified predetermined area; and

generating an output signal via the controller with a status that is indicative of the first and second pieces being properly mated responsive to the measured feature dimension falling within a calibrated range.

2. The method of claim 1, further comprising:

activating or displaying a corresponding one of a plurality of status indicators on the handheld inspection device in response to confirming the identity of the installed component in the selected location, the predetermined area being identified, and/or the measured feature dimension being within the calibrated range.

3. The method of claim 2, wherein the handheld inspection device includes a case and a plurality of lights connected to the case, and wherein the each of the plurality of lights acts as a corresponding one of the plurality of status indicators.

4. The method of claim 2, wherein activating or displaying one of a plurality of status indicators includes displaying a status symbol on the display screen.

5. The method of claim 1, wherein collecting a dynamic pixel image via the digital camera includes using a charge coupled device or a liquid lens camera as the digital camera.

6. The method of claim 1, further comprising using the controller to automatically increase a pixel resolution of the digital camera after identifying the installed component in the selected location.

7. A handheld inspection device comprising:

a digital camera;

a display screen; and

a controller having a processor and image processing instructions, wherein the controller is programmed to execute the image processing instructions and thereby;

receive an identity and a selected location of an installed component as an input signal, the installed component including first and second mated pieces;

collect a dynamic pixel image of the selected location in real-time using the digital camera;

display the collected dynamic pixel image in real-time via the display screen;

determining, via the controller from the identity and selected location, a set of acquisition guidance lines calibrated to align with designated edges of the installed component in the displayed dynamic pixel image;

superimposing the acquisition guidance lines onto the displayed dynamic pixel image via the display screen such that the virtual acquisition guidance lines correspond to the designated edges of the installed component within the selected location;

activate a first indicator in response to the edges of the installed component within the displayed dynamic pixel image aligning with the projected acquisition lines;

activate a second indicator in response to a predetermined area of the installed component being identified;

measure, via the processor, a predetermined feature dimension between the first and second mated pieces of the installed component within the identified predetermined area; and

activate a third indicator and generate an output signal via the processor, wherein the third indicator has a status indicative of the first and second pieces being properly mated responsive to the measured feature dimension falling within a calibrated range.

8. The handheld inspection device of claim 7, further comprising a case housing the digital camera, the display screen, and the controller.

9. The handheld inspection device of claim 8, wherein the case includes a pair of handles.

10. The handheld inspection device of claim 7, further comprising a plurality of lights connected to the case, wherein each of the lights acts a corresponding one of the first, second and third indicators.

11. The handheld inspection device of claim 7, wherein each of the first, second and third indicators corresponds to a status symbol displayed on the display screen.

12. The handheld inspection device of claim 7, wherein the digital camera is a charge coupled device.

13. The handheld inspection device of claim 7, wherein the digital camera is a liquid lens camera.

14. The handheld inspection device of claim 7, wherein the controller is configured to automatically increase a pixel resolution of the digital camera after identifying the installed component in the selected location.

15. An electronic inspection system for inspecting installed components of a motor vehicle, the electronic inspection system comprising:

a digital camera configured to capture dynamic pixel images;

a user interface with an electronic display screen and an input device; and

a controller communicatively connected to the digital camera and the user interface, the controller being programmed to;

receive, from the input device of the user interface, a user input signal indicative of a component identity and location within the motor vehicle of an installed component, the installed component including first and second mated pieces;

receive, from the digital camera, signals of real-time dynamic pixel images of the installed component and the selected location within the motor vehicle;

direct the display screen to display in real-time the dynamic pixel images;

determine, from the received component identity and location, a set of acquisition guidance lines calibrated to align with designated edges of the installed component;

direct the display screen to display the acquisition guidance lines superimposed onto the installed component displayed in the dynamic pixel images;

responsive to the superimposed acquisition guidance lines aligning with the corresponding designated edges of the installed component within the selected location, confirm the component identity of the installed component;

determine, from the confirmed component identity, a target inspection area corresponding to the installed component;

measure a feature dimension between the first and second mated pieces of the installed component within the target inspection area; and

responsive to the measured feature dimension falling within a calibrated range, output a status indicator indicating that the first and second pieces are properly mated and the installed component is properly installed in the motor vehicle.

16. The electronic inspection system of claim 15, wherein the controller is further programmed to:

activate a first status indicator responsive to the acquisition guidance lines being superimposed onto and aligning with the corresponding designated edges of the installed component thereby confirming the component identity;

activate a second status indicator responsive to the target inspection area being identified and displayed via the display device; and

activate a third status indicator responsive to the measured feature dimension being within the calibrated range.

17. The electronic inspection system of claim 16, wherein activating each of the first, second and third status indicators includes displaying a respective status symbol on the display screen of the user interface.

18. The electronic inspection system of claim 15, further comprising a case, wherein the digital camera, the user interface, and the controller are mounted to the case.

19. The electronic inspection system of claim 15, wherein the display screen displaying the real-time dynamic pixel images includes:

first, displaying the real-time dynamic pixel images of the installed component at a first resolution to confirm the component identity; and

, second, displaying the real-time dynamic pixel images of the installed component at a second resolution, higher than the first resolution, to measure the feature dimension.

20. The electronic inspection system of claim 15, wherein the feature dimension includes a gap of designated dimensions separating the first and second mated pieces, and wherein the measuring includes evaluating a size of the gap of the installed component displayed in the dynamic pixel images.