SYSTEM AND METHOD FOR RUNTIME DETERMINATION OF CAMERA MISCALIBRATION

• US 20110157373A1
• Filed: 12/24/2009
• Published: 06/30/2011
• Est. Priority Date: 12/24/2009
• Status: Active Grant

• Current Assignee: Cognex Corporation
• Original Assignee: Cognex Corporation
• Inventors: Li, David Y., Shivaram, Guruprasad, Michael, David J., Ye, Xiangyun
• Granted Patent: US 11,699,247 B2
• Time in Patent Office: Days
• US Class Current: 348/187
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/30108 Industrial image inspection
• CPC Class Codes: G06T 7/85 Stereo camera calibration

Abstract

This invention provides a system and method for runtime determination (self-diagnosis) of camera miscalibration (accuracy), typically related to camera extrinsics, based on historical statistics of runtime alignment scores for objects acquired in the scene, which are defined based on matching of observed and expected image data of trained object models. This arrangement avoids a need to cease runtime operation of the vision system and/or stop the production line that is served by the vision system to diagnose if the system'"'"'s camera(s) remain calibrated. Under the assumption that objects or features inspected by the vision system over time are substantially the same, the vision system accumulates statistics of part alignment results and stores intermediate results to be used as indicator of current system accuracy. For multi-camera vision systems, cross validation is illustratively employed to identify individual problematic cameras. The system and method allows for faster, less-expensive and more-straightforward diagnosis of vision system failures related to deteriorating camera calibration.

Claims

1. A method for determining camera miscalibration in a system of at least three cameras, comprising the steps of:

a) calibrating the at least three cameras, including finding respective extrinsic calibration parameters for each of the at least three cameras;

b) finding a first feature in three-dimensional space with a first plurality of the at least three cameras and determining a first measurement of the first feature;

c) finding a second feature in three-dimensional space with a second plurality of the at least three cameras and determining a second measurement of the second feature; and

d) comparing the first measurement with the second measurement with respect to at least one of (i) an accuracy determined during step (a), (ii) a desired system accuracy, and (iii) a known property of the first feature and the second feature.

2. The method as set forth in claim 1 wherein the first feature is either one of (a) substantially the same as the second feature and (b) different than the second feature.

3. The method as set forth in claim 2 wherein the first measurement comprises an estimated first location of the first feature and the second measurement comprises an estimated second location of the second feature.

4. The method as set forth in claim 2 wherein the first measurement comprises a score of success in finding the first feature and the second measurement comprises a score of success in finding the second feature.

5. The method as set forth in claim 1 wherein step (d) includes computing a discrepancy between the first measurement and the second measurement and comparing the discrepancy with respect to at least one of (i) the accuracy determined during step (a), (ii) the desired system accuracy, and (iii) a known property of the first feature and the second feature.

6. The method as set forth in claim 5 wherein the known property includes a known distance between the first feature and the second feature, and the first plurality of at least three cameras includes a first grouping of at least two cameras and the second plurality of at least three cameras includes a second grouping of at least two cameras.

7. The method as set forth in claim 1 further comprising, in response to step (d) issuing a signal indicating recalibration is required based upon a result of the step of comparing exceeding at least one of the (i) accuracy determined during step (a) and (ii) the desired system accuracy.

8. The method as set forth in claim 7 wherein the step of issuing includes generating new extrinsic calibration parameters based upon step (d) and providing the extrinsic calibration parameters to at least one of the at least three cameras so as to recalibrate the at least one of the at least three cameras.

9. The method as set forth in claim 1 wherein the accuracy determined during step (a) includes a collection of values based upon calibration residual errors.

10. The method as set forth in claim 9 further comprising providing the new extrinsic parameters so as to recalibrate at least one of the at least three cameras in accordance with step (a).

11. The method as set forth in claim 1 wherein the system of at least three cameras includes a machine vision system inspection function so as to perform runtime machine vision inspection to objects that pass through a volume space viewed by the at least three cameras.

12. The method as set forth in claim 1 wherein the desired system accuracy is based upon historical values for each of the first measurement and the second measurement.

13. The method as set forth in claim 1 wherein the desired system accuracy is based upon a predetermined threshold value.

14. The method as set forth in claim 13 wherein the threshold value is defined based upon a desired accuracy of a runtime vision system task.

15. The method as set forth in claim 1 further comprising providing intrinsic parameters for at least one of the at least three cameras in step (a) and recalibrating the at least one of the at least three cameras based upon new intrinsic parameters.

16. The method as set forth in claim 1 wherein the known property includes a known distance between the first feature and the second feature, and the first plurality of at least three cameras includes a first grouping of at least two cameras and the second plurality of at least three cameras includes a second grouping of at least two cameras.

17. A method for determining camera miscalibration in a system of at least three cameras, comprising the steps of:

a) calibrating the at least three cameras, including finding respective extrinsic calibration parameters for each of the at least three cameras;

b) finding a first object pose in three-dimensional space with a first plurality of the at least three cameras and determining a first measurement of the first object pose;

c) finding a second object pose in three-dimensional space with a second plurality of the at least three cameras and determining a second measurement of the second object pose; and

d) comparing the first measurement with the second measurement with respect to at least one of (i) an accuracy determined during step (a), (ii) a desired system accuracy.

18. The method as set forth in claim 17 wherein the first measurement is a first pose score and the second measurement is a second pose score.

19. A system for determining camera miscalibration in a system of at least three cameras, comprising:

a) at least three cameras, each including respective extrinsic calibration parameters;

b) a first plurality of the at least three cameras that find a first feature in three-dimensional space and determine a first measurement of the first feature;

c) a second plurality of the at least three cameras that find a second feature in three-dimensional space and determine a second measurement of the second feature; and

d) a comparison process that compares the first measurement with the second measurement with respect to at least one of (i) an accuracy associated with the extrinsic calibration parameters, (ii) a desired system accuracy, and (iii) a known property of the first feature and the second feature.

20. The method as set forth in claim 19 wherein the known property includes a known distance between the first feature and the second feature, and the first plurality of at least three cameras includes a first grouping of at least two cameras and the second plurality of at least three cameras includes a second grouping of at least two cameras.

21. The method as set forth in claim 20 wherein the at least two cameras of the first grouping each differ from the at least two cameras of the second grouping.

22. A system for determining camera miscalibration in a system of at least three cameras, comprising of:

a) at least three cameras, each including respective extrinsic calibration parameters calibration parameters;

b) a first plurality of the at least three cameras that find a first object pose in three-dimensional space and determine a first measurement of the first object pose;

c) a second plurality of the at least three cameras that find a second object pose in three-dimensional space and determine a second measurement of the second object pose; and

d) a comparison process that compares the first measurement with the second measurement with respect to at least one of (i) an accuracy associated with the extrinsic calibration parameters and (ii) a desired system accuracy.