System and method for validating camera calibration in a vision system

• US 9,734,419 B1
• Filed: 12/30/2008
• Issued: 08/15/2017
• Est. Priority Date: 12/30/2008
• Status: Active Grant

• Current Assignee: Cognex Corporation
• Original Assignee: Cognex Corporation
• Inventors: Ye, Xiangyun, Wallack, Aaron S., Shivaram, Guruprasad, Li, David Y., Marrion, Cyril C.
• Primary Examiner(s): Sosanya, Obafemi
• Application Number:
  US12/346,773
• Time in Patent Office: 3,150 Days
• Field of Search: 348135
• CPC Class Codes: G06T 7/00 Image analysis
• CPC Class Codes: G06T 7/85 Stereo camera calibration

Abstract

This invention provides a system and method to validate the accuracy of camera calibration in a single or multiple-camera embodiment, utilizing either 2D cameras or 3D imaging sensors. It relies upon an initial calibration process that generates and stores camera calibration parameters and residual statistics based upon images of a first calibration object. A subsequent validation process (a) acquires images of the first calibration object or a second calibration object having a known pattern and dimensions; (b) extracts features of the images of the first calibration object or the second calibration object; (c) predicts positions expected of features of the first calibration object or the second calibration object using the camera calibration parameters; and (d) computes a set of discrepancies between positions of the extracted features and the predicted positions of the features. The validation process then uses the computed set of discrepancies in a decision process that determines whether at least one of the discrepancies exceeds a predetermined threshold value. If so, recalibration is required. Where multiple cameras are employed, extrinsic parameters are determined and used in conjunction with the intrinsics.

Claims

1. A system for validating calibration of at least one camera in a vision system that images a volume space within a field of view thereof operatively connected to a processor comprising:

a calibration process performed during vision system set up that generates and stores camera calibration parameters and result statistics based upon calibration images of a first calibration object, the calibration images being captured by the at least one camera during the vision system set up, the first calibration object having a known pattern and dimensions acquired at at least one position within the volume space, the calibration process computing, during the vision system set up, a set of calibration discrepancies between positions of extracted calibration features from the calibration images within the field of view and predicted positions expected of the known pattern of the first calibration object;

a validation process that, during runtime system operation and subsequent to the calibration process, based on the camera calibration parameters and the result statistics,

(a) acquires at least one validation image of the first calibration object or a second calibration object having a known pattern and dimensions, (b) extracts validation features of the at least one validation image of the first calibration object or the second calibration object, (c) predicts positions expected of validation features of the first calibration object or the second calibration object using the camera calibration parameters from the calibration images acquired during the calibration process, and (d) computes a set of validation discrepancies between positions of the extracted validation features and the predicted positions of the validation features; and

wherein the validation process is constructed and arranged to determine whether at least one of the set of calibration discrepancies or at least one of the set of validation discrepancies exceeds an acceptance threshold associated therewith.

2. The system as set forth in claim 1 further comprising a discrepancy comparison process that (a) compares the set of discrepancies computed by the validation process to the set of discrepancies computed by the calibration process, and (b) based upon the comparison, if a predetermined number of the discrepancies of the set of discrepancies computed in the validation process differ in excess of the acceptance threshold from discrepancies of the set of discrepancies computed in the calibration process, indicates a recalibration requirement.

3. The system as set forth in claim 1 further comprising a plurality of cameras that image the volume space within a field of view thereof operatively connected to the processor, and wherein the calibration process further generates and stores extrinsic parameters and result statistics based upon images of the first calibration object and the validation process predicts positions expected of features of the first calibration object or the second calibration object further based upon camera calibration parameters.

4. The system as set forth in claim 3 wherein at least some of the cameras are constructed and arranged to generate 3D images.

5. The system as set forth in claim 4 wherein the at least some of the cameras are arranged as a stereo camera head.

6. The system as set forth in claim 3 wherein at least substeps (a), (b), (c) and (d) of the validation process are performed on the acquired images of the first calibration object or the second calibration object in predetermined subsets of the plurality of cameras.

7. The system as set forth in claim 1 wherein the first calibration object is substantially identical to the second calibration object.

8. The system as set forth in claim 1 wherein the acceptance threshold value is computed based upon statistics generated in the calibration process.

9. The system as set forth in claim 1 wherein the acceptance threshold value is defined based upon application requirements of a runtime vision process of the system.

10. The system as set forth in claim 1 wherein the set of discrepancies is computed based upon image triangulation.

11. The system as set forth in claim 1 wherein the set of discrepancies is computed based upon three-dimensional positions of the extracted features and three-dimensional positions of the expected features.

12. A method for validating calibration of at least one camera in a vision system that images a volume space within a field of view thereof operatively connected to a processor comprising:

(a) generating and storing, in a calibration process performed during vision system setup, calibration parameters and result statistics based upon calibration images of a first calibration object, the calibration images being captured by the at least one camera during the vision system setup, the first calibration object having a known pattern and dimensions acquired at at least one position within the volume space, the calibration process computing, during the vision system setup, a set of calibration discrepancies between positions of extracted calibration features from the calibration images within the field of view and the predicted positions expected of the known pattern of the first calibration object;

(b) in a validation process, performed during runtime system operation and subsequent to the calibration process, acquiring at least one validation image of either the first calibration object or a second calibration object having a known pattern and dimensions;

(c) extracting validation features of the at least one validation image of the first calibration object or the second calibration object;

(d) predicting positions expected of validation features of the first calibration object or the second calibration object using the camera calibration parameters from the calibration images acquired during the calibration process;

(e) computing a set of validation discrepancies between positions of the extracted validation features and the predicted positions of the validation features; and

(f) determining whether at least one of the set of calibration discrepancies or at least one of the set of validation discrepancies exceeds an acceptance threshold associated therewith.

13. The method as set forth in claim 12 further comprising (a) comparing the set of discrepancies computed by the validation process to the set of discrepancies computed by the calibration process, and (b) based upon the comparison, if a predetermined number of the discrepancies of the set of discrepancies computed in the validation process differ in excess of the acceptance threshold from discrepancies of the set of discrepancies computed in the calibration process, indicating a recalibration requirement.

14. The method as set forth in claim 12 further comprising providing a plurality of cameras that image the volume space within a field of view thereof, and in the calibration process, further generating and storing extrinsic parameters and result statistics based upon images of the first calibration object and the validation process predicts positions expected of features of the first calibration object or the second calibration object further based upon camera calibration parameters.

15. The method as set forth in claim 14 wherein, in the validation process, the steps of acquiring, extracting, predicting and determining are each performed on the acquired images of the first calibration object or the second calibration object in predetermined subsets of the plurality of cameras.

16. The method as set forth in claim 12 further comprising defining the acceptance threshold value based upon statistics generated in the calibration process.

17. The method as set forth in claim 12 further comprising defining the acceptance threshold value based upon application requirements of a runtime vision process of the system.

18. The method as set forth in claim 12 further comprising computing the set of discrepancies based upon image triangulation.

19. The method as set forth in claim 12 further comprising computing the set of discrepancies based upon three-dimensional positions of the extracted features and three-dimensional positions of the expected features.

20. A system for validating calibration of at least one camera in a vision system that images a volume space within a field of view thereof operatively connected to a vision processor comprising:

a calibration process that, during vision system setup, generates and stores (i) original intrinsic parameters based upon calibration images of a first calibration object having a known pattern and dimensions acquired at a plurality of positions within the volume space within the field of view and (ii) original feature correspondences with respect to the calibration images of the first calibration object;

a validation process that, during runtime system operation and subsequent to the calibration process,

(a) acquires at least one validation image of either the first calibration object or a second calibration object the same as, or with a pattern and dimensions substantially identical to, the first calibration object (b) extracts validation features of the at least one validation image of the first calibration object or the second calibration object and determines new feature correspondence between extracted validation features and known features of the first calibration object or the second calibration object, (c) generates and stores new intrinsic parameters based upon the stored original calibration feature correspondences from the calibration images acquired by the at least one camera during the calibration process and the new feature correspondences, and (d) computes a set of discrepancies with respect to the new intrinsic parameters relative to the original intrinsic parameters; and

wherein the validation process is constructed and arranged to determine whether at least one of the discrepancies exceeds a predetermined value associated therewith, and if the at least one of the discrepancies exceeds the predetermined threshold to indicate a recalibration requirement.

21. The system as set forth in claim 20 further comprising a plurality of cameras that image the volume space within a field of view thereof operatively connected to the vision processor, and wherein the calibration process further generates and stores original extrinsic parameters based upon images of a first calibration object and the validation process further generates and stores new extrinsic parameters based upon the stored original feature correspondences and the new feature correspondences, and computes the set of discrepancies further with respect to the new extrinsic parameters relative to the original extrinsic parameters.

22. A method for validating calibration of at least one camera in a vision system that images a scene within volume space within a field of view thereof operatively connected to a processor comprising:

(a) in a calibration process performed during vision system setup by providing a calibration image and associated calibration parameters and result statistics including calibration discrepancies between positions of extracted predetermined features from the calibration images and the predicted positions expected of the features within the field of view and associated with calibration using a calibration object, the calibration object being located in the scene at a known and repeatable position in the scene, the calibration image being captured by the at least one camera during the vision system setup; and

(b) in a validation process performed during runtime system operation and subsequent to the calibration process,

(i) acquiring a validation image of the calibration object at the known and repeatable position,

(ii) computing, from the validation image, image positions based upon the calibration parameters from the calibration images acquired during the calibration process and validation discrepancies of the predetermined features of the calibration object, and

(iii) determining whether the validation discrepancies remain within a predetermined acceptance threshold with respect to the calibration discrepancies.

23. The method as set forth in claim 22 wherein the calibration object is fixedly located within the scene at a location that, during a runtime vision process could be partially occluded from view by the at least one camera by an object in the scene being analyzed by the vision process and that is unoccluded when the object is absent from the scene.

24. The method as set forth in claim 23 wherein the validation process occurs in an ongoing basis at predetermined intervals in which the object is absent from the scene.

25. The method as set forth in claim 24 wherein the calibration object comprises fixed environmental elements in the scene.