TRAFFIC CAMERA CALIBRATION UPDATE UTILIZING SCENE ANALYSIS

• US 20140313347A1
• Filed: 04/23/2013
• Published: 10/23/2014
• Est. Priority Date: 04/23/2013
• Status: Active Grant

• Current Assignee: Conduent Business Services, LLC (Conduent, Inc.)
• Original Assignee: Xerox Corporation (Xerox Holdings Corp.)
• Inventors: Wu, Wencheng, Dalal, Edul N., Bala, Raja
• Granted Patent: US 9,185,402 B2
• Time in Patent Office: Days
• US Class Current: 348/187
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30236 Traffic on road, railway or crossing
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 17/002 for television cameras

Abstract

Methods and systems are disclosed for updating camera geometric calibration utilizing scene analysis. Geometric calibration parameters can be derived with respect to one or more cameras and selected reference points of interest identified from a scene acquired by one or more of such cameras. The camera geometric calibration parameters can be applied to image coordinates of the selected reference points of interest to provide real-world coordinates at a time of initial calibration of the camera(s). A subset of a video stream from the camera(s) can then be analyzed to identify features of a current scene captured by the camera(s) that match the selected reference points of interest and provide a current update of the camera geometric calibration parameters with respect to the current scene.

Claims

1. A method for updating camera geometric calibration utilizing scene analysis, said method comprising:

receiving initial camera geometric calibration parameters with respect to at least one camera;

discovering reference points of interest from an initial reference frame in a scene acquired by said at least one camera;

applying said camera geometric calibration parameters to image coordinates of said discovered reference points of interest to provide real-world coordinates of said discovered reference points of interest;

analyzing a subset of a video stream from said at least one camera to identify feature points of a current scene captured by said at least one camera that match said discovered reference points of interest; and

providing a current update of said camera geometric calibration parameters with respect to said current scene.

2. The method of claim 1 further comprising measuring changes between said current scene and said scene acquired for deriving initial camera calibration of said at least one camera, utilizing at least one of:

a difference between image pixel coordinates of said identified feature points of a current scene and image pixel coordinates of said discovered reference points of interest;

a difference between real-world coordinates of said identified feature points of a current scene and real-world coordinates of said discovered reference points of interest; and

a difference between the camera calibration parameters derived from the image pixel coordinates and real-world coordinates of identified feature points of a current scene, and said initial camera calibration parameters, to generate assessment data for triggering at least one of recalibration, maintenance, and diagnostics of said at least one camera.

3. The method of claim 2 wherein said real-world coordinates are determined by application of said initial camera geometric calibration parameters to said image pixel coordinates.

4. The method of claim 1 further comprising storing said image pixel coordinates and said real-world coordinates of said discovered reference points for future online calibration updates of said at least one camera.

5. The method of claim 1 further comprising:

identifying said feature points of a current scene captured by said at least one camera, which match said discovered reference points of interest, utilizing at least one of the following techniques;

SIFT matching, SURF matching, cross-correlation matching, or matching objects of interest.

6. The method of claim 1 wherein said discovered reference points of interest are discovered utilizing at least one of the following detection techniques:

SIFT, SURF, Harris corner, Shi-Tomasi corner, object of interest detection, RANSAC, and manual specification using a graphical user interface.

7. The method of claim 1 wherein said providing a current update of said camera geometric calibration parameters is performed utilizing at least one of the following updating techniques:

a rebuild of camera calibration based on the image pixel coordinates of said identified feature points identified by analyzing one or more current frames and the real-world coordinates of said discovered reference points stored in said database;

a weighted combination of the initial camera calibration and said rebuilt camera calibration; and

a modification of initial camera calibration using extended Kalman filtering techniques using the observed image pixel coordinates of said identified feature points identified by analyzing one or more current frames.

8. The method of claim 6 wherein said discovered reference points are selected such that said discovered reference points lie on the plane of the road.

9. A system for updating camera geometric calibration utilizing scene analysis, said system comprising:

receiving camera geometric calibration parameters with respect to at least one camera;

discovering reference points of interest from a scene acquired by said at least one camera;

applying said camera geometric calibration parameters to image coordinates of said discovered reference points of interest to provide real-world coordinates of said discovered reference points of interest;

analyzing a subset of a video stream from said at least one camera to identify feature points of a current scene captured by said at least one camera that match said discovered reference points of interest; and

providing a current update of said camera geometric calibration parameters with respect to said current scene.

10. The system of claim 9 wherein said instructions are further configured for measuring changes between said current scene and said scene acquired for deriving initial camera calibration of said at least one camera, utilizing at least one of:

a difference between image pixel coordinates of said identified feature points of a current scene and image pixel coordinates of said discovered reference points of interest;

a difference between rear-world coordinates of said identified feature points of a current scene and real-world coordinates of said discovered reference points of interest; and

a difference between the camera calibration parameters derived from the image pixel coordinates and real-world coordinates of identified feature points of a current scene and said initial camera calibration parameters,

to generate assessment data for triggering at least one of recalibration, maintenance, and diagnostics of said at least one camera.

11. The system of claim 9 wherein said real-world coordinates are determined by application of said initial camera geometric calibration to image pixel coordinates.

12. The system of claim 9 wherein said instructions are further configured for storing said image pixel coordinates and said real-world coordinates of said discovered reference points for future online calibration updates of said at least one camera.

13. The system of claim 9 wherein said instructions for identifying said feature points of a current scene further comprise instructions configured for:

identifying feature points of a current scene captured by said at least one camera that match said discovered reference points of interest utilizing at least one of the following techniques;

SIFT matching, SURF matching, cross-correlation matching, or matching objects of interest.

14. The system of claim 9 wherein said instructions are further configured for discovering said discovered reference points of interest utilizing at least one of the following detection techniques:

SIFT, SURF, Harris corner, Shi-Tomasi corner, object of interest detection RANSAC, and manual specification using a graphical user interface.

15. The system of claim 9 wherein said instructions configured for providing a current update of said camera geometric calibration parameters are performed utilizing at least one of the following updating techniques:

a rebuild of camera calibration based on the image pixel coordinates of said identified feature points identified by analyzing one or more current frames and the real-world coordinates of said discovered reference points stored in said database;

a weighted combination of the initial camera calibration and said rebuilt camera calibration; and

a modification of initial camera calibration using extended Kalman filtering techniques using the observed image pixel coordinates of said identified feature points identified by analyzing one or more current frames.

16. A traffic analysis and monitoring system, said system comprising:

a traffic camera, wherein a geometric calibration function relates image pixel coordinates of said traffic camera to real-world coordinates;

a database that stores a reference data set composed of real-world coordinates of discovered reference points present in a field of view of said traffic camera;

a feature point identification module that communicates with said database and which identifies image pixel coordinates of feature points of a new frame that match said discovered reference points, as imaged by said traffic camera;

a calibration testing module that communicates with said reference point identification module and said database and which calculates at least one of;

a difference between image pixel coordinates of said feature points in the new frame, identified by said feature point identification module, and image pixel coordinates of said discovered reference points stored in said database; and

a difference between real-world coordinates of said discovered reference points taken from said reference data set and real-world coordinates of corresponding said feature points in the new frame calculated from said geometric calibration function applied to image pixel coordinates of said feature points in the new frame, determined by said feature point identification module; and

a calibration updating module that communicates with said calibration testing module, said feature point identification module and said database, and which updates said geometric calibration function if said difference calculated by said calibration testing module is greater than a predefined threshold.

17. The system of claim 16 wherein said feature point identification module identifies data based on said new frame utilizing at least one of the following techniques:

SIFT matching, SURF matching, cross-correlation matching, or matching objects of interest.

18. The system of claim 16 wherein said discovered reference points are discovered utilizing at least one of the following detection techniques:

SIFT, SURF, Harris corner, Shi-Tomasi corner, object of interest detection, RANSAC, and manual specification using a graphical user interface.

19. The system of claim 16 wherein said updating said geometric calibration function is performed utilizing at least one of the following updating techniques:

a rebuild of camera calibration based on the image pixel coordinates of said feature points identified by said feature point identification module on one or more current frames and the real-world coordinates of said discovered reference points stored in said database;

a weighted combination of the initial camera calibration and the rebuilt camera calibration; and

a modification of initial camera calibration using extended Kalman filtering techniques using the observed image pixel coordinates of said feature points identified by said feature point identification module on one or more current frames.

20. The system of claim 18 wherein said discovered reference points are selected such that said discovered reference points lie on the plane of the road.