CAMERA RECALIBRATION SYSTEM AND THE METHOD THEREOF

• US 20120154604A1
• Filed: 09/23/2011
• Published: 06/21/2012
• Est. Priority Date: 12/17/2010
• Status: Abandoned Application

• Current Assignee: Industrial Technology Research Institute
• Original Assignee: Industrial Technology Research Institute
• Inventors: Chen, Jian-Ren, Chang, Leii H., Lee, Ho-Hsin, Luo, Yi-Fei, Kang, Chung-Chia
• Application Number:
  US13/242,268
• Publication Number:
  US 20120154604A1
• Time in Patent Office: Days
• US Class Current: 348/187
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
• CPC Class Codes: H04N 17/002 for television cameras

Abstract

The invention discloses a camera recalibration system and the method thereof. The camera recalibration system includes a first camera, which is to be recalibrated, for capturing image; an image processing unit comprising a storage unit for storing a first image and a second image, the second image being captured by the first camera; and a computing unit for measuring camera motion from the first image to the second image and computing calibration information corresponding to the camera motion; and a display unit for presenting the calibration information.

Claims

1. A camera recalibration system comprising:

a first camera, which is to be recalibrated, for capturing image;

an image processing unit comprising

a storage unit for storing at least two images, a first image and a second image, the first image being as a reference image for recalibration, the second image being captured by the first camera; and

a computing unit for measuring a camera motion from the first image to the second image and computing calibration information corresponding to the camera motion; and

a display unit for presenting the calibration information.

2. The camera recalibration system of claim 1, wherein the image processing unit is selected from a group consisting of a PDA, an MID, a smart phone, a laptop computer, and a portable multimedia device.

3. The camera recalibration system of claim 1, wherein the calibration information is attached to the second image.

4. The camera recalibration system of claim 1, wherein the first image is selected from an image captured by the first camera being originally set up, an image captured by a second camera being originally set up, and an image at a predetermined location captured by any camera.

5. The camera recalibration system of claim 1, wherein the display unit further displays a real-time image captured by the first camera.

6. The camera recalibration system of claim 1, wherein the camera motion comprises a motion of camera roll, a scaling factor, or horizontal and vertical motions between the first and second images.

7. The camera recalibration system of claim 6, wherein the motion of camera roll is measured by a central tendency of a data set which is composed of motion of camera yaw and camera pitch between a plurality of first feature vectors and their paired second feature vectors, wherein each first feature vector is formed by connecting a feature point to another feature point in the first image, each second feature vector is formed by connecting a feature point to another feature point in the second image, and each feature point extracted from the first image corresponds to each feature point extracted from the second image.

8. The camera recalibration system of claim 6, wherein the scaling factor is measured by a central tendency of a data set which is composed of ratios of length of a plurality of the first feature vectors to their paired second feature vectors, wherein each first feature vector is formed by connecting a feature point to another feature point in the first image, each second feature vector is formed by connecting a feature point to another feature point in the second image, and each feature point extracted from the first image corresponds to each feature point extracted from the second image.

9. The camera recalibration system of claim 6, wherein the first image is transformed into a third image according to the motion of camera roll and the scaling factor of the second image, the horizontal motion is measured by a central tendency of a data set which is composed of horizontal movements between feature points of the third image and their matched feature points of the second image, and the vertical motion is measured by a central tendency of a data set which is composed of vertical movements between feature points of the third image and their matched feature points of the second image.

10. The camera recalibration system of claim 6, wherein the second image is transformed into a fourth image according to the motion of camera roll and the scaling factor of the first image, the horizontal motion is measured by a central tendency of a data set which is composed of horizontal movements between feature points of the fourth image and their matched feature points of the first image, and the vertical motion is measured by a central tendency of a data set which is composed of vertical movements between feature points of the fourth image and their matched feature points of the first image.

11. The camera recalibration system of claim 1, wherein the calibration information comprises a prompt sign, a prompt sound, or a prompt frequency.

12. The camera recalibration system of claim 11, wherein the prompt sign comprises a linear arrow, wherein length of the linear arrow indicates the magnitude by which the first camera is required to be calibrated, and arrowhead of the linear arrow indicates the direction by which the first camera is required to be calibrated.

13. The camera recalibration system of claim 11, wherein the prompt sign comprises an arced arrow, wherein length of the arced arrow indicates the magnitude by which the first camera is required to be calibrated, and arrowhead of the arced arrow indicates the direction by which the first camera is required to be calibrated.

14. The camera recalibration system of claim 11, wherein the prompt sign comprises a scaling sign, wherein a plus sign in an icon of the scaling sign indicates that the first camera needs to perform a zoom-in operation, while a minus sign indicates that the first camera needs to perform a zoom-out operation.

15. The camera recalibration system of claim 1, further comprising a control unit coupled to the image processing unit so as to provide a warning signal when the measured camera motion of the first camera satisfies a predetermined condition.

16. The camera recalibration system of claim 15, wherein the control unit is operable to perform transformation of image coordinate so as to transform the coordinate system of the second image to that of its original setting, if the control unit is in an operational mode of auto-adjustment and the measured camera motion of the first camera does not satisfy the predetermined condition.

17. A method for recalibrating a first camera which is to be recalibrated, the method comprising the steps of:

providing a first image;

capturing a second image by using a first camera;

measuring a camera motion between the first and second images and computing calibration information corresponding to the camera motion; and

displaying the calibration information in the second image.

18. The method of claim 17, wherein the first image is selected from an image captured by the first camera being originally setup, an image captured by a second camera being originally setup, or an image at a predetermined location captured by any camera.

19. The method of claim 17, further comprising the step of:

displaying a real-time image captured by the first camera.

20. The method of claim 17, wherein the step of measuring the camera motion comprises the steps of:

extracting local feature points from the first and second images;

matching the feature points of the first image to those of the second image;

forming a set of first feature vectors and a set of their paired second feature vectors, respectively, wherein each first feature vector is formed by connecting a feature point to another feature point in the first image, and each second feature vector is formed by connecting the corresponding matched feature points in the second image; and

measuring the camera motion including a motion of camera roll and a scaling factor according the sets of the first and second feature vectors.

21. The method of claim 20, wherein the motion of camera roll can be measured by a central tendency of a data set which is composed of motion of camera yaw and camera pitch between a plurality of first feature vectors and their paired second feature vectors.

22. The method of claim 20, wherein the scaling factor is a central tendency measure of a data set which is composed of ratios of length of a plurality of the first feature vectors to their paired second feature vectors.

23. The method of claim 20, wherein the step of measuring the camera motion further comprises the steps of:

transforming the first image into a third image according to the motion of camera roll and the scaling factor of the second image;

extracting feature points from the third image in correspondence with the feature points in the first and second images; and

measuring a central tendency of a data set which is composed of horizontal movements between feature points of the third image and their matched feature points of the second image as a horizontal motion, and measuring a central tendency of a data set which is composed of vertical movements between feature points of the third image and their matched feature points of the second image as a vertical motion.

24. The method of claim 20, wherein the step of measuring the camera motion further comprises the steps of:

transforming the second image into a fourth image according to the motion of camera roll and the scaling factor of the first image;

extracting feature points from the fourth image in correspondence with the feature points in the first and second images; and

measuring a central tendency of a data set which is composed of horizontal movements between feature points of the fourth image and their matched feature points of the first image as a horizontal motion, and measuring a central tendency of a data set which is composed of vertical movements between feature points of the fourth image and their matched feature points of the first image as a vertical motion.

25. The method of claim 17, wherein the calibration information comprises a prompt sign, a prompt sound, or a prompt frequency.

26. The method of claim 25, wherein the prompt sign comprises a linear arrow, wherein length of the linear arrow indicates the magnitude by which the first camera is required to be calibrated, and arrowhead of the linear arrow indicates the direction by which the first camera is required to be calibrated.

27. The method of claim 25, wherein the prompt sign comprises an arced arrow, wherein length of the arced arrow indicates the magnitude by which the first camera is required to be calibrated, and arrowhead of the arced arrow indicates the direction by which the first camera is required to be calibrated.

28. The method of claim 25, wherein the prompt sign comprises a scaling sign, wherein a plus sign in an icon of the scaling sign indicates that the first camera needs to perform a zoom-in operation, while a minus sign indicates that the first camera needs to perform a zoom-out operation.

29. The method of claim 17, further comprising the step of:

transmitting a warning signal when the camera motion satisfies a predetermined condition.

30. The method of claim 17, if the camera motion does not satisfy a predetermined condition, the method further comprising the steps of:

extracting local feature points from the first and second images, and matching the feature points of the first image to those of the second image;

transforming coordinates of the feature points of the first image from the coordinate system of the first image to that of the second image according to the measured camera motion, so as to perform coordinate transformation of the feature points;

measuring a spatial distance between each transformed feature point of the first image and its corresponding feature point of the second image, according to the coordinate system of the second image;

if the spatial distance exceeds a predetermined threshold, then regarding the feature point as a mismatched feature point and discarding it from the group of feature points; and

computing a transform matrix according to at least four matched feature points in the group.

31. The method of claim 17, if the camera motion does not satisfy a predetermined condition, the method further comprising the steps of:

extracting local feature points from the first and second images, and matching the feature points of the first image to those of the second image;

transforming coordinates of the feature points of the second image from the coordinate system of the second image to that of the first image according to the measured camera motion, so as to perform coordinate transformation of the feature points;

measuring a spatial distance between each transformed feature point of the second image and its corresponding feature point of the first image, according to the coordinate system of the first image;

if the spatial distance exceeds a predetermined threshold, then regarding the feature point as a mismatched feature point and discarding it from the group of feature points; and

computing a transform matrix according to at least four matched feature points in the group.

32. A computer program product containing at least one instruction, the at least one instruction for being downloaded to a computer system to perform the method of claim 17.

33. A computer readable medium containing a computer program, the computer program performing the method of claim 17 after being downloaded to a computer system.