Camera modeling system

• US 10,348,961 B2
• Filed: 03/30/2015
• Issued: 07/09/2019
• Est. Priority Date: 04/03/2014
• Status: Active Grant

• Current Assignee: Hanwha Vision Company Limited (Hanwha Corporation)
• Original Assignee: Hanwha Precision Machinery Co., Ltd. (Hanwha Corporation)
• Inventors: Bae, Soonmin, Hong, Hana
• Primary Examiner(s): Hansell, Jr., Richard A
• Application Number:
  US14/672,353
• Publication Number:
  US 20150288878A1
• Time in Patent Office: 1,562 Days
• Field of Search: 348169
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2207/30244 Camera pose
• CPC Class Codes: G06T 7/73 using feature-based methods
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof

Abstract

A camera modeling system. The camera modeling system includes: a vanishing point detecting module configured to detect, from at least one reference detection object existing in a detection region, a first vanishing point and a second vanishing point; a view mode determining module configured to determine a camera view mode on the basis of a distance from a first position in the detection region to the first vanishing point and a distance from the first position to the second vanishing point; and a camera modeling module configured to select a camera modeling mode corresponding to the determined camera view mode and perform camera modeling through the selected camera modeling mode.

Claims

1. A camera calibration system comprising at least one processor which implements:

a vanishing point detecting module configured to detect, from at least one reference detection object existing in a detection region, a first vanishing point and a second vanishing point;

a view mode determining module configured to determine a camera view mode on the basis of a result of comparing a distance from a position in the detection region to the first vanishing point with a distance from the position to the second vanishing point, by determining the camera view mode, among a plurality of predetermined camera view modes corresponding to a plurality of predetermined camera calibration methods, that corresponds to the result of the comparing; and

a camera calibration module configured to select a camera calibration method corresponding to the determined camera view mode among the plurality of predetermined camera calibration methods, and perform camera calibration through the selected camera calibration method,

wherein the determined camera view mode corresponds to an angle between a photographing direction of a camera capturing the detection region and a horizontal plane, the angle being determined on the basis of the result of the comparing.

2. The camera calibration system of claim 1, wherein the vanishing point detecting module is configured to detect the first and second vanishing points from a plurality of reference detection objects and is further configured to surround each reference detection object with a respective box;

the first vanishing point is derived from intersection points of straight lines connecting correspondent points of at least one pair of boxes among the boxes; and

the second vanishing point is an intersection point of straight lines connecting tops and bottoms of the boxes respectively.

3. The camera calibration system of claim 1, wherein the position is a center of the detection region.

4. The camera calibration system of claim 1, wherein the view mode determining module is further configured to determine the camera view mode according to a difference between the distance from the position to the first vanishing point and the distance from the position to the second vanishing point.

5. The camera calibration system of claim 1, wherein the plurality of predetermined camera view modes comprises a side-view mode, a top-view mode, and a uniform-view mode, the uniform-view mode being a mode between the side-view mode and the top-view mode with respect to the determined angle.

6. The camera calibration system of claim 5, wherein the camera calibration module is further configured to:

perform the camera calibration by selecting an adaptive parameter determination mode as the camera calibration method, when the camera view mode is the side-view mode;

perform the camera calibration by selecting a direct parameter determination mode as the camera calibration method, when the camera view mode is the top-view mode; and

perform the camera calibration by selecting one of the adaptive parameter determination mode, the direct parameter determination mode, or a combination of the adaptive parameter determination mode and the direct parameter determination mode as the camera calibration method, when the camera view mode is the uniform-view mode.

7. The camera calibration system of claim 5, wherein the uniform-view mode includes one from among a first uniform-view mode when the determined angle is closer to an angle corresponding to the side-view mode than an angle corresponding to the top-view mode and a second uniform-view mode when the determined angle is closer to the angle corresponding to the top-view mode than the angle corresponding to the side-view mode.

8. The camera calibration system of claim 7, wherein the camera calibration module is further configured to:

perform the camera calibration by selecting an adaptive parameter determination mode as the camera calibration method, when the camera view mode is the first uniform-view mode; and

perform the camera calibration by selecting a direct parameter determination mode as the camera calibration method, when the camera view mode is the second uniform-view mode.

11. The camera calibration system of claim 1, wherein the first vanishing point is a vertical vanishing point and the second vanishing point is a horizontal vanishing point.

12. The camera calibration system of claim 5, wherein when the determined angle is less than 25 degrees, the determined camera view mode is the side-view mode,

wherein when the determined angle is greater than or equal to 25 degrees and less than or equal to 55 degrees, the determined camera view mode is the uniform-view mode, and

wherein when the determined angle is greater than 55 degrees and less than or equal to 90 degrees, the determined camera view mode is the top-view mode.

13. The camera calibration system of claim 7, wherein when the determined angle is less than 25 degrees, the determined camera view mode is the side-view mode,

wherein when the determined angle is greater than or equal to 25 degrees and less than or equal to 55 degrees and closer to 25 degrees than 55 degrees, the determined camera view mode is the first uniform-view mode,

wherein when the determined angle is greater than or equal to 25 degrees and less than or equal to 55 degrees and closer to 55 degrees than 25 degrees, the determined camera view mode is the second uniform-view mode, and

wherein when the determined angle is greater than 55 degrees and less than or equal to 90 degrees, the determined camera view mode is the top-view mode.

14. The camera calibration system of claim 6, wherein when the selected camera calibration method is the adaptive parameter determination mode, the camera calibration module is further configured to:

find a horizontal line from the first vanishing point;

measure a ratio of a distance from a bottom of a box on an image of the object to the horizontal line on the image to a width of the box;

determine an angle between the camera and a ground surface and calculate an installation height of the camera;

calculate a focal distance of the camera;

calculate a size of the box; and

extract a ratio of the calculated size of the box to a pre-set size of the box.

15. The camera calibration system of claim 14, wherein the camera calibration module is further configured to:

redetermine the angle between the camera and the ground surface according to the extracted ratio and perform again the operation of calculating the installation height of the camera.

9. A camera calibration method comprising:

detecting a first vanishing point and a second vanishing point from a plurality of reference detection objects existing in a detection region;

determining a camera view mode on the basis of a result of comparing a distance from a position in the detection region to the first vanishing point with a distance from the position to the second vanishing point, by determining the camera view mode, among a plurality of predetermined camera view modes corresponding to a plurality of predetermined camera calibration methods, that corresponds to the result of the comparing; and

selecting a camera calibration method corresponding to the determined camera view mode among the plurality of predetermined camera calibration methods, and performing camera calibration through the selected camera calibration method.

10. An apparatus comprising:

a processor configured to;

identify an object in a detection region as a reference detection object that provides orientation information relative to a surface;

detect a first vanishing point based on the reference detection object;

detect a second vanishing point based on the reference detection object;

detect a first distance between a first point in the detection region and the first vanishing point;

detect a second distance between the first point and the second vanishing point;

determine an orientation of a camera capturing the detection region based on a result of comparing the first distance with the second distance;

determine a view mode of the camera, among a plurality of predetermined view modes of the camera corresponding to a plurality of predetermined camera calibration methods, that corresponds to the determined orientation of the camera;

select a camera calibration method corresponding to the determined view mode of the camera among the plurality of predetermined camera calibration methods; and

perform camera calibration through the selected camera calibration method.