Device for determining disappearing direction and method thereof, apparatus for video camera calibration and method thereof

• US 9,202,277 B2
• Filed: 05/02/2012
• Issued: 12/01/2015
• Est. Priority Date: 05/16/2011
• Status: Expired due to Fees

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Liu, Yuyu, Nie, Zhi
• Primary Examiner(s): Ustaris, Joseph
• Assistant Examiner(s): NAWAZ, TALHA M
• Application Number:
  US13/462,210
• Publication Number:
  US 20120293663A1
• Time in Patent Office: 1,308 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 3/06 Topological mapping of higher dimensional structures onto lower dimensional surfaces
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A disappearing direction determination device and method, a video camera calibration apparatus and method, a video camera and a computer program product are provided. The device comprises: a moving target detecting unit for detecting in the video image a moving target area where a moving object locates; a feature point extracting unit for extracting at least one feature point on the moving object in the detected moving target area; a moving trajectory obtaining unit for tracking a movement of the feature point in a predetermined number of video image frames to obtain a movement trajectory of the feature point; and a disappearing direction determining unit for determining, according to the movement trajectories of one or more moving objects in the video image, a disappearing direction pointed by a major moving direction of the moving objects. Thus, a disappearing direction and video camera gesture parameters can be determined accurately.

Claims

1. A device for determining a disappearing direction, comprising:

circuitry configured to;

detect a plurality of edges of one or more objects in a video image to obtain corresponding line segments on the one or more objects;

group the line segments into a plurality of line segment groups by a predetermined rule, initially estimate the disappearing direction according to the plurality of the line segment groups, and obtain an initial estimation of the disappearing direction;

select at least one line segment which is closest to the initial estimation from a predetermined number of the line segment groups among all the line segment groups; and

determine the disappearing direction using the selected at least one line segment, wherein the circuitry is configured to

initially estimate the disappearing direction by using a second disappearing direction perpendicular to the disappearing direction to be estimated, and

combine the line segment groups into one or more combinations, each combination including at least two groups.

2. The device for determining the disappearing direction according to claim 1, wherein, the predetermined rule is:

line segments of a same object in the video image are grouped into a same line segment group.

3. The device for determining the disappearing direction according to claim 1,

wherein, each combination includes a first group and a second group, and

wherein the circuitry is configured to

for each combination in a predetermined number of combinations in all combinations, calculate an outer product of edge plane normal vectors of every two line segments respectively of the first group and the second group, and seek an inner product of the calculated outer product and the second disappearing direction;

and

compare all calculated inner products for the predetermined number of combinations with a first threshold, and determine a direction of an edge plane normal vector corresponding to any inner product smaller than the first threshold as an initial value of the initial estimation of the disappearing direction.

4. The device for determining the disappearing direction according to claim 3,

wherein, the circuitry is configured to select, from determined initial values, a final initial value with a maximum number of support groups as a final value of the initial estimation of the disappearing direction, and

wherein, for each support group, a minimum inner product in all the inner products calculated with the initial value and edge plane normal vectors of all line segments in the support group, is smaller than a second threshold.

5. The device for determining the disappearing direction according to claim 4,

wherein, when there are more than one initial value with a same maximum number of support groups, the circuitry is configured to select the final initial value which makes a target function smallest as the final value of the initial estimation, and

wherein, the target function includes a measurement of a perpendicular degree between the initial value and the second disappearing direction, and a measurement of a parallel degree between the initial value and line segments most parallel to the initial value in other support groups.

6. The device for determining the disappearing direction according to claim 5,

wherein, the target function is:

7. A method of determining a disappearing direction, comprising:

detecting a plurality of edges of one or more objects in a video image to obtain corresponding line segments on the one or more objects;

grouping the line segments into a plurality of line segment groups by a predetermined rule, and initially estimating the disappearing direction according to the plurality of the line segment groups to obtain an initial estimation of the disappearing direction;

selecting at least one line segment which is closest to the initial estimation from a predetermined number of the line segment groups among all the line segment groups; and

determining the disappearing direction by using the selected at least one line segments, wherein

the disappearing direction is initially estimated by using a second disappearing direction perpendicular to the disappearing direction to be estimated, and

the line segment groups are combined into one or more combinations, each combination including at least two groups.

8. A non-transitory computer program product, comprising program codes which, when executed by a processor of a computer, cause the processor to perform a method comprising:

detecting a plurality of edges of one or more objects in an input video image to obtain corresponding line segments on the one or more objects;

grouping the line segments into a plurality of line segment groups by a predetermined rule, and initially estimating the disappearing direction according to the plurality of the line segment groups to obtain an initial estimation of the disappearing direction;

selecting at least one line segment which is closest to the initial estimation from a predetermined number of the line segment groups among all the line segment groups; and

determining a disappearing direction by using the selected at least one line segment, wherein

the disappearing direction is initially estimated by using a second disappearing direction perpendicular to the disappearing direction to be estimated, and

the line segment groups are combined into one or more combinations, each combination including at least two groups.