Device for determining disappearing direction and method thereof, apparatus for video camera calibration and method thereof

• US 9,953,421 B2
• Filed: 03/24/2015
• Issued: 04/24/2018
• Est. Priority Date: 05/16/2011
• Status: Active Grant

• Current Assignee: Sony Corporation (Sony Group Corp.)
• Original Assignee: Sony Corporation (Sony Group Corp.)
• Inventors: Liu, Yuyu, Nie, Zhi
• Primary Examiner(s): Nawaz, Talha M
• Application Number:
  US14/667,502
• Publication Number:
  US 20150199814A1
• Time in Patent Office: 1,127 Days
• Field of Search: None
• CPC Class Codes: G06T 2207/20021 Dividing image into blocks, subimages or windows
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 3/06 Topological mapping of higher dimensional structures onto lower dimensional surfaces
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/70 Determining position or orientation of objects or cameras camera calibration G06T7/80
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A disappearing direction determination device and method, a video camera calibration apparatus and method, a video camera and a computer program product are provided. The device comprises: a moving target detecting unit for detecting in the video image a moving target area where a moving object locates; a feature point extracting unit for extracting at least one feature point on the moving object in the detected moving target area; a moving trajectory obtaining unit for tracking a movement of the feature point in a predetermined number of video image frames to obtain a movement trajectory of the feature point; and a disappearing direction determining unit for determining, according to the movement trajectories of one or more moving objects in the video image, a disappearing direction pointed by a major moving direction of the moving objects. Thus, a disappearing direction and video camera gesture parameters can be determined accurately.

Claims

1. A video camera, comprising:

an apparatus for video camera calibration configured to calibrate the video camera based on a plurality of video images shot by the video camera, the apparatus comprising a device for determining a disappearing direction and a video camera calibration device,

wherein the device for determining the disappearing direction comprises;

circuitry configured to;

detect, in the video images, a moving target area where a moving object is located;

extract at least one feature point on the moving object in the detected moving target area, the at least one feature point being a corner of the moving object, or a corner of texture of the moving object;

track a movement of the at least one feature point in a predetermined number of video image frames to obtain a movement trajectory of the at least one feature point;

for each feature point, fit a line segment according to the movement trajectory of the respective feature point in the predetermined number of video image frames; and

determine, by using the line segment of each feature point, the disappearing direction defined by a major moving direction of the moving objects, and

wherein the video camera calibration device is configured to determine a gesture of the video camera relative to a ground by using the disappearing direction determined by the device for determining the disappearing direction.

2. A method of determining a gesture of a video camera relative to a ground, comprising:

detecting, in a plurality of video images, a moving target area where a moving object is located;

extracting at least one feature point on the moving object in the detected moving target area;

tracking a movement of the at least one feature point in a predetermined number of video image frames to obtain a movement trajectory of the at least one feature point;

for each feature point, fitting a line segment according to the movement trajectory of the respective feature point in the predetermined number of video image frames;

by using the line segment of each feature point, determining the disappearing direction defined by a major moving direction of the moving objects; and

determining a gesture of a video camera relative to a ground by using the disappearing direction.

3. A non-transitory computer program product, comprising program codes which, when executed by a processor of a computer, cause the processor to perform:

detecting, in a plurality of input video images, a moving target area where a moving object is located;

extracting at least one feature point on the moving object in the detected moving target area;

tracking a movement of the at least one feature point in a predetermined number of video image frames to obtain a movement trajectory of the at least one feature point;

for each feature point, fitting a line segment according to the movement trajectory of the respective feature point in the predetermined number of video image frames;

by using the line segment of each feature point, determining a disappearing direction defined by a major moving direction of the moving objects; and

determining a gesture of a video camera relative to a ground by using the disappearing direction.

4. A video camera comprising:

a device for determining a disappearing direction configured to;

detect, in a plurality of video images, a moving target area where a moving object is located;

extract at least one feature point on the moving object in the detected moving target area, the at least one feature point being a corner of the moving object, or a corner of texture of the moving object;

track a movement of the at least one feature point in a predetermined number of video image frames to obtain a movement trajectory of the at least one feature point;

for each feature point, fitting a line segment according to the movement trajectory of the respective feature point in the predetermined number of video image frames; and

determine, by using the line segment of each feature point, the disappearing direction defined by a major moving direction of the moving objects, the video images being shot by the video camera; and

a video camera calibration device configured to determine a gesture of the video camera relative to a ground by using the disappearing direction determined by the device for determining the disappearing direction.