MULTI-VIEW ALIGNMENT BASED ON FIXED-SCALE GROUND PLANE RECTIFICATION

• US 20120327220A1
• Filed: 05/29/2012
• Published: 12/27/2012
• Est. Priority Date: 05/31/2011
• Status: Abandoned Application

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Ma, Zhonghua
• Application Number:
  US13/482,739
• Publication Number:
  US 20120327220A1
• Time in Patent Office: Days
• US Class Current: 348/135
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30232 Surveillance
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: G06T 7/292 Multi-camera tracking
• CPC Class Codes: G06T 7/38 Registration of image sequences
• CPC Class Codes: H04N 23/64 Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image

Abstract

A system and method of generating a common ground plane from a plurality of image sequences includes detecting at least three observations for each image sequence, generating a plurality of rectified ground planes for the plurality of image sequences, determining a geometric property of the plurality of observations in the plurality of image sequences, determining a relative scaling factor of each of the plurality of rectified ground planes, and generating the common ground plane from the plurality of image sequences based on the rectified ground planes and the determined relative scaling factors.

Claims

1. A method of generating a common ground plane from a plurality of image sequences, each image sequence captured by a corresponding one of a plurality of cameras, said plurality of cameras having disjoint fields of view of a scene, said method comprising the steps of:

detecting at least three observations for each image sequence;

generating a plurality of rectified ground planes for the plurality of image sequences, said generation being based on a scene geometry and a spatial property of each corresponding camera determined from said detected observations in each of the image sequences;

determining a geometric property of the plurality of observations in the plurality of image sequences;

determining a relative scaling factor of each of said plurality of rectified ground planes, said relative scaling factor based on the geometric property of the plurality of objects in the images and the spatial property of each camera; and

generating the common ground plane from the plurality of image sequences based on said rectified ground planes and said determined relative scaling factors.

2. The method according to claim 1, comprising the further step of:

generating an overhead perspective view of said scene, based on said relative scaling factors of said common ground plane.

3. The method according to claim 1, further comprising a step of determining the scene geometry, wherein said step of determining the scene geometry comprises:

estimating a horizon of the scene; and

estimating a vertical vanishing point of the scene.

4. The method according to claim 3, wherein said step of determining the scene geometry is based on a set of predetermined features associated with the observations.

5. The method according to claim 3, wherein said step of determining the scene geometry comprises the steps of:

retrieving a set of track data of the plurality of observations;

linking a set of first features of the plurality of detected observations to produce a first line for the detected observations;

linking a set of second features of the plurality of detected observations to produce a second line for the detected observations; and

determining an intersection point of at least the first line and the second line to be the vertical vanishing point of the scene.

6. The method according to claim 5, further comprising the step of:

linking a plurality of the vertical vanishing points of the scene to be the horizon of the scene.

7. The method according to claim 1, wherein the spatial property of each camera includes a camera roll angle and a camera tilt angle of the respective camera.

8. The method according to claim 1, wherein determining said geometric properties of the plurality of observations in the images of all cameras is based on a vertical position of the object in the image plane from the horizon position.

9. The method according to claim 1, wherein said observations relate to at least three detections of a single object in an image sequence or at least two detections of each of two objects in an image sequence.

10. A computer readable storage medium having recorded thereon a computer program for directing a processor to execute a method of generating a common ground plane from a plurality of image sequences, each image sequence captured by a corresponding one of a plurality of cameras, said plurality of cameras having disjoint fields of view of a scene, said computer program comprising code for performing the steps of:

detecting at least three observations for each image sequence;

generating a plurality of rectified ground planes for the plurality of image sequences, said generation being based on a scene geometry and a spatial property of each corresponding camera determined from said detected observations in each of the image sequences;

determining a geometric property of the plurality of observations in the plurality of image sequences;

determining a relative scaling factor of each of said plurality of rectified ground planes, said relative scaling factor based on the geometric property of the plurality of objects in the images and the spatial property associated with each camera; and

generating the common ground place from the plurality of image sequences based on said rectified ground planes and said determined relative scaling factors.

11. A multi-camera system comprising:

a plurality of cameras having disjoint fields of view of a scene, each camera having a lens system, an associated sensor, and a control module for controlling said lens system and said sensor to capture an image of said scene;

a storage device for storing a computer program; and

a processor for executing the program, said program comprising;

computer program code for generating a common ground plane from a plurality of image sequences captured by said plurality of cameras, each image sequence derived from one of said plurality of cameras, the generating including the steps of;

detecting at least three observations for each image sequence;

generating a plurality of rectified ground planes for the plurality of image sequences, said generation being based on a scene geometry and a spatial property of each corresponding camera determined from said detected observations in each of the image sequences;

determining a geometric property of the plurality of observations in the plurality of image sequences;

determining a relative scaling factor of each of said plurality of rectified ground planes, said relative scaling factor based on the geometric property of the plurality of objects in the images and the spatial property associated with each camera; and

generating the common ground plane from the plurality of image sequences based on said rectified ground planes and said determined relative scaling factors.

12. A multi-camera system comprising:

a plurality of cameras having disjoint fields of view of a scene, each camera having a lens system, an associated sensor, and a control module for controlling said lens system and said sensor to capture a respective image sequence of said scene;

a computer server coupled to each of said plurality of cameras, said server including;

a storage device for storing a computer program; and

a processor for executing the program, said program comprising;

computer program code for generating a common ground plane from a plurality of image sequences captured by said plurality of cameras, each image sequence derived from one of said plurality of cameras, the generating including the steps of;

detecting at least three observations for each image sequence;

generating a plurality of rectified ground planes for the plurality of image sequences, said generation being based on a scene geometry and a spatial property of each corresponding camera determined from said detected observations in each of the image sequences;

determining a geometric property of the plurality of observations in the plurality of image sequences;

determining a relative scaling factor of each of said plurality of rectified ground planes, said relative scaling factor based on the geometric property of the plurality of objects in the images and the spatial property associated with each camera; and

generating the common ground plane from the plurality of image sequences based on said rectified ground planes and said determined relative scaling factors.