Apparatus for recovering background in image sequence and method thereof

• US 7,555,158 B2
• Filed: 11/18/2005
• Issued: 06/30/2009
• Est. Priority Date: 12/07/2004
• Status: Active Grant

• Current Assignee: Electronics and Telecommunications Research Institute
• Original Assignee: Electronics and Telecommunications Research Institute
• Inventors: Kim, Hyun-Bin, Kim, Hong-Seok, Lee, In-Ho, Park, Chang-Joon, Kim, Sung-Eun
• Primary Examiner(s): Bella; Matthew C
• Assistant Examiner(s): CHU, RANDOLPH I
• Application Number:
  US11/283,118
• Publication Number:
  US 20060120592A1
• Time in Patent Office: 1,320 Days
• Field of Search: 382/173, 382/154, 382/282, 382/283, 345/619, 348/139, 348/159
• US Class Current: 382/154
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 15/20 Perspective computation

Abstract

Provided is a background recovering apparatus for deleting an object of an arbitrary region and recovering a background covered with the object. The apparatus includes a storing block for storing the input image sequence, and storing the three dimensional (3D) position and posture information, and focal length information; a geometric information extracting block for extracting 3D with respect to the background; a background image generating block for generating the background image; and a background image inserting block for recovering the background by inserting the background image into an arbitrary region of a first view point.

Claims

1. A background recovering apparatus for deleting an object of an arbitrary region in an input image sequence and recovering a background, comprising:

a storing means for storing the input image sequence, and storing the three dimensional (3D) position and posture information, and focal length information of a camera photographing an image;

a projection matrix generating means for generating a projection matrix of a first view point by using the 3D position and posture information and focal length information of the camera photographing an image frame of the first view point, in which a deletion target object exits;

a geometric information extracting means for extracting 3D geometric information with respect to the background by using the 3D position and posture information, focal length information, and the background region coordinates of the camera photographing an image frame of another view point that reveals the background behind the deletion target object;

a background image generating means for generating the background image whose view point is changed into the first view point by multiplying the 3D geometric information for the background extracted from image frame of another view point by the projection matrix; and

a background image inserting means for recovering the background by inserting the background image, whose view point is changed into the first view point, into an arbitrary region of the first view point.

2. The apparatus as recited in claim 1, wherein the geometric information extracting means extracts 3D geometric information with respect to the background covered with deletion object target by using the 3D position and posture information, focal length information, and the background region coordinates of a camera photographing the image frame of the second view point that reveals background covered with the deletion target object, and by using the 3D position and posture information, focal length information and the background region coordinates of the camera photographing the image frame of the third view point that reveals background covered with an deletion target object.

3. The apparatus as recited in claim 1, further comprising:

a resolution interpolating means for correcting resolution by interpolating the background image whose view point is changed into the first view point, wherein the background image being generated in the background image generating means.

4. A method for deleting an object of an arbitrary region in the input image sequence and recovering the background comprising:

using a processor to perform steps of

a) generating the projection matrix of a first view point by using 3D position and posture information, and the focal length information of a camera photographing an image frame of the first view point, where the deletion target object exists;

b) extracting 3D geometric information for the background by using the 3D position and posture information, the focal length information, and the background region coordinates of the camera photographing the image frame that reveals the background covered with the deletion target object;

c) generating the background image whose view point is changed into the first view point by multiplying the projection matrix by the 3D geometric information for the background extracted from an image frame of another view point; and

d) recovering the background by inserting the background image that the view point is changed into the first view point into an arbitrary region of the first point.

5. The method as recited in claim 4, wherein, in the step b), 3D geometric information with respect to the background covered with deletion target object is extracted by using the 3D position and posture information, focal length information, and the background region coordinates of the camera photographing the image frame of a second view point that reveals the background covered with the deletion target object, and by using the 3D position and posture information, focal length information and the background region coordinates of the camera photographing the image frame of a third view point that reveals the background covered with an deletion target object.

6. The method as recited in claim 4, further comprising the step of:

e) correcting resolution by interpolating the background image whose view point is changed into the first view point, the background image is generated in the background image generating means.