Object segmentation from images acquired by handheld cameras

• US 7,280,685 B2
• Filed: 11/14/2002
• Issued: 10/09/2007
• Est. Priority Date: 11/14/2002
• Status: Expired due to Fees

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Beardsley, Paul A.
• Primary Examiner(s): Bali; Vikkram
• Application Number:
  US10/294,050
• Publication Number:
  US 20040095484A1
• Time in Patent Office: 1,790 Days
• Field of Search: 382/154, 345/420, 345/413
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/20132 Image cropping
• CPC Class Codes: G06T 7/187 involving region growing; involving region merging; involving connected component labelling
• CPC Class Codes: G06T 7/564 from contours
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06V 10/267 by performing operations on regions, e.g. growing, shrinking or watersheds

Abstract

A method determines a shape of an object in from an image. A visual hull larger than the object is constructed such that the object can be entirely contained within the visual hull. An image is acquired of the object by a camera. The visual hull is projected onto the image according to a position and pose of the camera.

Seed pixels in the image are selected. The seed pixels are located on the projected visual hull, and a region is grown around each seed pixel. Each region includes neighboring pixels resembling the seed pixel. Then, the visual hull is reduced in size according to the grown regions to determine a two-dimensional shape of the object. When multiple images are used from different points of view, the three-dimensional shape can be determined.

Claims

1. A method for determining a shape of an object in an image, comprising:

constructing a visual hull for an object such that the object can be entirely contained within the visual hull, in which the visual hull is approximate;

acquiring a shape image of the object with a camera;

projecting the visual hull onto the shape image according to a position and pose of the camera; and

reducing the projected visual hull in size to conform substantially to a shape of the object in the shape image.

2. The method of claim 1 further comprising:

acquiring, projecting, and reducing for a plurality of shape images from a plurality of points of view to determine a three dimensional shape of the object.

3. The method of claim 1 further comprising:

segmenting the object from the shape image according to the reduced size visual hull.

4. The method of claim 1, wherein the camera has a narrow field of view.

5. The method of claim 1, wherein an initial shape of the visual hull is parametrically defined.

6. The method of claim 5 wherein the initial visual hull is a cube.

7. The method of claim 5 wherein the initial visual hull is a sphere.

8. The method of claim 1 further comprising:

applying image subtraction to define an initial shape of the visual hull.

9. The method of claim 1 further comprising:

acquiring a position image of a known background with another camera fixed to the camera to determine the position and pose of the camera.

10. The method of claim 4 wherein the field of views of the camera and the other camera are disjoint.

11. The method of claim 1, wherein the reducing further comprises:

selecting pixels located on the projected visual hull as seed pixels;

growing a region around each seed pixel, each region including neighboring pixels resembling the seed pixel, wherein the each pixel in the neighboring pixels satisfies predetermined resemblance criterion; and

reducing the visual hull in size according to the grown regions.

12. The method of claim 11, wherein the resemblance criterion is based on a gradient magnitude of the pixels.

13. The method of claim 1, in which the shape image is acquired by a handheld camera.