Image processing apparatus

• US 20040155877A1
• Filed: 02/05/2004
• Published: 08/12/2004
• Est. Priority Date: 02/12/2003
• Status: Abandoned Application

• Current Assignee: Canon Europa N.V. (Canon Inc.)
• Original Assignee: Canon Europa N.V. (Canon Inc.)
• Inventors: Hong, Qi He, Lyons, Alexander Ralph, Baumberg, Adam Michael
• Application Number:
  US10/771,416
• Publication Number:
  US 20040155877A1
• Time in Patent Office: Days
• US Class Current: 345/419
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20092 Interactive image processing based on input by user
• CPC Class Codes: G06T 7/11 Region-based segmentation

Abstract

In an image processing apparatus 2, images of a subject object 210 and data defining the positions and orientations at which the images were recorded are processed to generate a three-dimensional computer model of the subject object 210. As part of the processing, image data relating to the subject object 210 is segmented from other image data in each input image to define the silhouette of the subject object in each image, and the silhouettes are processed to generate the three-dimensional computer model. To improve the accuracy of the segmentation processing, and therefore the accuracy of each silhouette and the three-dimensional computer model, processing apparatus 2 defines a volume of three-dimensional space enclosing the subject object, projects the volume into at least one of the input images, selects pixels representative of the background by using the projection of the volume to prevent the selection of pixels representing the subject object, and uses the pixels to establish parameters to be used in the segmentation processing.

Claims

1. A method of processing data defining a plurality of images of an object recorded at different positions and orientations and data defining the positions and orientations to generate data defining a three-dimensional computer model of the object, the method comprising:

defining a volume in three-dimensional space enclosing the object;

determining the two-dimensional projection of the volume in at least one of the images;

selecting pixels from at least one image in dependence upon the volume projection therein;

determining segmentation parameters in dependence upon at least one image property of the selected pixels, the segmentation parameters comprising parameters for distinguishing between subject object image data and other image data during segmentation processing;

processing the image data to segment image data relating to the object from other image data in at least some of the images using the generated segmentation parameters; and

generating data defining a three-dimensional computer model of the object using the results of the segmentation processing and the data defining the positions and orientations at which the images were recorded.

2. A method according to claim 1, wherein the pixels are selected from an image by determining the position of the outer perimeter of the volume projection in the image and selecting pixels in dependence upon the determined outer perimeter position.

3. A method according to claim 2, wherein pixels are selected from an image by selecting pixels from a band adjacent the outer perimeter of the volume projection.

4. A method according to claim 1, wherein the processing operations are repeated at least once, and wherein, on the second and each subsequent time the operations are performed, the process of defining a volume in the three-dimensional space enclosing the object comprises defining the volume to be the three-dimensional computer model of the object generated a previous time the operations were performed.

5. A method according to claim 1, wherein, in the processing to segment image data relating to the object from other image data in an image, the segmentation processing using the generated segmentation parameters is performed only on image data within the projection of the volume in the image, and the image data outside the projection of the volume is classified as image data which does not relate to the object.

6. A method according to claim 1, wherein the segmentation parameters are determined in dependence upon the value of at least one colour component of each selected pixel.

7. A method according to claim 1, wherein:

each image to be processed shows the object together with a calibration object and the data defining the positions and orientations of the images defines the positions and orientations of the images and the position of the calibration object in the same three-dimensional coordinate system; and

the volume enclosing the object is defined in the three-dimensional coordinate system of the images and calibration object in dependence upon the calibration object.

8. A method according to claim 1, further comprising generating a signal carrying data defining the generated three-dimensional computer model.

9. A method according to claim 8, further comprising making a recording of the signal either directly or indirectly.

23. A storage medium storing computer program instructions to program a programmable processing apparatus to become operable to perform a method as set out in any one of claims 1 to 7 and 10.

24. A signal carrying computer program instructions to program a programmable processing apparatus to become operable to perform a method as set out in any one of claims 1 to 7 and 10.

10. A method of processing data defining a plurality of images of an object recorded at different positions and orientations and data defining the positions and orientations to segment image data relating to the object from other image data in the images, the method comprising:

defining a volume in three-dimensional space enclosing the object;

determining the two-dimensional projection of the volume in at least one of the images;

selecting pixels from at least one image in dependence upon the volume projection therein;

determining segmentation parameters in dependence upon at least one image property of the selected pixels, the segmentation parameters comprising parameters for distinguishing between subject object image data and other image data during segmentation processing; and

segmenting image data relating to the object from other image data in at least some of the images using the generated segmentation parameters.

11. A method according to claim 10, further comprising generating a signal carrying data defining the silhouette of the subject object in each of the at least some images.

12. A method according to claim 11, further comprising making a recording of the signal either directly or indirectly.

13. An apparatus for processing data defining a plurality of images of an object recorded at different positions and orientations and data defining the positions and orientations to generate data defining a three-dimensional computer model of the object, the apparatus comprising:

a volume definer operable to define a volume in three-dimensional space enclosing the object;

a volume projector operable to determine a two-dimensional projection of the volume in at least one of the images;

a pixel selector operable to select pixels from at least one image in dependence upon the volume projection therein;

a segmentation parameter definer operable to determine segmentation parameters in dependence upon at least one image property of the selected pixels, the segmentation parameters comprising parameters for distinguishing between subject object image data and other image data during segmentation processing;

an image data segmenter operable to process the image data to segment image data relating to the object from other image data in at least some of the images using the generated segmentation parameters; and

a three-dimensional computer model data generator operable to generate data defining a three-dimensional computer model of the object using the results of the segmentation processing and the data defining the positions and orientations at which the images were recorded.

14. An apparatus according to claim 13, wherein said pixel selector is operable to select pixels from an image by determining the position of an outer perimeter of the volume projection in the image and selecting pixels in dependence upon the determined outer perimeter position.

15. An apparatus according to claim 14, wherein said pixel selector is operable to select pixels from an image by selecting pixels from a band adjacent the outer perimeter of the volume projection.

16. An apparatus according to claim 13, wherein the apparatus is operable to repeat the processing operations at least once, and wherein, on the second and each subsequent time the operations are performed, said volume definer is arranged to define the volume to be the three-dimensional computer model of the object generated a previous time the operations were performed.

17. An apparatus according to claim 13, wherein said image data segmenter is operable to perform segmentation processing using the generated segmentation parameters only on image data within the projection of the volume in the image, and to classify the image data outside the projection of the volume as image data which does not relate to the object.

18. An apparatus according to claim 13, wherein said segmentation parameter definer is operable to determine the segmentation parameters in dependence upon the value of at least one colour component of each selected pixel.

19. An apparatus according to claim 13, wherein:

each image to be processed shows the object together with a calibration object and the data defining the positions and orientations of the images defines the positions and orientations of the images and the position of the calibration object in the same three-dimensional coordinate system; and

said volume definer is operable to define the volume enclosing the object in the three-dimensional coordinate system of the images and calibration object in dependence upon the calibration object.

20. An apparatus for processing data defining a plurality of images of an object recorded at different positions and orientations and data defining the positions and orientations to segment image data relating to the object from other image data in the images, the apparatus comprising:

a volume definer operable to define a volume in three-dimensional space enclosing the object;

a projection calculator operable to determine the two-dimensional projection of the volume in at least one of the images;

a pixel selector operable to select pixels from at least one image in dependence upon the volume projection therein;

a segmentation parameter definer operable to determine segmentation parameters in dependence upon at least one image property of the selected pixels, the segmentation parameters comprising parameters for distinguishing between subject object image data and other image data during segmentation processing; and

an image data segmenter operable to segment image data relating to the object from other image data in at least some of the images using the generated segmentation parameters.

21. An apparatus for processing data defining a plurality of images of an object recorded at different positions and orientations and data defining the positions and orientations to generate data defining a three-dimensional computer model of the object, the apparatus comprising:

means for defining a volume in three-dimensional space enclosing the object;

means for determining the two-dimensional projection of the volume in at least one of the images;

means for selecting pixels from at least one image in dependence upon the volume projection therein;

means for determining segmentation parameters in dependence upon at least one image property of the selected pixels, the segmentation parameters comprising parameters for distinguishing between subject object image data and other image data during segmentation processing;

means for processing the image data to segment image data relating to the object from other image data in at least some of the images using the generated segmentation parameters; and

means for generating data defining a three-dimensional computer model of the object using the results of the segmentation processing and the data defining the positions and orientations at which the images were recorded.

22. An apparatus for processing data defining a plurality of images of an object recorded at different positions and orientations and data defining the positions and orientations to segment image data relating to the object from other image data in the images, the apparatus comprising:

means for defining a volume in three-dimensional space enclosing the object;

means for determining the two-dimensional projection of the volume in at least one of the images;

means for selecting pixels from at least one image in dependence upon the volume projection therein;

means for determining segmentation parameters in dependence upon at least one image property of the selected pixels, the segmentation parameters comprising parameters for distinguishing between subject object image data and other image data during segmentation processing; and

means for segmenting image data relating to the object from other image data in at least some of the images using the generated segmentation parameters.