Image processing methods and apparatus

• US 8,417,021 B2
• Filed: 10/12/2006
• Issued: 04/09/2013
• Est. Priority Date: 10/13/2005
• Status: Expired due to Fees

• Current Assignee: Cambridge University (Government of United Kingdom)
• Original Assignee: Cambridge University (Government of United Kingdom)
• Inventors: Cipolla, Roberto, Vogiatzis, George, Favaro, Paolo, Funayama, Ryuji, Yanagihara, Hiromichi
• Primary Examiner(s): Le, Vu
• Assistant Examiner(s): WOLDEMARIAM, AKLILU K
• Application Number:
  US12/089,140
• Publication Number:
  US 20090169096A1
• Time in Patent Office: 2,371 Days
• Field of Search: 382/232, 382/235, 382/154, 375/240.26, 375/240.03
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10021 Stereoscopic video; Stereoscopic image sequence
• CPC Class Codes: G06T 2207/30264 Parking
• CPC Class Codes: G06T 7/557 from light fields, e.g. from plenoptic cameras
• CPC Class Codes: G06T 7/579 from motion
• CPC Class Codes: G06T 7/586 from multiple light sources, e.g. photometric stereo

Abstract

We describe methods of characterizing a set of images to determine their respective illumination, for example for recovering the 3D shape of an illuminated object. The method comprises: inputting a first set of images of the object captured from different positions; determining frontier point data from the images, this defining a plurality of frontier points on the object and for each said frontier point a direction of a normal to the surface of the object at the frontier point, and determining data defining the image capture positions; inputting a second set of images of said object, having substantially the same viewpoint and different illumination conditions; and characterizing the second set of images said frontier point data to determine data comprising object reflectance parameter data (β) and, for each image of said second set, illumination data (L) comprising data defining an illumination direction and illumination intensity for the image.

Claims

1. A computer-implemented of determining data characterising a set of images of an object, the method being implemented in a computer comprising processor and a memory in communication with the processor, the method comprising:

inputting, to the processor, data from a first set of images of said object captured from a plurality of different positions;

determining, by the processor, frontier point data from said data for said first set of images and data defining said image capture positions, said frontier point data defining a plurality of frontier points on said object, and for each said frontier point, a direction of a normal to the surface of the object at the frontier point;

inputting, to the processor, data from a second set of captured images of said object, said second set of images having substantially the same viewpoint and different illumination conditions; and

determining, by the processor, data characterising said second set of images using said data for said second set of images and said frontier point data, said image set characterising data comprising object reflectance parameter data (β

) and, for each image of said second set, illumination data (L) comprising data defining an illumination direction and illumination intensity for the image;

wherein said determining of frontier point data uses pair of images of said first set of images, the two images of said pair being captured by first and second respective cameras; and

wherein each frontier point is a point in three-dimensions on the surface of the object wherein the plane defined by the frontier point and centres of said first and second cameras viewing the frontier point is tangent to the object.

2. A method as claimed in claim 1 further comprising generating, by the processor, a plurality of said pairs of images by pairing an image captured by said first camera at a first position with a plurality of images captured by said second camera at a plurality of different second positions.

3. A method as claimed in claim 1 wherein said frontier point determining comprises determining, by the processor, silhouette data for each image of said first set of images;

and determining, by the processor, said frontier point data using said silhouette data.

4. A method as claimed in claim 1 wherein said determining of image set characterising data further comprises determining, by the processor, projected positions of said frontier points defined by said frontier point data into an image of said second set of images, determining, by the processor, image intensity data at each of said projected positions, and determining, by the processor, said image set characterising data responsive to said projected positions and said image intensity data at said projected positions.

5. A method as claimed in claim 1 wherein said determining of image set characterising data further comprises:

determining, by the processor, postulated image set characterising data comprising postulated object reflectance parameter data and postulated illumination data for said images of said second set of images;

determining, by the processor, predicted second set of image data from said postulated image set characterising data and said frontier data, said predicted second set image data defining for said images of said second set of images, predicted image values at projections of said frontier points onto said images of said second set of images; and

minimising, by the processor, a cost function dependent upon a summed discrepancy between said predicted second set image data and said data for said second set of images.

6. A method as claimed in claim 1 further comprising determining, by the processor, geometry data defining a geometry of said object surface using said image set characterising data.

7. A method as claimed in claim 6 wherein said geometry data determining comprises determining, by the processor, said geometry data by a photometric stereo geometry determining procedure.

8. A method as claimed in claim 6 wherein a pixel of an image of said second set of images has a substantially spatially corresponding counterpart in each other image of said second set of images, and wherein said geometry data determining comprises determining, by the processor, for a set of corresponding pixels, a best fit of a surface normal to the object given pixel values of said set of pixels and said image set characterising data.

10. A method of assisting parking using the method of claim 1.

11. A non-transitory computer readable medium carrying processor control code to, when running, implement the method of claim 1.

9. A method of determining data characterising a set of images of an object, the method comprising:

inputting, by the processor, data from a first set of images of said object captured from a plurality of different positions;

determining , by the processor, frontier point data from said data for said first set of images and data defining said image capture positions, said frontier point data defining a plurality of frontier points on said object, and for each said frontier point, a direction of a normal to the surface of the object at the frontier point;

inputting, to the processor, data from a second set of captured images of said object, said second set of images having different viewpoints and illumination conditions; and

determining, by the processor, data characterising said second set of images using said data for said second set of images and said frontier point data, said image set characterising data comprising object reflectance parameter data (β

) and illumination data comprising data defining a common angular illumination intensity distribution for all the images of said second set of images;

wherein said determining of frontier point data uses pair of images of said first set of images, the two images of a said pair being captured by first and second respective cameras; and

wherein each frontier point is a point in three-dimensions on the surface of the object where the plane defined by the frontier point and centres of said first and second cameras viewing the frontier point is tangent to the object.

12. A system for determining data characterising a set of images of an object, the system comprising:

an input to input data from a first set of images of said object captured from a plurality of different positions, and data from second set of captured images of said object, said second set of images having substantially the same viewpoint and different illumination conditions;

data memory for storing data to be processed;

program memory storing processor control code; and

a processor coupled to said input, data memory and program memory to load and implement said stored code, said code comprising code for controlling the processor to;

determine frontier point data from said data for said first set of images and data defining said image capture positions, said frontier point data defining a plurality of frontier points on said object and for each said frontier point, a direction of a normal to the surface of the object at the frontier point; and

determine data characterising said second set of images using said data for said second set of images and said frontier point data, said image set characterising data comprising object reflectance parameter data (β

) and, for each image of said second set, illumination data (L) comprising data defining an illumination direction and illumination intensity for the image;

wherein said determining of frontier point data uses pairs of images of said first set of images, the two images of a said pair being captured by first and second respective cameras; and

wherein each frontier point is a point in three-dimensions on the surface of the object where the plane defined by the frontier point and centres of said first and second cameras viewing the frontier is tangent to the object.

13. A system as claimed in claim 12 wherein said code further comprises code to determine geometry data defining a geometry of said object surface using said image set characterising data.

14. A system as claimed in claim 12 further comprising at least one camera controller for controlling a position of a camera to capture an image for said first set of images, and an image capture system for capturing images for said first and second sets of images, and wherein said code further comprises code to control said camera controller and capture images for said first and second sets of images.

15. A parking assistance system including the system of claim 12.

16. A computer-implemented method of determining data characterising a set of images of an object, the method being implemented in a computer comprising processor and a memory in communication with the processor, the method comprising:

inputting, to the processor, data from a first set of images of said object captured from a plurality of different positions by at least a first camera and a second camera;

determining, by the processor, frontier point data from said data for said first set of images and data defining said image capture position, said frontier point data defining a plurality of frontier points on said object, and for each said frontier point, a direction of a normal to the surface of the object at the frontier point, wherein each frontier point is a point in three-dimensions on the surface of the object where the plane defined by the frontier point and centres of said first and second cameras viewing the frontier point is tangent to the object;

inputting, to the processor, data from a second set of captured images of said object, said second set of images having substantially the same viewpoint and different illumination conditions; and

determining, by the processor, data characterising said second set of images using said data for said second set of images and said frontier point data, said image set characterising data comprising object reflectance parameter data (β

) and, for each image of said second set, illumination data (L) comprising data defining an illumination direction and illumination intensity for the image.