Deconvolution-based structured light system with geometrically plausible regularization

• US 8,411,995 B2
• Filed: 05/21/2009
• Issued: 04/02/2013
• Est. Priority Date: 05/23/2008
• Status: Active Grant

• Current Assignee: National Research Council Canada
• Original Assignee: National Research Council Canada
• Inventors: Godin, Guy, Drouin, Marc-Antoine
• Primary Examiner(s): TSAI, TSUNG YIN
• Application Number:
  US12/453,765
• Publication Number:
  US 20100034429A1
• Time in Patent Office: 1,412 Days
• Field of Search: None
• US Class Current: 382/279
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light

Abstract

Systems and methods for processing images in a structured light system which may be used to determine the correspondences in a camera-projector system. Those correspondences can later be used to construct a 3D model, to calibrate a projector or for other purposes. The optical and geometric characteristics of the system are initially determined. The capability of establishing correspondences is affected by the limitations of the system and the properties of the surfaces. Once images of the patterns projected on the surface are acquired, they are iteratively segmented and deconvolved using the known characteristics of the system. The result is a set of correspondences with a reduction of artifacts introduced by the limitations of the system. The characteristics of the structured light system which may be used in the iterative segmentation and deconvolution may include the characteristics of a pattern projected on the scene or object, the physical characteristics of the structured light system, as well as the characteristics of the scene or object itself.

Claims

1. A method for processing images from a structured light system for use with data processing systems, the method comprising:

obtaining at least one initial image from a structured light system, the or each of said at least one initial image being of a subject having projected on it at least one pattern by said structured light system;

determining characteristics of said structured light system;

iteratively segmenting and deconvolving said at least one initial image using said characteristics of said structured light system and at least one point spread function to result in at least one resulting image, said at least one resulting image being a more accurate image of said subject compared to said at least one image.

2. A method according to claim 1 wherein at least one of said characteristics of said structured light system comprises at least one of:

shapes present in a pattern projected by said structured light system;

a number of colors or gray levels in said the or each initial image;

expected color intensities in said the or each initial image;

a position of a camera and/or projector in said structured light system;

a type of projector used in said structured light system;

a type of camera used in said structured light system;

a distance of an object from a scanner used in said structured light system;

surface characteristics of at least one object in said the or each initial image;

an expected number of objects in said the or each initial image;

characteristics of said the or each pattern projected by said structured light system on said subject.

3. A method according to claim 1 wherein said at least one initial image is a combination of at least two images of said subject.

4. A method according to claim 1 wherein said method further comprises establishing correspondences between at least one camera pixel and at least a fraction of a pixel of at least one projector using said at least resulting image.

5. A method according to claim 1 wherein said step of deconvolving and segmenting includes generating at least one interim image from the or each said initial image.

6. A method according to claim 5 wherein said step of deconvolving and segmenting includes combining said at least one interim image with at least one other image.

7. A method according to claim 1 wherein said at least one point spread function comprises at least one of:

a point spread function of at least one camera in said structured light system

a point spread function of at least one projector in said structured light system

a point spread function of at least one scene which is said subject of said initial image.

8. A method according to claim 1 wherein said at least one point spread function is a combination of at least two point spread functions, said at least two point spread functions comprising at least two of:

a point spread function of at least one camera in said structured light system;

a point spread function of at least one projector in said structured light system;

a point spread function of at least one scene which is said subject of said initial image.

9. A method according to claim 1 wherein the or each pattern is a pattern belonging to least one of the following categories:

direct codification

neighborhood codification

time-multiplexing codification.

10. A method according to claim 1 further including the step of using at least one regularization term for said deconvolving and segmenting, said at least one regularization term being based on probabilities derived from said characteristics of said structured light system.

12. A method according to claim 1 wherein said segmenting comprises grouping image pixels into meaningful structures.

11. A non-transitory computer readable storage medium having stored thereon at least one computer executable command for execution by a data processing system which, when executed by said data processing system, results in performance of the steps of a method for processing images from a structured light system for use with data processing systems, the method comprising:

obtaining at least one initial image from a structured light system, the or each of said at least one initial image being of a subject having projected on it at least one pattern by said structured light system;

determining characteristics of said structured light system;

iteratively segmenting and deconvolving said at least one initial image using said characteristics of said structured light system and at least one point spread function to result in at least resulting image, said resulting image being a more accurate image of said subject compared to said at least one image.

13. Computer readable media according to claim 11 wherein said segmenting comprises grouping image pixels into meaningful structures.