Projector-camera misalignment correction for structured light systems

• US 9,256,943 B2
• Filed: 03/27/2014
• Issued: 02/09/2016
• Est. Priority Date: 03/28/2013
• Status: Active Grant

• Current Assignee: Texas Instruments, Inc.
• Original Assignee: Texas Instruments, Inc.
• Inventors: Appia, Vikram VijayanBabu, Pekkucuksen, Ibrahim Ethem
• Primary Examiner(s): MARIAM, DANIEL G
• Application Number:
  US14/228,236
• Publication Number:
  US 20140294290A1
• Time in Patent Office: 684 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06T 7/80 Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Abstract

A method of misalignment correction in a structured light device is provided that includes extracting features from a first captured image of a scene, wherein the first captured image is captured by an imaging sensor component of the structured light device, and wherein the first captured image includes a pattern projected into the scene by a projector component of the structured light device, matching the features of the first captured image to predetermined features of a pattern image corresponding to the projected pattern to generate a dataset of matching features, determining values of alignment correction parameters of an image alignment transformation model using the dataset of matching features, and applying the image alignment transformation model to a second captured image using the determined alignment correction parameter values.

Claims

1. A method of misalignment correction in a structured light device, the method comprising:

extracting features from a first captured image of a scene, wherein the first captured image is captured by an imaging sensor component of the structured light device, and wherein the first captured image comprises a pattern projected into the scene by a projector component of the structured light device;

matching the features of the first captured image to predetermined features of a pattern image corresponding to the projected pattern to generate a dataset of matching features;

determining values of alignment correction parameters of an image alignment transformation model using the dataset of matching features; and

applying the image alignment transformation model to a second captured image using the determined alignment correction parameter values.

2. The method of claim 1, wherein the first captured image and the second captured image are a same captured image.

3. The method of claim 1, wherein the image alignment transformation model is an affine transformation model modified to remove a component corresponding to a separation baseline of the imaging sensor component and the projector component.

4. The method of claim 1, wherein the image alignment transformation model is a perspective transformation model modified to remove a component corresponding to a separation baseline of the imaging sensor component and the projector component.

5. The method of claim 1, wherein the predetermined features of the pattern image are a subset of a total number of features of the pattern image.

6. The method of claim 1, wherein the extracting, matching, and determining are performed at intervals during operation of the structured light device.

7. The method of claim 1, wherein the pattern is a two-dimensional high frequency binary pattern.

8. A structured light device comprising:

an imaging sensor component configured to capture images of a scene;

a projector component configured to project a pattern into the scene;

a memory configured to store predetermined features of a pattern image corresponding to the pattern;

means for extracting features from a first image of the scene captured by the imaging sensor component, wherein the first image comprises the pattern projected into the scene by the projector component;

means for matching the features of the first image to the predetermined features to generate a dataset of matching features;

means for determining values of alignment correction parameters of an image alignment transformation model using the dataset of matching features; and

means for applying the image alignment transformation model with the determined alignment correction parameter values to a second image of the scene captured by the imaging sensor component.

9. The structured light device of claim 8, wherein the first image and the second image are a same image.

10. The structured light device of claim 8, wherein the image alignment transformation model is an affine transformation model modified to remove a component corresponding to a separation baseline of the imaging sensor component and the projector component.

11. The structured light device of claim 8, wherein the image alignment transformation model is a perspective transformation model modified to remove a component corresponding to a separation baseline of the imaging sensor component and the projector component.

12. The structured light device of claim 8, wherein the predetermined features of the pattern image are a subset of a total number of features of the pattern image.

13. The structured light device of claim 8, wherein the extracting, matching, and determining are performed at intervals during operation of the structured light device.

14. The structured light device of claim 8, wherein the pattern is a two-dimensional high frequency binary pattern.

15. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor in a structured light device, cause a method of misalignment correction to be performed, the method comprising:

extracting features from a first captured image of a scene, wherein the first captured image is captured by an imaging sensor component of the structured light device, and wherein the first captured image comprises a pattern projected into the scene by a projector component of the structured light device;

matching the features of the first captured image to predetermined features of a pattern image corresponding to the projected pattern to generate a dataset of matching features;

determining values of alignment correction parameters of an image alignment transformation model using the dataset of matching features; and

applying the image alignment transformation model to a second captured image using the determined alignment correction parameter values.

16. The computer-readable medium of claim 15, wherein the first captured image and the second captured image are a same captured image.

17. The computer-readable medium of claim 15, wherein the image alignment transformation model is an affine transformation model modified to remove a component corresponding to a separation baseline of the imaging sensor component and the projector component.

18. The computer-readable medium of claim 15, wherein the image alignment transformation model is a perspective transformation model modified to remove a component corresponding to a separation baseline of the imaging sensor component and the projector component.

19. The computer-readable medium of claim 15, wherein the predetermined features of the pattern image are a subset of a total number of features of the pattern image.

20. The computer-readable medium of claim 15, wherein the extracting, matching, and determining are performed at intervals during operation of the structured light device.