Formation of three-dimensional video sequences with a scannerless range imaging system

• US 6,856,382 B2
• Filed: 02/06/2003
• Issued: 02/15/2005
• Est. Priority Date: 02/06/2003
• Status: Active Grant

• Current Assignee: Monument Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Eastman Kodak Company
• Inventors: Ray, Lawrence A., Cahill, Nathan D.
• Primary Examiner(s): Gregory, Bernarr E.
• Assistant Examiner(s): Andrea, Brian K
• Application Number:
  US10/360,380
• Publication Number:
  US 20040156034A1
• Time in Patent Office: 740 Days
• Field of Search: 356/5.04, 348/31, 382/154
• US Class Current: 356/5.04
• CPC Class Codes: G06T 7/521 from laser ranging, e.g. using interferometry; from the projection of structured light

Abstract

A method for generating a range image sequence from a scannerless range imaging system of the type comprising an illuminator for illuminating a scene with modulated illumination; an image modulating component for receiving and modulating the modulated illumination reflected from the scene; an image capture element for capturing images output by the image modulating component, including an image bundle of at least three phase offset images each incorporating a phase delay corresponding to the distance of objects in the scene from the illuminator, wherein each phase offset image of an image bundle also incorporates a phase offset distinct for each image. According to the method, a sequence of three or more phase offset images are acquired that correspond to the modulated illumination reflected from the scene, whereby the sequence contains overlapping subsequences of successive phase offset images, each subsequence forming an image bundle. A range image is then computed using the phase offset images from each corresponding image bundle, thereby forming a range image sequence.

Claims

1. A method for generating a range video sequence from a scannerless range imaging system of the type comprising an illuminator for illuminating a scene with modulated illumination;

an image modulating component for receiving and modulating the modulated illumination reflected from the scene;

an image capture element for capturing frames output by the image modulating component, said method comprising the steps of;

defining a frame rate, wherein the frame rate is the number of frames of the range video sequence per unit of time;

illuminating a scene with sinusoidally amplitude modulated illumination;

receiving and sinusoidally amplitude modulating the sinusoidally amplitude modulated illumination reflected from the scene;

acquiring, for each of the frames, three or more successive images of the scene to define an image bundle;

during said acquiring, shifting the phase of said illuminating relative to said modulating, wherein, in each said image bundle, respective said images each have a different phase offset; and

computing for each said image bundle a range image using the corresponding phase offset images, thereby forming a range video sequence at the defined frame rate.

2. The method as claimed in claim 1 wherein the step of computing a range image comprises using a system of linear equations at each point in each frame, wherein the system is characterized by a matrix of distinct phase offsets that is pre-computed for the range video sequence and used repeatedly for computing each range image.

3. The method as claimed in claim 2 wherein the distinct phase offsets are given by α

={α

₀,α

₁, . . . ,α

<sub>k−

1</sub>} and the matrix is a (k×

3) matrix B given by;

$B=\left(\begin{array}{ccc}1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_0& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_0\\ 1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_1& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_1\\ \mathrm{⋮<br><br>}& \mathrm{⋮<br><br>}& \mathrm{⋮<br><br>}\\ 1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_{k-1}& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_{k-1}\end{array}\right).$

4. A method for generating a range video sequence from a scannerless range imaging system of the type comprising an illuminator for illuminating a scene with modulated illumination;

an image modulating component for receiving and modulating the modulated illumination reflected from the scene;

an image capture element for capturing images output by the image modulating component, said method comprising the steps of;

illuminating a scene with sinusoidally amplitude modulated illumination;

receiving and sinusoidally amplitude modulating the sinusoidally amplitude modulated illumination reflected from the scene;

acquiring a sequence of images of the scene, wherein the sequence contains a plurality of overlapping subsequences of at least three successive images, each of the subsequences defining an image bundle;

during said acquiring, shifting the phase of said illuminating relative to said modulating, wherein said images each have a different phase offset; and

computing for each said image bundle a range image using the corresponding phase offset images, thereby forming a range video sequence.

5. A method for generating a range video sequence from a scannerless range imaging system of the type comprising an illuminator for illuminating a scene with modulated illumination;

an image modulating component for receiving and modulating the modulated illumination reflected from the scene;

an image capture element for capturing images output by the image modulating component, said method comprising the steps of;

illuminating a scene with sinusoidally amplitude modulated illumination;

receiving and modulating the sinusoidally amplitude modulated illumination reflected from the scene;

acquiring a sequence of successive images of the scene;

during said acquiring, shifting the phase of said illuminating relative to said modulating, wherein said images each have a different phase offset;

obtaining overlapping subsequences of phase offset images from the sequence of successive phase offset images, each of the subsequences defining an image bundle of at least three phase offset images each incorporating a distinct phase offset, wherein each image bundle contains at least one phase offset image that is shared with at least one other image bundle from an overlapping one of the subsequences; and

computing for each said image bundle a range image using the corresponding phase offset images, thereby forming a range video sequence.

6. The method as claimed in claim 5 wherein the step of acquiring further comprises the steps of choosing a set of three or more distinct phase offsets and applying the set of three or more distinct offsets to the successive phase offset images in a repetitive sequence.

7. The method as claimed in claim 5 wherein the step of computing a range image utilizes a plurality of distinct computations in computing a range image for an image bundle, and at least one of the distinct computations can be reused in computing a range image for at least one other image bundle.

8. The method as claimed in claim 7 wherein the phase offsets are given by θ

={θ

₀, θ

₁, . . . , θ

<sub>n−

1</sub>} and the step of computing a range image comprises uses a system of linear equations at each point P in each phase offset image, $\left(\begin{array}{c}{P}_0\\ P_1\\ \mathrm{⋮<br><br>}\\ P_{n-1}\end{array}\right)=\left(\begin{array}{ccc}1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\theta <br><br>}}_0& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\theta <br><br>}}_0\\ 1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\theta <br><br>}}_1& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\theta <br><br>}}_1\\ \mathrm{⋮<br><br>}& \mathrm{⋮<br><br>}& \mathrm{⋮<br><br>}\\ 1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\theta <br><br>}}_{n-1}& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\theta <br><br>}}_{n-1}\end{array}\right)<br><br>\left(\begin{array}{c}{\mathrm{\Lambda <br><br>}}_0\\ {\mathrm{\Lambda <br><br>}}_1\\ {\mathrm{\Lambda <br><br>}}_2\end{array}\right)$

where this system is solved by a singular value decomposition to yield the vector Λ

=[Λ

₀,Λ

₁,Λ

₂]^T.

9. The method as claimed in claim 8 wherein the distinct computation that is reused is one or more of the components of the vector Λ

=[Λ

₀,Λ

₁,Λ

₂]^T.

10. The method as claimed in claim 5 wherein the step of computing a range image comprises using a system of linear equations at each point in each image, wherein the system is characterized by a matrix of distinct phase offsets that is pre-computed for the range video sequence and used repeatedly for computing each range image.

11. The method as claimed in claim 9 wherein the distinct phase offsets are given by a α

={α

₀,α

₁, . . . ,α

<sub>k−

1</sub>} and the matrix is a (k×

3) matrix B given by;

$B=\left(\begin{array}{ccc}1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_0& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_0\\ 1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_1& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_1\\ \mathrm{⋮<br><br>}& \mathrm{⋮<br><br>}& \mathrm{⋮<br><br>}\\ 1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_{k-1}& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_{k-1}\end{array}\right).$

12. A method for generating a range video sequence from a scannerless range imaging system comprising the steps of:

illuminating a scene with sinusoidally amplitude modulated illumination;

receiving and sinusoidally amplitude modulating the sinusoidally amplitude modulated illumination reflected from the scene;

capturing a plurality of sequential frames at a defined rate per unit time, each said frame including an image bundle of at least three images;

during said capturing, shifting the phase of said illuminating relative to said modulating, wherein said images each have a different phase offset; and

computing for each image bundle a range image using the corresponding phase offset images to form a range video sequence at the defined frame rate.

13. The method as claimed in claim 12 wherein the step of computing a range image comprises using a system of linear equations at each point in each frame, wherein the system is characterized by a matrix of distinct phase offsets that is pre-computed for the range video sequence and used repeatedly for computing each range image.

14. The method as claimed in claim 12 wherein the distinct phase offsets are given by α

={α

₀,α

₁, . . . ,α

<sub>k−

1</sub>} and the matrix is a (k×

3) matrix B given by;

$B=\left(\begin{array}{ccc}1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_0& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_0\\ 1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_1& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_1\\ \mathrm{⋮<br><br>}& \mathrm{⋮<br><br>}& \mathrm{⋮<br><br>}\\ 1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_{k-1}& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_{k-1}\end{array}\right).$

15. A method for generating a range video sequence from a scannerless range imaging system comprising the steps of:

illuminating a scene with sinusoidally amplitude modulated illumination;

receiving and sinusoidally amplitude modulating the sinusoidally amplitude modulated illumination reflected from the scene;

capturing a sequence of images at a defined rate per unit time;

during said capturing, shifting the phase of said illuminating relative to said modulating, wherein said images each have a different phase offset;

obtaining overlapping subsequences of successive phase offset images from the sequence of images, each of the subsequences defining an image bundle; and

computing for each image bundle a range image using corresponding phase offset images to form a range video sequence.

16. The method as claimed in claim 15 wherein the step of computing a range image comprises using a system of linear equations at each point in each frame, wherein the system is characterized by a matrix of distinct phase offsets that is pre-computed for the range video sequence and used repeatedly for computing each range image.

17. The method as claimed in claim 16 wherein the distinct phase offsets are given by α

={α

₀,α

₁, . . . ,α

<sub>k−

1</sub>} and the matrix is a (k×

3) matrix B given by;

$B=\left(\begin{array}{ccc}1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_0& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_0\\ 1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_1& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_1\\ \mathrm{⋮<br><br>}& \mathrm{⋮<br><br>}& \mathrm{⋮<br><br>}\\ 1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_{k-1}& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_{k-1}\end{array}\right).$

18. A method for generating a range video sequence comprising:

illuminating a scene with sinusoidally amplitude modulated illumination;

receiving and sinusoidally amplitude modulating the sinusoidally amplitude modulated illumination reflected from the scene;

capturing a sequence of images at a defined rate per unit time;

during said capturing, shifting the phase of said illuminating relative to said modulating, wherein said images each have a different phase offset;

obtaining overlapping subsequences of phase offset images from the sequence of images, each of the subsequences defining an image bundle of at least three phase offset images each incorporating a distinct phase offset, wherein each image bundle contains at least one phase offset image that is shared with at least one other image bundle from an overlapping one of the subsequences; and

computing for each image bundle a range image using corresponding phase offset images to form a range video sequence.

19. The method as claimed in claim 18 wherein said capturing further comprises choosing a set of three or more distinct phase offsets and applying the set of three or more distinct offsets to the successive phase offset images in a repetitive sequence.

20. The method as claimed in claim 18 wherein the step of computing a range image utilizes a plurality of distinct computations in computing a range image for an image bundle, and at least one of the distinct computations can be reused in computing a range image for at least one other image bundle.

21. The method as claimed in claim 20 wherein the phase offsets are given by θ

={θ

₀, θ

₁, . . . , θ

<sub>n−

1</sub>} and the step of computing a range image comprises uses a system of linear equations at each point P in each phase offset image, $\left(\begin{array}{c}{P}_0\\ P_1\\ \mathrm{⋮<br><br>}\\ P_{n-1}\end{array}\right)=\left(\begin{array}{ccc}1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\theta <br><br>}}_0& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\theta <br><br>}}_0\\ 1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\theta <br><br>}}_1& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\theta <br><br>}}_1\\ \mathrm{⋮<br><br>}& \mathrm{⋮<br><br>}& \mathrm{⋮<br><br>}\\ 1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\theta <br><br>}}_{n-1}& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\theta <br><br>}}_{n-1}\end{array}\right)<br><br>\left(\begin{array}{c}{\mathrm{\Lambda <br><br>}}_0\\ {\mathrm{\Lambda <br><br>}}_1\\ {\mathrm{\Lambda <br><br>}}_2\end{array}\right)$

where this system is solved by a singular value decomposition to yield the vector Λ

=[Λ

₀,Λ

₁,Λ

₂]^T.

22. The method as claimed in claim 21 wherein the distinct computation that is reused is one or more of the components of the vector Λ

=[Λ

₀,Λ

₁,Λ

₂]^T.

23. The method as claimed in claim 18 wherein the step of computing a range image comprises using a system of linear equations at each point in each image, wherein the system is characterized by a matrix of distinct phase offsets that is pre-computed for the range video sequence and used repeatedly for computing each range image.

24. The method as claimed in claim 23 wherein the distinct phase offsets are given by α

={α

₀,α

₁, . . . ,α

<sub>k−

1</sub>} and the matrix is a (k×

3) matrix B given by;

$B=\left(\begin{array}{ccc}1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_0& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_0\\ 1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_1& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_1\\ \mathrm{⋮<br><br>}& \mathrm{⋮<br><br>}& \mathrm{⋮<br><br>}\\ 1& \cos <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_{k-1}& -\sin <br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\alpha <br><br>}}_{k-1}\end{array}\right).$