One-touch projector alignment for 3D stereo display

• US 8,310,525 B2
• Filed: 10/07/2008
• Issued: 11/13/2012
• Est. Priority Date: 10/07/2008
• Status: Expired due to Fees

• Current Assignee: Columbia Peak Ventures, LLC (Dominion Harbor Enterprises, LLC)
• Original Assignee: Seiko Epson Corporation (Seiko Group)
• Inventors: Tan, Kar-Han
• Primary Examiner(s): Gilles, Jude Jean
• Assistant Examiner(s): Means, Jaren M
• Application Number:
  US12/247,108
• Publication Number:
  US 20100085425A1
• Time in Patent Office: 1,498 Days
• Field of Search: 348/47
• US Class Current: 348/47
• CPC Class Codes: H04N 13/10 Processing, recording or transmission of stereoscopic or multi-view image signals
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/261 with monoscopic-to-stereoscopic image conversion
• CPC Class Codes: H04N 13/324 Colour aspects
• CPC Class Codes: H04N 13/327 Calibration thereof
• CPC Class Codes: H04N 13/337 using polarisation multiplexing
• CPC Class Codes: H04N 13/363 using image projection screens volumetric display H04N13/388
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

A 3D image is created by using two digital projectors having overlapping projection regions. The two digital projectors both emit polarized light, and are arranged at different angles to each other, such that individual images projected from respective digital projectors can be discerned by the use of polarized eyeglasses. Individualized light transport matrices are used to coordinate together the two digital projectors.

Claims

1. A method of creating a perspective image (3-Dimensional image or 3D image) as perceived from a predefined reference view point, comprising:

providing a first digital projector having a first projection region arranged along a first orientation with reference to a reference base;

providing a second digital projector having a second projection region arranged along a second orientation with reference to said reference base, said second projection region at least partially overlapping said first projection region;

defining a 3D imaging region within the overlapping area of said first projection region and said second projection region;

establishing a first light transport matrix T₁ covering the portion of the first projection region that encompasses said 3D imaging region, said first light transport matrix T₁ relating a first image c₁ as viewed from said predefined reference view point to a first projected image p₁ from said first digital projector according to relation c₁=T₁ p₁;

establishing a second light transport matrix T₂ covering the portion of the second projection region that encompasses said 3D imaging region, said second light transport matrix T₂ relating a second image c₂ as viewed from said predefined reference view point to a second projected image p₂ from said second digital projector according to relation c₂=T₂ p₂;

providing a first imaging sequence to said first digital projector, said first imaging sequence providing a first angled view of an image subject within said 3D imaging region with reference to said reference view point as defined by said first light transport matrix T₁;

providing a second imaging sequence to said second digital projector, said second imaging sequence providing a second angled view of said image subject within said 3D imaging region with reference to said reference view point as defined by said second light transport matrix T₂;

wherein establishing said first light transport matrix T₁, includes;

using a camera at said predefined reference view point, capturing a reference noise image C₀ of at least part of said first projection region under ambient light with no image projected from said first digital projector;

using said camera at said predefined reference view point, capturing a first image of a first test pattern projected by said first digital projector within said first projection region, said first test pattern being created by simultaneously activating a first group of projection pixels within said first digital projector, all projection pixels not in said first test pattern being turned off;

using said camera at said predefined reference view point, capturing a second image of a second test pattern projected by said first digital projector within said first projection region, said second test pattern being created by simultaneously activating a second group of projection pixels within said first digital projector, all projection pixels not in said second test pattern being turned off, wherein said first and second groups of projection pixels have only one projection pixel in common defining a target projection pixel;

subtracting said reference noise image C₀ from said first image to make a first corrected image;

subtracting said reference noise image C₀ from said second image to make a second corrected image;

comparing image pixels of said first corrected image to corresponding image pixels of said second corrected image and retaining the darker of two compared image pixels, the retained image pixels constituting a composite image;

identifying all none-dark image pixels in said composite image;

selectively identifying said none-dark pixels in said composite image as non-zero light transport coefficients associated with said target projection pixel; and

wherein identified non-dark image pixels are arranged into adjoining pixel cluster according to their pixel position relation within an image, and only pixels within a cluster having a size not less than a predefined number of pixels are identified as non-dark pixels.

2. The method of claim 1, wherein said first light transport matrix T₁ is constructed only for projector pixels of said first digital projector that contribute to said 3D imaging region.

3. The method of claim 1, further providing a pair of eyeglasses at said predefined reference view point, said pair of eyeglasses having:

a first polarized lens for capturing light from said first digital projector and rejecting light from said second digital projector; and

a second polarized lens for capturing light from said second digital projector and rejecting light from said second digital projector.

4. The method of claim 1, wherein said first orientation is at 90 degrees to said second orientation.

5. The method of claim 1, wherein:

said first digital projector has a first set of Red, Green, and Blue projection panels, two projection panels within said first set being polarized along a first polarization orientation with reference to said reference base, and a third projection panel within said first set being polarized along a second polarization orientation with reference to said reference base; and

said second digital projector has a second set of Red, Green, and Blue projection panels, two projection panels within said second set being polarized along said second polarization orientation with reference to said reference base, and a third projection panel within said second set being polarized along said first polarization orientation with reference to said reference base;

said method further comprising;

identifying first imaging information within said first imaging sequence corresponding to said third projection panel within said first set, and transferring to said second digital projector said first imaging information for projection by said second digital projector; and

identifying second imaging information within said second imaging sequence corresponding to said third projection panel within said second set, and transferring to said first digital projector said second imaging information for projection by said first digital projector.

6. The method of claim 1, wherein:

said first imaging sequence has R1 (Red

     1), G1 (Green

     1), and B1 (Blue

     1) imaging channels;

said second imaging sequence has R2 (Red

     2), G2 (Green

     2), and B2 (Blue

     2) imaging channels;

said method further comprising, swapping the R1 and R2 channels of said first and second imaging sequences.

7. The method of claim 1, wherein only pixels within said composite image having a light intensity value not less than a predefined minimum value are identified as said none-dark image pixels.

8. The method of claim 7, wherein said predefined minimum value is 25% of the maximum light intensity value.

9. The method of claim 1, wherein said predefined number of pixels is defined by the number of pixels within an average sized light footprint created on the camera-pixel-array within said camera created by turning ON a single projection pixel within said first digital projector.

10. The method of claim 1, wherein the centroid within a cluster having a size not less than said predefined number of pixels constitutes an estimate of the camera pixel array coordinate corresponding to said target projection pixel.

11. The method of claim 1, further comprising:

(a) defining a desired composite image “

c”

;

(b) setting p₂ equal to zero, solving for p₁ in formula p₁=T₁<sup>−

1</sup>−

T₂p₂);

(c) using the current solved value for p₁, solving for p₂ in formula p₂=T₂<sup>−

1</sup> (c−

T₁p₁);

(d) using the current solved value for p₂, solving for p₁ in formula p₁=T₁<sup>−

1</sup> (c−

T₂p₂); and

repeating steps (c) and (d) in sequence until p₁ converges to a first mosaicing image and p₂ converges to a second mosaicing image.

12. The method of claim 11, wherein said desired composite image “

c”

is defined as “

c”

=c₁+c₂.

13. The method of claim 11, wherein at least one of T₁⁻

1 or T₂<sup>−

1</sup> is an estimate of an inverse of a general light transport matrix T, said estimate being generated by;

identifying in turn, each column in said general light transport matrix T as a target column, calculating normalized values for not-nullified entry values in said target column with reference to said target column;

creating an intermediate matrix of equal size as said general light transport matrix T;

populating each column in said intermediate matrix with the calculated normalized values of its corresponding target column in said general light transport matrix T, each normalized value in each populated column in said intermediate matrix maintaining a one-to-one correspondence with said not-nullified entry values in its corresponding column in said general light transport matrix T;

applying a transpose matrix operation on said intermediate matrix.

14. The method of claim 13, wherein if said intermediate matrix is denoted as {hacek over (T)}, a target column in general light transport matrix T is denoted as Tr and a corresponding column in {hacek over (T)} is denoted as {hacek over (T)}r, then the construction and population of {hacek over (T)} is defined as {hacek over (T)}r=Tr/(∥

Tr∥

)² such that p₁=({hacek over (T)}₁)^T (c−

T₂p₂) if T₁<sup>−

1</sup> is the estimate of the inverse of general light transport matrix T<sup>−

1</sup> and p₂=({hacek over (T)}₂)^T (c−

T₁p₁) if T₂<sup>−

1</sup> is the estimate of the inverse of a general light transport matrix T<sup>−

1</sup>.

15. The method of claim 11, wherein at least one of T₁ or T₂ is an estimated light transport matrix generated from a sampling of light footprint information not having a one-on-one correspondence with every projector pixel in its respective digital projector.

16. The method of claim 1, wherein said reference noise image C₀ is an image capture of said 3D imaging region under ambient light with no image projected from any digital projector.

17. The method of claim 1, wherein said reference noise image C₀ is captured with said first and second projectors displaying solid black images.

18. A method of creating a perspective image (3-Dimensional image or 3D image) as perceived from a predefined reference view point, comprising:

providing a first digital projector having a first projection region arranged along a first orientation with reference to a reference base;

providing a second digital projector having a second projection region arranged along a second orientation with reference to said reference base, said second projection region at least partially overlapping said first projection region;

defining a 3D imaging region within the overlapping area of said first projection region and said second projection region;

establishing a first light transport matrix T₁ covering the portion of the first projection region that encompasses said 3D imaging region, said first light transport matrix T₁ relating a first image c₁ as viewed from said predefined reference view point to a first projected image p₁ from said first digital projector according to relation c₁=T₁ p₁;

establishing a second light transport matrix T₂ covering the portion of the second projection region that encompasses said 3D imaging region, said second light transport matrix T₂ relating a second image c₂ as viewed from said predefined reference view point to a second projected image p₂ from said second digital projector according to relation c₂=T₂ p₂;

providing a first imaging sequence to said first digital projector, said first imaging sequence providing a first angled view of an image subject within said 3D imaging region with reference to said reference view point as defined by said first light transport matrix T₁;

providing a second imaging sequence to said second digital projector, said second imaging sequence providing a second angled view of said image subject within said 3D imaging region with reference to said reference view point as defined by said second light transport matrix T₂;

wherein establishing said first light transport matrix T₁, includes;

using a camera at said predefined reference view point, capturing a first image of a first test pattern projected by said first digital projector within said first projection region, said first test pattern being created by simultaneously activating a first group of projection pixels within said first digital projector, all projection pixels not in said first test pattern being turned off;

using said camera at said predefined reference view point, capturing a second image of a second test pattern projected by said first digital projector within said first projection region, said second test pattern being created by simultaneously activating a second group of projection pixels within said first digital projector, all projection pixels not in said second test pattern being turned off, wherein said first and second groups of projection pixels have only one projection pixel in common defining a target projection pixel;

comparing image pixels of said first image to corresponding image pixels of said second image and retaining the darker of two compared image pixels, the retained image pixels constituting a composite image;

identifying all none-dark image pixels in said composite image;

selectively identifying said none-dark pixels in said composite image as non-zero light transport coefficients associated with said target projection pixel;

wherein identified non-dark image pixels are arranged into adjoining pixel cluster according to their pixel position relation within an image, and only pixels within a cluster having a size not less than a predefined number of pixels are identified as non-dark pixels.

19. The method of claim 18, wherein said predefined number of pixels is defined by the number of pixels within an average sized light footprint created on the camera-pixel-array within said camera created by turning ON a single projection pixel within said first digital projector.

20. The method of claim 18, wherein the centroid within a cluster having a size not less than said predefined number of pixels constitutes an estimate of the camera pixel array coordinate corresponding to said target projection pixel.