Invisible or low perceptibility of image alignment in dual projection systems

• US 10,326,968 B2
• Filed: 10/19/2012
• Issued: 06/18/2019
• Est. Priority Date: 10/20/2011
• Status: Active Grant

• Current Assignee: IMAX Corporation
• Original Assignee: IMAX Corporation
• Inventors: Iversen, Steen Svendstorp, Tan, Weining, O'Dor, Matthew
• Primary Examiner(s): Le, Bao-Luan Q
• Application Number:
  US14/351,839
• Publication Number:
  US 20140292817A1
• Time in Patent Office: 2,433 Days
• CPC Class Codes: G03B 17/54 with projector
• CPC Class Codes: G03B 35/20 using two or more projectors
• CPC Class Codes: G06T 3/20 Linear translation of whole images or parts thereof, e.g. panning
• CPC Class Codes: H04N 13/207 using a single 2D image sensor
• CPC Class Codes: H04N 13/236 using varifocal lenses or mirrors
• CPC Class Codes: H04N 13/246 Calibration of cameras
• CPC Class Codes: H04N 13/327 Calibration thereof
• CPC Class Codes: H04N 13/363 using image projection screens volumetric display H04N13/388
• CPC Class Codes: H04N 9/3147 Multi-projection systems displays in general H04N9/12; video walls G06F3/1446, G09G2300/026
• CPC Class Codes: H04N 9/317 Convergence or focusing systems electronic adjustment of convergence H04N9/3185; convergence or focusing arrangements for cathode ray tubes H04N9/28; means for automatic focusing of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/53
• CPC Class Codes: H04N 9/3185 Geometric adjustment, e.g. keystone or convergence optical or mechanical adjustment of convergence H04N9/317; using scanning means H04N3/22; optical or mechanical adjustments of projectors not peculiar to the presence of an electronic spatial light modulator G03B21/14
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

A dual projection system can align displayed images using an alignment pattern. The alignment pattern can be used to modify part of an image content frame of an image data sequence. Two image data sequences can be spatially aligned based on the modification to part of the image content frame using the alignment pattern. An image content frame may be warped and displayed. The displayed warped image content frame can be captured. A set of transformation vectors can be determined based on the captured image content frame and alignment image data. Stored transformation data can be updated using the set of transformation vectors and the updated transformation data can be used to spatially align two image data sequences.

Claims

1. A method comprising:

receiving a first digital image sequence representing a visual presentation and comprising a plurality of frames that include image content;

receiving an alignment feature that is separate from the first digital image sequence and the visual presentation represented by the first digital image sequence;

generating modified frames by a processor in a projection system modifying at least some frames of the plurality of frames to represent at least part of the alignment feature but retaining part of the image content in the at least some frames that represent a portion of the visual presentation, while maintaining as unmodified with part or all of the alignment feature the image content in other frames of the plurality of frames, the modified frames being positioned in the first digital image sequence such that a first set of unmodified frames precedes the modified frames in the first digital image sequence and a second set of unmodified frames is subsequent to the modified frames in the first digital image sequence, wherein the alignment feature is divided into a plurality of alignment portions such that each modified frame is modified to include a different portion of the alignment feature in a different position of the modified frame as compared to other of the modified frames and, together, the modified frames include all of the alignment feature; and

spatially aligning the first digital image sequence and a second digital image sequence, during the visual presentation, based on the modified frames in projecting the first digital image sequence by a first projector and the second digital image sequence by a second projector, to reduce an effect of spatial changes in image alignment between projected images from the first digital image sequence and the second digital image sequence during the visual presentation.

2. The method of claim 1, wherein generating the modified frames comprises modifying at least one color channel of the at least some frames using data from the part of the alignment feature.

3. The method of claim 1, wherein the different portion of the alignment feature has a random spatial sequence among the modified frames.

4. The method of claim 1, wherein generating the modified frames comprises removing a portion of each frame of the at least some frames and adding the different portion of the plurality of alignment features to replace the portion in the frame, wherein the image content of the modified frames includes black image pixels around the part of the alignment feature.

5. The method of claim 4, wherein the plurality of alignment portions has an average illumination level that substantially corresponds to the average illumination level of the portions of the modified frames that are removed.

6. The method of claim 4, wherein the plurality of alignment portions has an average color balance corresponding to the average color balance of the portions of the modified frame that are removed.

7. The method of claim 4, wherein the plurality of alignment portions has a Gaussian illumination profile.

8. The method of claim 1, further comprising:

capturing, as a captured second image sequence frame, a displayed second digital image sequence frame modified with alignment image data that comprises the alignment feature;

capturing, as a captured first image sequence frame, a displayed first digital image sequence modified with the alignment image data;

determining a set of transformation vectors based on a spatial difference between the alignment image data in the captured second image sequence frame and the alignment image data in the captured first image sequence frame; and

spatially aligning a displayed first image of the first digital image sequence and a displayed second image of the second digital image sequence using the set of transformation vectors.

9. The method of claim 1, wherein the plurality of alignment portions has content of at least one color channel, wherein the modified frames include image content pixels around the plurality of alignment portions of the alignment feature, the image content pixels being lower frequency image content of the at least one color channel than the content of the plurality of alignment portions of the alignment feature.

10. The method of claim 1, wherein generating the modified frames comprises changing pixel values of pixels in the frame based on alignment pixel values of alignment pixels in the alignment feature, the alignment pixels and the pixels having corresponding frame locations.

11. A method, comprising:

during display of a visual presentation in a theatre environment,

capturing, as captured first image sequence frames, displayed first image sequence content frames having image content modified with alignment data of an alignment feature by a projection system modifying at least some of the first image sequence content frames to represent part of the alignment feature but retaining part of the image content that represents the visual representation in the at least some of the first image sequence content frames, while maintaining as unmodified with part or all of the alignment feature the image content in other frames of the first image sequence content frames, wherein modified first image sequence content frames are positioned in the first image sequence content frames such that a first set of unmodified first image sequence content frames precedes the modified first image sequence content frames in the first image sequence content frames and a second set of unmodified first image sequence content frames is subsequent to the modified first image sequence content frames in the first image sequence content frames, wherein the alignment feature is divided into a plurality of alignment portions such that each modified first image sequence frame is modified to include a different portion of the alignment feature in a different position of the modified first image sequence frame as compared to other of the modified first image sequence frames and, together, the modified first image sequence frames include all of the alignment feature;

capturing, as captured second image sequence frames, displayed second image sequence frames having image content modified with the alignment data by the projection system modifying at least some of the second image sequence content frames to represent the part of the alignment feature but retaining the part of the image content that represents the visual representation in the at least some of the second image sequence content frames, while maintaining as unmodified with part or all of the alignment feature the image content in other frames of the second image sequence content frames, wherein modified second image sequence frames are positioned in the second image sequence frames such that a first set of unmodified second image sequence frames precedes the modified second image sequence frames in the second image sequence frames and a second set of unmodified second image sequence frames is subsequent to the modified second image sequence frames in the second image sequence frames, each modified second image sequence frame is modified to include a different portion of the alignment feature in a different position of the modified second image sequence frame as compared to other of the modified second image sequence frames and, together, the modified second image sequence frames include all of the alignment feature;

determining a set of transformation vectors based on a spatial difference between the alignment data in the captured second image sequence frames and the alignment data in the captured first image sequence frames; and

spatially aligning a displayed first image of a first image sequence and a displayed second image of a second image sequence using the set of transformation vectors to reduce an effect of spatial changes in image alignment between projected images from the first image sequence and the second image sequence.

12. The method of claim 11, further comprising:

modifying image content in the first image sequence content frames and the second image sequence content frames by the projection system replacing part of the first image sequence and part of the second image sequence with the alignment data to generate the modified first image sequence frames and the modified second image sequence frames;

warping the modified second image sequence frames with stored transformation data to generate warped second image sequence frames;

displaying the warped second image sequence frames as the displayed second image sequence frames and displaying the modified first image sequence frames as the displayed first image sequence frames; and

updating the stored transformation data with the set of transformation vectors to produce updated transformation data,

wherein spatially aligning the displayed first image of the first image sequence and the displayed second image of the second image sequence using the set of transformation vectors comprises using the updated transformation data.

13. The method of claim 12, wherein modifying image content in the first image sequence content frames and the second image sequence content frames comprises:

accessing the alignment data comprising an alignment pattern having at least one alignment feature; and

modifying a portion of image content in a frame in the first image sequence by the projection system replacing the portion of the first image sequence with the different portion of the alignment feature, wherein the first image sequence and the second image sequence each comprise digital image data,

wherein spatially aligning the displayed first image of the first image sequence and the displayed second image of the second image sequence using the set of transformation vectors comprises spatially aligning the first image sequence and the second image sequence based on the portion modified by the projection system when displaying the first image sequence and the second image sequence.

14. The method of claim 12, wherein modifying the image content in the first image sequence content frames and the second image sequence content frames comprises:

accessing the alignment data comprising an alignment pattern;

modifying a first portion of image content in a frame of the first image sequence using part of the alignment pattern; and

modifying, using another part of the alignment pattern, a second portion of image content in a subsequent frame of the first image sequence that is different than the first portion,

wherein the frame with the first portion modified using the part of the alignment pattern and the subsequent frame with the second portion modified using another part of the alignment pattern comprise all of the alignment pattern.

15. The method of claim 12, wherein modifying the image content in the first image sequence content frames and the second image sequence content frames comprises:

watermarking the alignment data onto image data in the first image sequence and the second image sequence by one of;

spatial encoding to watermark the alignment data onto the image data;

or

time domain encoding to watermark the alignment data onto the image data, the method further comprising;

extracting the alignment data from at least one of the captured second image sequence frames or the captured first image sequence frames by identifying the alignment data watermarked into the first image sequence or the second image sequence.

16. The method of claim 11, wherein determining the set of transformation vectors based on the spatial difference between the alignment data in the captured second image sequence frames and the captured first image sequence frames comprises:

determining a first set of transformation vectors based on a difference between the alignment data and the captured second image sequence frames including a modification using the alignment data; and

determining a second set of transformation vectors based on a difference between the alignment data and the captured first image sequence frames including a modification using the alignment data,

wherein spatially aligning the displayed first image of the first image sequence and the displayed second image of the second image sequence using the set of transformation vectors comprises spatially aligning the displayed first image of the first image sequence and the displayed second image of the second image sequence using the first set of transformation vectors and the second set of transformation vectors.

17. The method of claim 11, further comprising:

determining a centroid of alignment features of the captured first image sequence frames and the captured second image sequence frames using a low resolution image sensor.

24. The method of claim 11, wherein the alignment data includes an alignment pattern, the method further comprising:

randomly displaying different features of the alignment pattern among multiple frames.

18. A dual display projection system, comprising:

a server comprising;

a memory unit for storing a first image sequence and alignment data that is an alignment pattern having at least one alignment feature and that is separate from the first image sequence, the first image sequence representing a visual presentation and comprising a plurality of frames with image content; and

a processor device configured for;

generating modified frames for the first image sequence by modifying at least some frames of the plurality of frames to represent at least part of the alignment feature but retaining part of the image content that represents a portion of the visual presentation in the frames, while maintaining as unmodified with part or all of the alignment feature the image content in other frames of the plurality of frames, the modified frames being positioned in the first image sequence such that a first set of unmodified frames precedes the modified frames in the first image sequence and a second set of unmodified frames is subsequent to the modified frames in the first image sequence, wherein the processor device is configured to divide the alignment feature into a plurality of alignment portions such that each modified frame is modified to include a different portion of the alignment feature in a different position of the modified frame as compared to other of the modified frames and, together, the modified frames include all of the alignment feature; and

spatially aligning the first image sequence and a second image sequence, during the visual presentation, based on the modified frames in displaying the first image sequence by a first display device and displaying the second image sequence by a second device, to reduce an effect of spatial changes in image alignment between projected images from the first image sequence and the second image sequence during the visual presentation.

19. The system of claim 18, wherein the dual display projection system is configured for randomly displaying different features of the alignment pattern among multiple frames.

20. The dual display projection system of claim 18, wherein the processor device is adapted for:

modifying image content in frames of the second image sequence with the alignment data that includes the alignment pattern by replacing part of the image content in the frames of the second image sequence with the alignment feature to create modified second image frames; and

warping the modified second image frames with stored transformation data to generate warped second image frames,

wherein the dual display projection system further comprises;

the first display device for displaying the warped second image frames;

the second display device for displaying the modified frames in the first image sequence;

an image sensor for capturing the displayed warped second image frames and the displayed modified frames as a captured second image frames and a captured first image frames; and

a calibration circuit for determining a set of transformation vectors based on a spatial difference between the alignment data in the captured second image frames and the captured first image frames,

wherein the processor device is adapted for updating the stored transformation data with the set of transformation vectors to produce updated transformation data and for spatially aligning a displayed first image of the first image sequence and a displayed second image of the second image sequence using the updated transformation data.

21. The dual display projection system of claim 20, wherein the image sensor is adapted for being synchronized for capturing the displayed warped second image content frames and the displayed modified frames.

22. The dual display projection system of claim 20, wherein the image sensor is a camera.

23. The dual display projection system of claim 20, wherein the first image sequence and the second image sequence are stereographic image sequences.