Determining correspondence mappings from infrared patterns projected during the projection of visual content

• US 9,052,575 B2
• Filed: 07/12/2007
• Issued: 06/09/2015
• Est. Priority Date: 07/12/2007
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Chang, Nelson Liang An, Damera-Venkata, Niranjan, Widdowson, Simon
• Primary Examiner(s): Epps, Georgia Y
• Assistant Examiner(s): Wyatt, Kevin
• Application Number:
  US11/776,796
• Publication Number:
  US 20090015791A1
• Time in Patent Office: 2,889 Days
• Field of Search: 359/630, 348/370, 348/164, 353/28, 353/6, 353/69, 353/94
• US Class Current: 1/1
• CPC Class Codes: G03B 21/00 Projectors or projection-type viewers; Accessories therefor devices for changing pictures G03B23/00
• CPC Class Codes: G03B 21/005 Projectors using an electronic spatial light modulator but not peculiar thereto
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 11/006 Inverse problem, transformation from projection-space into object-space, e.g. transform methods, back-projection, algebraic methods
• CPC Class Codes: G06T 3/00 Geometric image transformations in the plane of the image
• CPC Class Codes: G06T 3/08 Projecting images onto non-planar surfaces, e.g. geodetic screens
• CPC Class Codes: G06T 3/10 Selection of transformation methods according to the characteristics of the input images
• CPC Class Codes: G06V 10/62 relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking

Abstract

A method performed by a display system is provided. The method includes projecting a first infrared pattern from a first projection plane of a first projector into a scene, capturing the first infrared pattern from the scene in a capture plane of at least one image capture device, and determining a first correspondence mapping between the first projector and the image capture device from at least the first infrared pattern in the first projection plane and the first infrared pattern in the capture plane.

Claims

1. A method performed by a display system, the method comprising:

providing, to a first projector, an input of a first infrared pattern combined with a first portion of visual content;

projecting, with the first projector, the first portion of the visual content onto a display surface for viewing by a human observer;

projecting, with the first projector, the first infrared pattern from a first projection plane of the first projector onto the display surface during the projection of the first portion of the visual content by the first projector;

capturing the first infrared pattern from the display surface in a capture plane of at least one image capture device having an infrared filter configured to transmit the first infrared pattern and block the visual content; and

determining a first correspondence mapping between the first projector and the image capture device from at least the first infrared pattern in the first projection plane and the first infrared pattern in the capture plane.

2. The method of claim 1 further comprising:

providing, to a second projector, an input of a second infrared pattern combined with a second portion of the visual content;

projecting, with the second projector, the second portion of the visual content onto the display surface for viewing by the human observer;

projecting, with the second projector, the second infrared pattern from a second projection plane of the second projector onto the display surface during the projection of the second portion of the visual content by the second projector;

capturing the second infrared pattern from the display surface in the capture plane of the image capture device, the infrared filter configured to transmit the second infrared pattern; and

determining a second correspondence mapping between the second projector and the image capture device from at least the second infrared pattern in the second projection plane and the second infrared pattern in the capture plane.

3. The method of claim 2 further comprising:

projecting the first infrared pattern onto a region of the display surface at a first time; and

projecting the second infrared pattern onto at least a portion of the region at a second time that is subsequent to the first time.

4. The method of claim 2 further comprising:

remapping the first and the second correspondence mappings to a reference coordinate system.

5. The method of claim 1 further comprising:

generating a frame for display onto the display surface by the first projector with the first correspondence mapping.

6. The method of claim 1 wherein the first infrared pattern is substantially visually imperceptible to the human observer while viewing the visual content.

7. The method of claim 1 wherein the first infrared pattern includes a space-time code.

8. The method of claim 1 wherein the input of the first infrared pattern combined with the first portion of visual content comprises a single input to the first projector.

9. An image display system comprising:

a processing system;

a first projector with a first projection plane;

at least one image capture device with a capture plane and an infrared filter configured to transmit infrared light and block visual content;

wherein the first projector is configured to project a first portion of visual content onto a display surface for viewing by a human observer,

wherein the first projector is configured to project a first pattern onto the display surface with first infrared light during the projection of the first portion of the visual content by the first projector,

wherein the infrared filter is configured to transmit the first infrared light and the image capture device is configured to capture the first pattern from the display surface,

wherein the processing system is configured to determine a first correspondence mapping between the first projector and the image capture device from at least the first pattern in the first projection plane and the first pattern in the capture plane,

wherein the processing system is configured to provide to the first projector a combined input of the first pattern with the first portion of the visual content.

10. The image display system of claim 9 wherein the first projector includes a color wheel with a region configured to transmit the first infrared light.

11. The image display system of claim 9 wherein the first projector includes a color wheel with a plurality of blanking regions that are configured to transmit the first infrared light.

12. The image display system of claim 9 wherein the first projector includes a controllable infrared blocking filter.

13. The image display system of claim 9 further comprising:

a second projector with a second projection plane;

wherein the second projector is configured to project a second portion of the visual content onto the display surface for viewing by the human observer,

wherein the second projector is configured to project a second pattern onto the display surface with second infrared light during the projection of the second portion of the visual content by the second projector,

wherein the infrared filter is configured to transmit the second infrared light and the image capture device is configured to capture the second pattern from the display surface,

wherein the processing system is configured to determine a second correspondence mapping between the second projector and the image capture device from at least the second pattern in the second projection plane and the second pattern in the capture plane, and

wherein the processing system is configured to provide to the second projector a combined input of the second pattern with the second portion of the visual content.

14. The image display system of claim 9 wherein the combined input of the first pattern with the first portion of the visual content comprises a single input to the first projector.

15. A non-transitory computer-readable medium comprising:

a program executable by a processing system for causing an image display system to;

project, with a first projector, at least a first portion of visual content onto a display surface for viewing by a human observer;

project, with the first projector, a first infrared pattern from a first projection plane of the first projector onto the display surface during the projection of the first portion of the visual content by the first projector, the first infrared pattern and the first portion of the visual content provided to the first projector as a single input;

capture the first infrared pattern from the display surface in a capture plane of at least one image capture device having an infrared filter configured to transmit the first infrared pattern and block the visual content; and

determine a first correspondence mapping between the first projector and the image capture device from at least the first infrared pattern in the first projection plane and the first infrared pattern in the capture plane; and

a medium for storing the program.

16. The non-transitory computer-readable medium of claim 15 wherein the program is executable by the processing system for causing the processing system to:

project, with a second projector, at least a second portion of the visual content onto the display surface for viewing by the human observer;

project, with the second projector, a second infrared pattern from a second projection plane of a second projector onto the display surface during the projection of the second portion of the visual content by the second projector, the second infrared pattern and the second portion of the visual content provided to the second projector as a single input;

capture the second infrared pattern from the display surface in the capture plane of the image capture device, the infrared filter configured to transmit the second infrared pattern; and

determine a second correspondence mapping between the second projector and the image capture device from at least the second infrared pattern in the second projection plane and the second infrared pattern in the capture plane.

17. The non-transitory computer-readable medium of claim 15 wherein the first infrared pattern is substantially visually imperceptible the human observer while viewing the visual content.

18. The non-transitory computer-readable medium of claim 15 wherein the first infrared pattern includes a space-time code.

19. The non-transitory computer-readable medium of claim 15 wherein the single input of the first infrared pattern and the first portion of the visual content comprises a combined input of the first pattern together with the first portion of the visual content to the first projector.