Mesh for rendering an image frame

• US 7,854,518 B2
• Filed: 06/16/2006
• Issued: 12/21/2010
• Est. Priority Date: 06/16/2006
• Status: Expired due to Fees

• Current Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Original Assignee: Hewlett-Packard Development Company, L.P. (HP Inc.)
• Inventors: Harville, Michael, Fitzhugh, Andrew E., Culbertson, W. Bruce, Tanguay, Donald O. Jr., Sobel, Irwin E., Gelb, Daniel G.
• Primary Examiner(s): Epps; Georgia Y
• Assistant Examiner(s): REILLY-DIAKUN, JORI S
• Application Number:
  US11/455,306
• Publication Number:
  US 20070291233A1
• Time in Patent Office: 1,649 Days
• Field of Search: 353 69- 70, 353/94, 348/745, 345/1.1, 345/619, 345/694
• US Class Current: 353/69
• CPC Class Codes: H04N 9/3147 Multi-projection systems displays in general H04N9/12; video walls G06F3/1446, G09G2300/026
• CPC Class Codes: H04N 9/3194 including sensor feedback

Abstract

A method includes generating a first plurality of meshes configured to map a first domain associated with a display surface having a plurality of fiducial marks to a second domain associated with a image capture device configured to capture a first image of the display surface, generating a second plurality of meshes configured to map the second domain to a third domain associated with a first projector configured to display a second image onto the display surface, and generating a first mesh configured to map the first domain to the third domain in conjunction with a second mesh using the first plurality of meshes and the second plurality of meshes.

Claims

1. A method performed by a processing system, the method comprising:

generating, with the processing system, a first plurality of meshes configured to map a first domain associated with a display surface having a plurality of fiducial marks to a second domain associated with an image capture device configured to capture a first image of the display surface;

generating, with the processing system, a second plurality of meshes configured to map the second domain to a third domain associated with a first projector configured to display a second image onto the display surface; and

generating, with the processing system, a first mesh configured to map the first domain to the third domain in conjunction with a second mesh using the first plurality of meshes and the second plurality of meshes, the first mesh and the second mesh including a first set of point correspondences configured into a first plurality of polygons in the first mesh and a second plurality of polygons in the second mesh, and each of the first plurality of polygons corresponding to a respective one of the second plurality of polygons.

2. The method of claim 1 further comprising:

applying the first mesh to render a first image frame with the processing system; and

projecting the first image frame with the first projector.

3. The method of claim 1 further comprising:

capturing the first image of the display surface using the image capture device;

locating the plurality of fiducial marks in the first image with the processing system;

generating a second set of point correspondences between the plurality of fiducial marks in the first image and the plurality of fiducial marks on the display surface with the processing system; and

determining the first plurality of meshes from the second set of point correspondences with the processing system, the second set of point correspondences configured into a third plurality of polygons in a first one of the first plurality of meshes and a fourth plurality of polygons in a second one the first plurality of meshes, and each of the third plurality of polygons corresponding to a respective one of the fourth plurality of polygons.

4. The method of claim 3 wherein the plurality of fiducial marks does not overlap with the second image on the display surface.

5. The method of claim 3 further comprising:

constructing, with the processing system, a first one of the first plurality of meshes to include a first set of points in the first domain from the second set of point correspondences; and

constructing, with the processing system, a second one of the first plurality of meshes to include a second set of points in the second domain from the second set of point correspondences such that the second one of the first plurality of meshes corresponds to the first one of the first plurality of meshes using the set of point correspondences.

6. The method of claim 1 further comprising:

capturing one or more images of one or more known patterns projected on the display surface;

generating, with the processing system, a second set of point correspondences between the one or more known patterns and the one or more known patterns in the one or more images; and

determining the second plurality of meshes from the second set of point correspondences with the processing system, the second set of point correspondences configured into a third plurality of polygons in a first one of the first plurality of meshes and a fourth plurality of polygons in a second one the first plurality of meshes, and each of the third plurality of polygons corresponding to a respective one of the fourth plurality of polygons.

7. The method of claim 6 further comprising:

constructing, with the processing system, a first one of the second plurality of meshes to include a first set of points in the second domain from the second set of point correspondences; and

constructing, with the processing system, a second one of the second plurality of meshes to include a second set of points in the third domain from the set of point correspondences such that the second one of the second plurality of meshes corresponds to the first one of the second plurality of meshes using the second set of point correspondences.

8. The method of claim 1 further comprising:

constructing, with the processing system, the second mesh to include a first set of points in the first domain;

generating, with the processing system, the first set of point correspondences between the first set of points and a second set of points in the third domain using the first plurality of meshes and the second plurality of meshes; and

constructing, with the processing system, the first mesh to include the second set of points such that the first mesh corresponds to the second mesh using the first set of point correspondences.

9. The method of claim 8 further comprising:

for each of the first set of points;

identifying a first triangle in a first one of the first plurality of meshes that includes or is nearest to one of the first set of points with the processing system;

determining first barycentric coordinates for the one of the first set of points in the first triangle with the processing system;

applying the first barycentric coordinates to a second triangle in a second one of the first plurality of meshes that corresponds to the first triangle to identify a first point in the second domain with the processing system;

identifying a third triangle in a first one of the second plurality of meshes that includes or is nearest to the first point with the processing system;

determining second barycentric coordinates for the first point in the third triangle with the processing system; and

applying the second barycentric coordinates to a fourth triangle in a second one of the second plurality of meshes that corresponds to the third triangle to identify one of the second set of points with the processing system.

10. The method of claim 1 further comprising:

generating at least one of the first plurality of meshes, the second plurality of meshes, and the first mesh using Delaunay triangulation with the processing system.

11. The method of claim 1 further comprising:

generating, with the processing system, a third plurality of meshes configured to map the second domain to a fourth domain associated with a second projector configured to display a third image onto the display surface simultaneously with the display of the second image by the first projector; and

generating, with the processing system, a third mesh configured to map the first domain to the fourth domain using the first plurality of meshes and the third plurality of meshes, the third mesh and the second mesh including a second set of point correspondences configured into a third plurality of polygons in the third mesh and a fourth plurality of polygons in the second mesh, and each of the third plurality of polygons corresponding to a respective one of the fourth plurality of polygons.

12. A system comprising:

a frame generator configured to render a first image frame using a first mesh to generate a second image frame; and

a first projector configured to store the second image frame in a first frame buffer and project the second image frame onto a display surface to display a first image;

wherein the first mesh defines a first mapping between the display surface and the first frame buffer, wherein the first mapping is configured into a first plurality of polygons, and wherein each of the first plurality of polygons correspond to a respective one of a second plurality of polygons in a domain corresponding to the display surface.

13. The system of claim 12 wherein the frame generator is configured to produce the second image frame by interpolating a first plurality of pixel values using the first mesh and the second mesh.

14. The system of claim 12 wherein the frame generator is configured to warp the first image frame using the first mesh and the second mesh to generate the second image frame.

15. The system of claim 12 wherein the frame generator is configured to render a third image frame using a second mesh to generate a fourth image frame;

and further comprising;

a second projector configured to store the fourth image frame in a second frame buffer and project the fourth image frame onto the display surface to display a second image such that the second image partially overlaps with the first image on the display surface;

wherein the second mesh defines a second mapping between the display surface and the second frame buffer, wherein the second mapping is configured into a third plurality of polygons, and wherein each of the third plurality of polygons correspond to a respective one of a fourth plurality of polygons in the domain corresponding to the display surface.

16. The system of claim 15 wherein the frame generator is configured to render the first image frame and the third image frame to cause the first image and the second image to be spatially aligned on the display surface.

17. The system of claim 12 wherein the display surface forms a non-planar developable surface.

18. A program product stored on a medium readable by a processing system, the program product comprising instructions executable by the processing system to:

generate a first plurality of meshes from at least a first image that includes a plurality of fiducial marks on a display surface in a predetermined arrangement;

generate a second plurality of meshes from a first known pattern sequence and one or more second images that includes the first known pattern sequence, the one or more second images captured of the display of the first known pattern sequence by a first projector;

generate a first mesh that includes a first set of points that are associated with the first projector and determined using the first plurality of meshes and the second plurality of meshes, wherein the first set of points is configured into a first plurality of polygons, and wherein each of the first plurality of polygons correspond to a respective one of a second plurality of polygons in a domain corresponding to the display surface.

19. The program product of claim 18 and further comprising instructions executable by the processing system to:

provide the first mesh to a frame generator.

20. The program product of claim 18 and further comprising instructions executable by the processing system to:

generate a third plurality of meshes from a second known pattern sequence and one or more third images that includes the second known pattern sequence, the one or more third image captured of the display of the second known pattern sequence by a second projector;

generate a second mesh that includes a second set of points that are associated with the second projector and determined using the first plurality of meshes and the third plurality of meshes, wherein the second set of points is configured into a third plurality of polygons, and wherein each of the third plurality of polygons correspond to a respective one of a fourth plurality of polygons in the domain corresponding to the display surface.