Blending 3D model textures by image projection

• US 9,224,233 B2
• Filed: 05/24/2012
• Issued: 12/29/2015
• Est. Priority Date: 05/24/2012
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Hsu, Stephen Charles
• Primary Examiner(s): Tung, Kee M
• Assistant Examiner(s): Prendergast, Roberta D
• Application Number:
  US13/479,952
• Publication Number:
  US 20150154805A1
• Time in Patent Office: 1,314 Days
• US Class Current: 1/1
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G09G 2340/10 Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• CPC Class Codes: G09G 2340/12 Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels

Abstract

An example method and system for blending textures of a composite image formed by a plurality of source images mapped onto a three dimensional model are presented. The composite image is projected to obtain an unblended projected image having textures. The textures are blended to obtain a blended projected image. Both the unblended and the blended projected images are backprojected onto the three dimensional model. A difference is determined between a pixel of the backprojected, blended image and a corresponding pixel of the backprojected, unblended image. The determined difference is then applied to a further corresponding pixel of the composite image to obtain a modified composite image representing a blending of the plurality of source images in the composite image.

Claims

1. A method, performed on one or more processors, for blending textures of a composite image formed by a plurality of source images mapped onto a three dimensional model, comprising:

projecting the composite image to obtain a two dimensional, unblended, projected image having textures;

blending the textures of the unblended projected image to obtain a blended projected image;

backprojecting the unblended projected image and the blended projected image onto the three dimensional model;

determining a difference between a pixel of the backprojected, blended image and a corresponding pixel of the backprojected, unblended image; and

applying the determined difference to a further corresponding pixel of the composite image to obtain a modified composite image representing a blending of the plurality of source images in the composite image.

2. The method of claim 1, wherein the projecting comprises projecting an oblique composite image.

3. The method of claim 1, wherein the projecting comprises orthorectifying the composite image to obtain an unblended orthorectified image.

4. The method of claim 1, wherein the blending comprises blending the textures using a multiband blending technique.

5. The method of claim 1, further comprising tagging each pixel of the composite image with an ID corresponding to the source image associated with the pixel.

6. The method of claim 1, further comprising dividing the unblended projected image into a plurality of tiles.

7. The method of claim 6, further comprising overlapping each of the plurality of tiles in at least one of X and Y dimensions.

8. The method of claim 6, wherein the textures within each of the plurality of tiles are blended individually.

9. The method of claim 8, wherein the blending the textures comprises blending each of the plurality of tiles together to obtain the blended projected image.

10. The method of claim 1, wherein the determining a difference comprises determining an additive change between a pixel of the backprojected, blended image and a corresponding pixel of the backprojected, unblended image.

11. The method of claim 1, wherein the determining a difference comprises determining a multiplicative change between a pixel of the backprojected, blended image and a corresponding pixel of the backprojected, unblended image.

12. A system for blending textures of a composite image formed by a plurality of source images mapped onto a three dimensional model, comprising:

an image projector configured to project the composite image to obtain a two dimensional, unblended projected image having textures;

a blending engine configured to blend the textures of the unblended projected image to obtain a blended projected image;

a backprojecting engine configured to backproject the unblended projected image and the blended projected image onto the three dimensional model; and

a pixel processing unit configured to;

determine a difference between a pixel of the backprojected, blended image and a corresponding pixel of the backprojected, unblended image, and

apply the difference to a further corresponding pixel of the composite image to obtain a modified composite image representing a blending of the plurality of source images in the composite image.

13. The system of claim 12, wherein the image projector is configured to orthorectify the composite image to obtain an unblended orthorectified image.

14. The system of claim 12, wherein the plurality of source images illustrate content of the model at different perspectives.

15. The system of claim 12, wherein the plurality of source images illustrate content of the model at various brightness levels.

16. The system of claim 12, wherein the plurality of source images illustrate content of the model at various exposure settings.

17. The system of claim 12, wherein the blending engine is configured to blend the textures using a multiband blending technique.

18. The system of claim 12, wherein the composite image is derived from a digital surface model (DSM).

19. The system of claim 12, wherein the blending engine is further configured to separately blend tiled regions of the unblended projected image.

20. An apparatus comprising at least one non-transitory computer readable storage medium encoding instructions thereon that, in response to execution by a computing device, cause the computing device to perform operations comprising:

projecting the composite image formed by a plurality of the source images mapped onto a three dimensional model to obtain a two dimensional, unblended projected image having textures;

blending the textures of the unblended projected image to obtain a blended projected image;

backprojecting the unblended projected image and the blended projected image onto the three dimensional model;

determining a difference between a pixel of the backprojected, blended image and a corresponding pixel of the backprojected, unblended image; and

applying the determined difference to a further corresponding pixel of the composite image to obtain a modified composite image representing a blending of the plurality of source images in the composite image.

21. The apparatus of claim 20, wherein the plurality of source images illustrate content of the model at different perspectives.

22. The apparatus of claim 20, wherein the plurality of source images illustrate content of the model at various brightness levels.

23. The apparatus of claim 20, wherein the plurality of source images illustrate content of the model at various exposure settings.

24. The apparatus of claim 20, wherein the composite image is derived from a digital surface model (DSM).