Automated Texturing Mapping and Animation from Images

• US 20160125638A1
• Filed: 11/04/2014
• Published: 05/05/2016
• Est. Priority Date: 11/04/2014
• Status: Abandoned Application

• Current Assignee: Dassault Systemes SA
• Original Assignee: Dassault Systemes SA
• Inventors: Grimaud, Jean-Jacques
• Application Number:
  US14/532,683
• Publication Number:
  US 20160125638A1
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G03B 37/00 Panoramic or wide-screen photography; Photographing extended surfaces, e.g. for surveying; Photographing internal surfaces, e.g. of pipe
• CPC Class Codes: G06T 11/001 Texturing; Colouring; Generation of texture or colour inpainting G06T5/77
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 15/205 Image-based rendering
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 2200/04 involving 3D image data
• CPC Class Codes: G06T 2207/10004 Still image; Photographic image
• CPC Class Codes: G06T 2207/30241 Trajectory
• CPC Class Codes: G06T 2210/61 Scene description
• CPC Class Codes: G06T 3/4038 Image mosaicing, e.g. composing plane images from plane sub-images
• CPC Class Codes: H04N 23/00 Cameras or camera modules comprising electronic image sensors; Control thereof

Abstract

A system for generating texture maps for 3D models of real-world objects includes a camera and reflective surfaces in the field of view of the camera. The reflective surfaces are positioned to reflect one or more reflected views of a target object to the camera. The camera captures a direct image of the target object and reflected images from the reflective surfaces. An image processor device separates the reflected views/images from the direct image of the target object in the captured image by detecting distortion in the reflected views. The image processor reduces distortion in the reflected views, and generates a texture map based on 3D space characteristics of the target object and on the reflected views. Reducing distortion in the reflected views may include scaling the reflected views to correspond to a size of the target object in the camera field of view.

Claims

1. A method comprising:

capturing an image of a field of view, the field of view having a direct view of a target object and at least one reflected view of the target object from at least one reflective surface, the captured image having a direct image of the direct view and at least one reflected image of the at least one reflected view of the target object;

separating the at least one reflected image from the direct image in the captured image;

reducing distortion from the separated reflected image to provide at least one distortion-reduced reflected image;

generating a texture map from the direct image and the at least one distortion-reduced reflected image; and

projecting the generated texture map onto a 3D model representation of the target object.

2. The method of claim 1, wherein the at least one reflective surface includes a first reflective surface and a second reflective surface, the at least one reflected view includes a first reflected view of the target object from the first reflective surface and a second reflected view of the target object from the second reflective surface, and the at least one reflected image includes a first reflected image from the first reflected view and a second reflected image from the second reflected view.

3. The method of claim 2, wherein the first and second reflected views and the direct view of the target object observe at least a portion of a circumference of the target object.

4. The method of claim 1, wherein the at least one reflective surface includes a first reflective surface, a second reflective surface, and a third reflective surface, the at least one reflected view includes a first reflected view of the target object from the first reflective surface, a second reflected view of the target object from the second reflective surface, and a third reflected view of the target object from the third reflective surface, and the at least one reflected image includes a first, second, and third reflected image of the first, second, and third reflected views, respectively.

5. The method of claim 4, wherein the first and second reflected views and the direct view observe at least a portion of a circumference of the target object the third reflected view observes a top or bottom surface of the target object.

6. The method of claim 2, further including:

detecting an overlap between at least two of the following;

the direct image of the target object, the first reflected image, and the second reflected image; and

removing the detected overlap from the at least one image.

7. The method of claim 6, wherein detecting an overlap further includes:

detecting a common region in each of at least two of the images having the detected overlap;

removing the common regions from each of the at least two of the images having the detected overlap;

calculating the size and shape of the common regions using known positions of the first and second reflective surfaces;

correcting the common regions to represent the calculated a portion of the surface of the target object;

determining on an image quality of each corrected common regions and merging the corrected common regions into a merged region using the determined image quality; and

using the joined region in generating the at least one texture map.

8. The method of claim 1, wherein separating the at least one reflected image from the direct image of target object in the captured image includes:

detecting a region of distortion in the captured image; and

flagging the detected region as the at least one reflected image of the target object.

9. The method of claim 1, wherein reducing distortion from the at least one reflected image includes at least one of:

scaling the at least one reflected image by a size of the target object, correcting perspective distortion in the at least one reflect image, and reshaping the at least one reflected image based on a position of the at least one reflected surface and a shape of the target object.

10. The method of claim 1, further including:

capturing a plurality of captured images of the field of view having a target object as the target object moves from a first position to a second position; and

generating a plurality of texture maps from the plurality of images.

11. The method of claim 10, further including:

detecting movements of the target object;

correlating the detected movements of the target object with the 3D model to generate corresponding movements of the 3D model; and

animating the 3D model from a corresponding first position to a corresponding second position based on the corresponding movements of the 3D model.

12. A system for generating a texture map for a 3D model from a target object with a single image, the system comprising:

a camera having a field of view, the camera capturing an image of the field of view, the captured image including a direct image of target object;

one or more reflective surfaces in the field of view of the camera, the one or more reflective surfaces positioned to reflect one or more reflected views of the target object to the camera, the captured image further including one or more reflected images of the one or more reflected views; and

an image processing device receiving the captured image, separating the one or more reflected images from the direct image, reducing distortion in the one or more reflected images, and generating a texture map based on direct image and the one or more separated reflected images.

13. The system of claim 12, wherein the one or more reflective surfaces includes a first reflective surface and a second reflective surface, and the one or more reflected images of the target object includes a first reflected images and a second reflected image.

14. The system of claim 13, wherein the first and second reflected images and the direct image of the target object image substantially all of a circumference of the target object.

15. The system of claim 13, wherein the field of view further includes a third reflective surface, and the one or more reflected views includes a third reflected image of the target object.

16. The system of claim 15, wherein the first and second reflected images and the direct image observe substantially all of a circumference of the target object and the third reflective image observes a top or bottom surface of the target object.

17. The system of claim 13, further including the image processing device detecting an overlap between at least two of the following:

the direct image of the target object, the first reflected image, and the second reflected image, and removing the detected overlap from the captured image.

18. The system of claim 17, where the image processing device detecting an overlap further includes the image processing device:

detecting a common region in each of at least two of the images having the detected overlap;

removing the common regions from each of the at least two of the images having the detected overlap;

calculating the size and shape of the common regions using given positions of the first and second reflective surfaces;

correcting the common regions to represent the calculated a portion of the surface of the target object;

determining on an image quality of each corrected common regions and merging the corrected common regions into a merged region using the determined image quality; and

using the joined region in generating the at least one texture map.

19. The system of claim 12, wherein the image processing device separating the one or more reflected views from the target object in the captured image includes:

detecting one or more regions of keystone distortion in the captured image; and

flagging detected regions as the at least one reflected views.

20. The system of claim 12, wherein the image processing device reducing distortion in the one or more reflected views includes scaling the at least one or more reflected views to correspond to a size of the target object.

21. The system of claim 12, wherein the camera capturing a plurality of images of the field of view as the target object moves from a first position to a second position, the image processing device receiving the plurality of captured images and generating a plurality of texture maps.

22. The system of claim 21, wherein the image processing device detects the movement of the target object, correlates the detected movements of the target object with the 3D model to generate corresponding movements of the 3D model, and animates the 3D model from a corresponding first position to a corresponding second position based on the corresponding movements of the 3D model.