Method, medium, and system with 3 dimensional object modeling using multiple view points

• US 20080211809A1
• Filed: 01/24/2008
• Published: 09/04/2008
• Est. Priority Date: 02/16/2007
• Status: Active Grant

• Current Assignee: Samsung Electronics Co. Ltd.
• Original Assignee: Samsung Electronics Co. Ltd.
• Inventors: Zhang, Hui, Konouchine, Anton, Vezhnevets, Vladimir, Jung, Seok-yoon, Kim, Do-kyoon, Kim, Keun-ho
• Granted Patent: US 8,681,150 B2
• Time in Patent Office: Days
• US Class Current: 345/420
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2200/08 involving all processing steps from image acquisition to 3D model generation
• CPC Class Codes: G06V 20/38 Outdoor scenes
• CPC Class Codes: G06V 20/39 Urban scenes

Abstract

A modeling method, medium, and system. The modeling method may include specifying an object in a plurality of 2 dimensional (2D) images expressed from different view points, considering an edge and contents of each component of each 2D image, generating a 3 dimensional (3D) model of each specified object, and matching the generated 3D models considering relative locations of each generated 3D model. Accordingly, a realistic 3D model of an object expressed in an image can be accurately generated.

Claims

1. A modeling method comprising combining at least two respective 3D models from at least two different view points representing separate and/or overlapping portions of a same identified object, represented by each 3D model, placing an image including the identified object over the combined 3D models, and outputting a corresponding resultant 3D model with the over-placed image.

2. The modeling method of claim 1, further comprising identifying the identified object from the image of one of the at least two different view points through comparison of characteristic information of the identified object with a characteristics of a corresponding plurality of pre-defined 3D models and combining the at least two respective 3D models based upon respective relationship with the identified object through the different view points.

41. At least one medium comprising computer readable code to control at least one processing element to implement the method of claim 1.

3. A modeling method generating a 3D model for a 2D object, comprising comparing a first image of a first view point including the object and a second image of a second view point including the object, modifying the first image to include a portion of the second image, with reference to the object in the second image, through modification of the portion of the second image for insertion into the first image to replace a corresponding portion of the first image, and placing the modified first image over a 3D model corresponding to the object.

4. The modeling method of claim 3, wherein the 3D model corresponding to the object is selected from a plurality of pre-defined 3D models based upon identified characteristics of the object.

5. The modeling method of claim 3, wherein the first view point is different from the second view point.

6. The modeling method of claim 3, further comprising combining at least two respective 3D models from at least two different view points representing separate and/or overlapping portions of the object and placing the modified first image over a corresponding combined 3D model.

42. At least one medium comprising computer readable code to control at least one processing element to implement the method of claim 3.

7. A modeling method, comprising:

identifying an object present in at least a first image of a first view point by comparing analyzed edges and/or contents of components of the first image with predefined edge and/or content information of pre-defined objects;

generating a 3D model based upon results of the comparing of the analyzed edges and/or contents of components; and

placing the first image over the generated 3D model.

8. The modeling method of claim 7, further comprising combining at least two respective 3D models from at least two different view points representing separate and/or overlapping portions of the object and placing the first image over a corresponding combined 3D model.

9. The modeling method of claim 7, further comprising arranging a plurality of generated 3D models, including the generated 3D model, in a 3D space so that relative locations between different identified objects, including the object, in the first image are relatively reflected in the 3D space.

10. The modeling method of claim 9, wherein the arranging of the plurality of generated 3D models includes merging generated 3D models.

11. The modeling method of claim 7, further comprising comparing the first image and a second image of a second view point, modifying the first image to include a portion of the second image with reference to the object, as included in the second image, through modification of the portion of the second image for insertion into the first image to replace a corresponding portion of the first image, and placing the modified first image over the generated 3D model.

12. The modeling method of claim 11, wherein the first view point is different from the second view point.

13. The modeling method of claim 7, further comprising:

comparing the first image and a second image of a second view point, modifying the first image to include a portion of the second image with reference to the object, as included in the second image, through modification of the portion of the second image for insertion into the first image to replace a corresponding portion of the first image;

combining at least two respective 3D models from at least two different view points representing separate and/or overlapping portions of the object; and

placing the modified image over a corresponding combined 3D model.

14. The modeling method of claim 7, wherein the generating of the 3D model includes calculating depth values of pixels corresponding to the identified object.

15. The modeling method of claim 7, wherein the comparing of the analyzed edges and/or contents of components comprises comparing detected texture information in the first image with pre-defined texture information of respective plural pre-defined objects for identifying the object in the first image.

16. The modeling method of claim 7, wherein the comparing of the analyzed edges and/or contents of components comprises comparing detected edge information in the first image with pre-defined edge information of respective plural pre-defined objects for identifying the object in the first image.

17. The modeling method of claim 7, wherein the comparing of the analyzed edges and/or contents of components comprises comparing detected color component information in the first image with pre-defined color component information of respective plural pre-defined objects for identifying the object in the first image.

18. The modeling method of claim 7, wherein the comparing of the analyzed edges and/or contents of components comprises comparing detected location information in the first image with pre-defined location information of respective plural pre-defined objects for identifying the object in the first image.

19. The modeling method of claim 7, wherein the comparing of the analyzed edges and/or contents of components comprises converting the first image into multiple images, each representing differing information of the first image3, including edge information, texture information, color information, location information, and/or 3D geometry information for the comparing with the edge and/or content information of pre-defined objects.

20. The modeling method of claim 19, further comprising dividing the first image according to super pixels representing preset similarities between pixels for the converting of the first image into the multiple images.

43. At least one medium comprising computer readable code to control at least one processing element to implement the method of claim 7.

44. At least one medium comprising computer readable code to control at least one processing element to implement the method of claim 13.

21. A modeling system, comprising:

a model matching unit to combine at least two identified respective 3D models from at least two different view points representing separate and/or overlapping portions of a same identified object, represented by each 3D model; and

an arranging unit to place an image including the identified object over the combined 3D models.

22. The modeling system of claim 21, further comprising an identification unit to identify the identified object from the image of one of the at least two different view points through comparison of characteristic information of the identified object with a characteristics of a corresponding plurality of pre-defined 3D models, such that the model matching unit combines the at least two respective 3D models based upon respective relationship with the identified object through the different view points.

23. A modeling system generating a 3D model for a 2D object, comprising:

a model generator to compare a first image of a first view point including the object and a second image of a second view point including the object, modifying the first image to include a portion of the second image, with reference to the object in the second image, through modification of the portion of the second image for insertion into the first image to replace a corresponding portion of the first image; and

an arranging unit to place the modified first image over a 3D model corresponding to the object.

24. The modeling system of claim 23, wherein the 3D model corresponding to the object is selected by an identification unit from a plurality of pre-defined 3D models based upon identified characteristics of the object.

25. The modeling system of claim 23, wherein the first view point is different from the second view point.

26. The modeling system of claim 23, further comprising a model matching unit to combine at least two respective 3D models from at least two different view points representing separate and/or overlapping portions of the object, such that the arranging unit places the modified first image over a corresponding combined 3D model.

27. A modeling method, comprising:

an object identification unit to identify an object present in at least a first image of a first view point by comparing analyzed edges and/or contents of components of the first image with predefined edge and/or content information of pre-defined objects;

a model generator to generate a 3D model based upon results of the comparing of the analyzed edges and/or contents of components; and

an arranging unit to place the first image over the generated 3D model.

28. The modeling system of claim 27, further comprising a model matching unit to combine at least two respective 3D models from at least two different view points representing separate and/or overlapping portions of the object,

wherein the arranging unit places the first image over a corresponding combined 3D model.

29. The modeling system of claim 27, wherein the arranging unit arranges a plurality of generated 3D models, including the generated 3D model, in a 3D space so that relative locations between different identified objects, including the object, in the first image are relatively reflected in the 3D space.

30. The modeling system of claim 29, wherein the arranging of the plurality of generated 3D models includes merging generated 3D models.

31. The modeling system of claim 27, wherein the model generator further compares the first image and a second image of a second view point, and modifies the first image to include a portion of the second image with reference to the object, as included in the second image, through modification of the portion of the second image for insertion into the first image to replace a corresponding portion of the first image,

wherein the arranging unit places the modified first image over the generated 3D model.

32. The modeling system of claim 31, wherein the first view point is different from the second view point.

33. The modeling system of claim 27, wherein the model generator compares the first image and a second image of a second view point and modifies the first image to include a portion of the second image with reference to the object, as included in the second image, through modification of the portion of the second image for insertion into the first image to replace a corresponding portion of the first image, and the system further comprises:

a model matching unit to combine at least two respective 3D models from at least two different view points representing separate and/or overlapping portions of the object,

wherein the arranging unit places the modified image over a corresponding combined 3D model.

34. The modeling system of claim 27, wherein the arranging unit generates the 3D model by at least calculating depth values of pixels corresponding to the identified object.

35. The modeling system of claim 27, wherein the comparing of the analyzed edges and/or contents of components comprises comparing detected texture information in the first image with pre-defined texture information of respective plural pre-defined objects for identifying the object in the first image.

36. The modeling system of claim 27, wherein the comparing of the analyzed edges and/or contents of components comprises comparing detected edge information in the first image with pre-defined edge information of respective plural pre-defined objects for identifying the object in the first image.

37. The modeling system of claim 27, wherein the comparing of the analyzed edges and/or contents of components comprises comparing detected color component information in the first image with pre-defined color component information of respective plural pre-defined objects for identifying the object in the first image.

38. The modeling system of claim 27, wherein the comparing of the analyzed edges and/or contents of components comprises comparing detected location information in the first image with pre-defined location information of respective plural pre-defined objects for identifying the object in the first image.

39. The modeling system of claim 27, wherein the comparing of the analyzed edges and/or contents of components comprises converting the first image into multiple images, each representing differing information of the first image3, including edge information, texture information, color information, location information, and/or 3D geometry information for the comparing with the edge and/or content information of pre-defined objects.

40. The modeling system of claim 39, wherein the comparing of the analyzed edges and/or contents of components further comprises dividing the first image according to super pixels representing preset similarities between pixels for the converting of the first image into the multiple images.