Automatic generation of 3D models from packaged goods product images

• US 8,570,343 B2
• Filed: 04/20/2010
• Issued: 10/29/2013
• Est. Priority Date: 04/20/2010
• Status: Active Grant

• Current Assignee: Dassault Systemes SA
• Original Assignee: Dassault Systemes SA
• Inventors: Halstead, Rodd M.
• Primary Examiner(s): HARRISON, CHANTE E
• Application Number:
  US12/763,307
• Publication Number:
  US 20110254840A1
• Time in Patent Office: 1,288 Days
• Field of Search: 345/420, 345/419, 345/630, 345/442, 345/629
• US Class Current: 345/630
• CPC Class Codes: G06T 15/04 Texture mapping
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation

Abstract

A computer-implemented method and apparatus of 3D modeling accesses product data of a consumer good product. The product data includes 2D image data of the subject consumer good product and shape information of the subject consumer good product. An extractor extracts the shape information from the product data, including a 3D shape class of the subject consumer good product, and derives a detailed 3D shape graph. The detailed 3D shape graph represents an orthographically correct 3D outline shape of the subject consumer good product. As a function of the determined 3D shape class, a mesh generator generates a 3D mesh of the subject consumer good product from the detailed 3D shape graph. An image mapper projects images onto the 3D mesh, the images being of the subject consumer good product and from the 2D image data. A 3D model of the subject consumer good product results and is configured for interactive display on web pages and in other user-interactive environments.

Claims

1. A computer-implemented method of 3D modeling comprising:

accessing product data of a subject consumer good product, the product data including 2D image data of the subject consumer good product, the 2D image data being from a single given calibrated viewpoint, the single given calibrated viewpoint including at least a front viewpoint, the product data further including shape information of the subject consumer good product;

extracting the shape information from the product data, said extracting including obtaining a 2D outline shape of the consumer good product from the single given calibrated viewpoint of the 2D image data and obtaining a 3D shape class of the subject consumer good product from the shape information, the 3D shape class including a respective 3D template providing proportional geometries in three dimensions for the object shape representative of the class and corresponding rules including depth profile rules;

using the obtained 2D outline shape and 3D shape class, generating a detailed 3D shape graph, the detailed 3D shape graph representing an orthographically correct outline shape of the subject from at least the front viewpoint;

as a function of the obtained 3D shape class, generating a 3D mesh of the subject consumer good product from the detailed 3D shape graph; and

projecting onto the 3D mesh images of the subject consumer good product from the 2D image data, said projecting forming a 3D model of the subject consumer good product, the formed 3D model being configured for display on output.

2. The method of claim 1 wherein the shape information includes width, height and depth measurements of the subject consumer good product and includes an indication of 3D shape class of the subject consumer good product.

3. The method of claim 1 wherein the step of generating a 3D mesh of the subject consumer good product comprises:

determining a series of cross sectional regions of the detailed 3D shape graph, each cross sectional region having a respective cross section with respect to a longitudinal axis of the subject consumer good product and a dimension, wherein the dimension of the cross section for a given cross sectional region is different than (i) the dimension of the respective cross section of an immediately succeeding cross sectional region in series order and (ii) the dimension of the respective cross section of an immediately preceding cross sectional region in series order, such that the dimension of each of the cross sections change from one cross sectional region to a next cross sectional region in series order; and

using the series of cross sectional regions, defining the 3D mesh of the subject consumer good product.

4. The method of claim 3 wherein the step of defining the 3D mesh of the subject consumer good product further includes determining a depth profile of the subject consumer good product along the longitudinal axis.

5. The method of claim 4 wherein the step of determining a depth profile employs shape generation.

6. The method of claim 3 wherein the step of determining a series of cross sectional regions of the detailed 3D shape graph includes curve fitting between any respective cross sectional region and at least one of the immediately succeeding cross sectional region and the immediately preceding cross sectional region, wherein the cross sectional regions correspond to the 2D outline shape.

7. The method of claim 3 wherein the step of generating a mesh includes combining the detailed 3D shape graph, the determined series of cross sectional regions and a depth profile for the determined 3D shape class of the subject consumer good product.

8. The method of claim 3 wherein the step of generating a mesh includes combining the detailed 3D shape graph, the determined series of cross sectional regions and a depth profile of the subject consumer good product.

9. The method of claim 1 wherein the step of projecting employs planar projection to project 2D images of the subject consumer good product onto the generated mesh, and as a result forms the 3D model of the subject consumer good product.

10. The method of claim 1 wherein the subject consumer good product is a packaged good.

11. The method of claim 10 wherein the packaged good has a container of any one or combination of types including:

a generally cylindrical bottle,

a generally cylindrical bottle with pump action type dispensing end,

a generally tubular dispensing tube,

an effectively box shaped carton,

a generally pillow shaped sack or pouch, and

a generally sack-like container or pouch, with a planar end.

24. The computer-implemented method of claim 1, wherein the calibrated viewpoint also includes one or more of a back viewpoint, top viewpoint and side viewpoint.

12. A computer-based 3D modeling apparatus comprising:

a source of product data of a subject consumer good product, the product data including 2D image data of the subject consumer good product, the 2D image data being from a single given calibrated viewpoint, the single given calibrated viewpoint including at least a front viewpoint, the product data further including shape information of the subject consumer good product;

an extractor accessing the source and extracting the shape information from the product data, the extractor obtaining a 2D outline shape of the consumer good product from the single given calibrated viewpoint of the 2D image data and obtaining a 3D shape class of the subject consumer good product, the 3D shape class including a respective 3D template providing proportional geometries in three dimensions for the object shape representative of the class and corresponding rules including depth profile rules;

using the obtained 2D outline shape and 3D shape class, generating a detailed 3D shape graph, the detailed 3D shape graph representing an orthographically correct outline shape of the subject consumer good product from at least the front viewpoint;

a mesh generator coupled to receive the detailed 3D shape graph, and as a function of the obtained 3D shape class, the mesh generator generating a 3D mesh of the subject consumer good product from the detailed 3D shape graph; and

a modeler projecting onto the 3D mesh images of the subject consumer good product from the 2D image data, said projecting forming a 3D model of the subject consumer good product, the formed 3D model being configured for display on output.

13. The computer-based 3D modeling apparatus of claim 12, wherein the calibrated viewpoint also includes one or more of a back viewpoint, top viewpoint and side viewpoint.

14. The apparatus of claim 12 wherein the shape information includes width, height and depth measurements of the subject consumer good product and includes an indication of 3D shape class of the subject consumer good product.

15. The apparatus of claim 12 wherein the mesh generator generates the 3D mesh of the subject consumer good product by:

determining a series of cross sectional regions of the detailed 3D shape graph, each cross sectional region having a respective cross section with respect to a longitudinal axis of the subject consumer good product and a dimension, wherein the dimension of the cross section for a given cross sectional region is different than (i) the dimension of the respective cross section of an immediately succeeding cross sectional region in series order and (ii) the dimension of the respective cross section of an immediately preceding cross sectional region in series order, such that the dimension of each of the cross sections change from one cross sectional region to a next cross sectional region in series order; and

using the series of cross sectional regions, defining the 3D mesh of the subject consumer good product.

16. The apparatus of claim 15 wherein the mesh generator defining the 3D mesh of the subject consumer good product further includes determining a depth profile of the subject consumer good product along the longitudinal axis.

17. The apparatus of claim 16 wherein the determining a depth profile employs shape generation.

18. The apparatus of claim 15 wherein the determining a series of cross sectional regions of the detailed 3D shape graph includes curve fitting between any respective cross sectional region and at least one of the immediately succeeding cross sectional region and the immediately preceding cross sectional region, wherein the cross sectional regions correspond to the 2D outline shape.

19. The apparatus of claim 15 wherein the mesh generator combines the detailed 3D shape graph, the determined series of cross sectional regions and a depth profile of the determined 3D shape class of the subject consumer good product.

20. The apparatus of claim 15 wherein the mesh generator combines the detailed 3D shape graph, the determined series of cross sectional regions and a depth profile of the subject consumer good product.

21. The apparatus of claim 12 wherein the modeler employs planar projection to project 2D images of the subject consumer good product onto the generated mesh, and as a result forms the 3D model of the subject consumer good product.

22. The apparatus of claim 12 wherein the subject consumer good product is a packaged good.

23. The apparatus of claim 22 wherein the packaged good has a container of any one or combination of types including:

a generally cylindrical bottle,

a generally cylindrical bottle with pump action type dispensing end,

a generally tubular dispensing tube,

an effectively box shaped carton,

a generally pillow shaped sack, bag or pouch, and

a generally sack-like container, bag or pouch, with a planar end.