System and method of 3D modeling and virtual fitting of 3D objects

• US 10,013,803 B2
• Filed: 10/06/2015
• Issued: 07/03/2018
• Est. Priority Date: 09/30/2014
• Status: Active Grant

• Current Assignee: Fitfully Ltd
• Original Assignee: Fitfully Ltd
• Inventors: Mach Shepherd, Batia, Kaminski, Yirmeyahu, Naor, Gadi, Tertman, Roy
• Primary Examiner(s): Hoang, Phi
• Application Number:
  US15/515,876
• Publication Number:
  US 20170249783A1
• Time in Patent Office: 1,001 Days
• CPC Class Codes: A43D 1/025 comprising optical means, e.g. mirrors, photo-electric cells, for measuring or inspecting feet
• CPC Class Codes: A43D 2200/60 Computer aided manufacture of footwear, e.g. CAD or CAM
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 17/20 Finite element generation, e.g. wire-frame surface description, tesselation
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2210/16 Cloth
• CPC Class Codes: G06T 2210/56 Particle system, point based geometry or rendering

Abstract

Method and system of virtual fitting a foot with a shoe by obtaining a first and second 3D model associated with the surface of the foot and interior cavity of the shoe, respectively, each 3D model comprising a point cloud; aligning the point clouds such that the heel extreme point of the two point clouds align and the ankle back extreme point of the first point cloud aligns with the top back point of the second point cloud; calculating for at least one point p in the first point cloud a value of at least one fit indicator determined based on a distance between p and a corresponding point q in the aligned second point cloud; and outputting, to a display device, a graphical representation of the first 3D model colored in accordance with the fit calculated at p.

Claims

1. A method of virtual fitting of a foot with a shoe, the method comprising, by a processor:

obtaining data informative of a first 3D model associated with the foot, the first 3D model having a first point cloud comprising a plurality of data points representing the surface of the first 3D model;

obtaining data informative of a second 3D model associated with an interior cavity of the shoe, the second 3D model having a second point cloud comprising a plurality of data points representing the surface of the second 3D model;

aligning the first and second point clouds such that the heel extreme point of the second point cloud aligns with the heel extreme point of the first point cloud and the ankle back extreme point of the first point cloud aligns with the top back point of the second point cloud;

calculating for at least one point p in the first point cloud a value of at least one fit indicator indicative of a fit between the foot and the shoe at the point p, the value of fit indicator determined based on a distance between the at least one point p and a corresponding point q in the aligned second point cloud; and

outputting, to a display device, a graphical representation of the first 3D model colored in accordance with the calculated at least one fit indicator, wherein the color at p is indicative of the fit calculated at p,

wherein aligning the first and second point clouds comprises calculating a Principal Component Analysis (PCA) of each point cloud, aligning the centers and the axes of each point cloud, detecting the plane of each point cloud, calculating the residual rigid transformation to make the two planes coincide, and refining the results of the transformation to make the heel extreme point of the second point cloud align with the heel extreme point of the first point cloud and the ankle back extreme point of the first point cloud align with the top back point of the second point cloud.

2. The method of claim 1, wherein the second 3D model is associated with a shoe last of the shoe representative of the interior cavity of the shoe.

3. The method of claim 1, wherein aligning the centers and the axes of each point cloud comprises finding the optimal rigid transformation that merges the local coordinate systems of each point cloud.

4. The method of claim 1, wherein detecting the plane of the second point cloud comprises detecting points of low curvature and applying a statistical technique to the detected points to identify the points that form the most planar pattern.

5. The method of claim 1, wherein the refining further comprises detecting the heel extreme point as the point of local maximal curvature in the region of the heel bottom, and detecting the ankle back extreme point as lying beneath the malleoli.

6. The method of claim 1, wherein the at least one fit indicator is calculated at each of a plurality of different regions of the first point cloud, each region comprising at least two points, wherein each region is colored according to the at least one fit indicator calculated for the given region.

7. The method of claim 1, further comprising, by the processor, providing a recommendation in accordance with the at least one fit indicator, said recommendation being indicative of at least one of the following:

a measure of the fit of the shoe;

a different shoe to try;

a different size to try.

8. A method of virtual fitting of a foot with a shoe, the method comprising, by a processor:

obtaining data informative of a first 3D model associated with the foot, the first 3D model having a first point cloud comprising a plurality of data points representing the surface of the first 3D model;

obtaining data informative of a second 3D model associated with an interior cavity of the shoe, the second 3D model having a second point cloud comprising a plurality of data points representing the surface of the second 3D model;

aligning the first and second point clouds such that the heel extreme point of the second point cloud aligns with the heel extreme point of the first point cloud and the ankle back extreme point of the first point cloud aligns with the top back point of the second point cloud;

calculating for at least one point p in the first point cloud a value of at least one fit indicator indicative of a fit between the foot and the shoe at the point p, the value of fit indicator determined based on a distance between the at least one point p and a corresponding point q in the aligned second point cloud; and

outputting, to a display device, a graphical representation of the first 3D model colored in accordance with the calculated at least one fit indicator, wherein the color at p is indicative of the fit calculated at p,

wherein aligning the first and second point clouds further comprises at least one of;

applying an articulated motion to the toe region of the first point cloud at an articulation axis detected as the front point where the interior arc of the foot is tangent to the ground;

simulating an articulated motion of the ankle with respect to the foot.

9. A method of virtual fitting of a foot with a shoe, the method comprising, by a processor:

obtaining data informative of a first 3D model associated with the foot, the first 3D model having a first point cloud comprising a plurality of data points representing the surface of the first 3D model;

obtaining data informative of a second 3D model associated with an interior cavity of the shoe, the second 3D model having a second point cloud comprising a plurality of data points representing the surface of the second 3D model;

aligning the first and second point clouds such that the heel extreme point of the second point cloud aligns with the heel extreme point of the first point cloud and the ankle back extreme point of the first point cloud aligns with the top back point of the second point cloud;

calculating for at least one point p in the first point cloud a value of at least one fit indicator indicative of a fit between the foot and the shoe at the point p, the value of fit indicator determined based on a distance between the at least one point p and a corresponding point q in the aligned second point cloud; and

outputting, to a display device, a graphical representation of the first 3D model colored in accordance with the calculated at least one fit indicator, wherein the color at p is indicative of the fit calculated at p,

wherein calculating the value for the at least one fit indicator comprises calculating the normal n_p of the first point cloud surface at p, considering the line l_p passing through p and parallel to the normal n_p, calculating the intersection point q between the line l_p and the second point cloud, and calculating the distance between p and q, the distance being indicative of a fit indicator.

10. A system for virtual fitting of a foot with a shoe, comprising:

a memory configured to store data informative of first 3D model associated with the foot, the first 3D model having a first point cloud comprising a plurality of data points representing the surface of the first 3D model, and to store data informative of a second 3D model associated with an interior cavity of the shoe, the second 3D model having a second point cloud comprising a plurality of data points representing the surface of the second 3D model; and

a processor operatively coupled to the memory and configured to process the first and second point clouds by;

aligning the first and second point clouds such that the heel extreme point of the second point cloud aligns with the heel extreme point of the first point cloud and the ankle back extreme point of the first point cloud aligns with the top back point of the second point cloud;

calculating for at least one point p in the first point cloud a value of at least one fit indicator indicative of a fit between the foot and the shoe at the point p, the value of the at least one fit indicator determined based on a distance between the at least one point p and a corresponding point q in the aligned second point cloud; and

outputting, to a display device, a graphical representation of the first 3D model colored in accordance with the calculated at least one fit indicator, wherein the color at p is indicative of the fit calculated at p,

wherein aligning the first and second point clouds comprises calculating a Principal Component Analysis (PCA) of each point cloud, aligning the centers and the axes of each point cloud, detecting the plane of each point cloud, calculating the residual rigid transformation to make the two planes coincide, and refining the results of the transformation to make the heel extreme point of the second point cloud align with the heel extreme point of the first point cloud and the ankle back extreme point of the first point cloud align with the top back point of the second point cloud.

11. The system of claim 10, wherein the second 3D model is associated with a shoe last of the shoe, the shoe last representative of the interior cavity of the shoe.

12. The system of claim 10, wherein aligning the centers and the axes of each point cloud comprises finding the optimal rigid transformation that merges the local coordinate systems of each point cloud.

13. The system of claim 10, wherein detecting the plane of the second point cloud comprises detecting points of low curvature and applying a statistical technique to the detected points to identify the points that form the most planar pattern.

14. The system of claim 10, wherein the refining further comprises detecting the heel extreme point as the point of local maximal curvature in the region of the heel bottom, and detecting the ankle back extreme point as lying beneath the malleoli.

15. The system of claim 10, wherein aligning the first and second point clouds further comprises at least one of:

applying an articulated motion to the toe region of the first point cloud at an articulation axis detected as the front point where the interior arc of the foot is tangent to the ground;

simulating an articulated motion of the ankle with respect to the foot.

16. The system of claim 10, wherein calculating the value for the at least one fit indicator comprises calculating the normal n_p of the first point cloud surface at p, considering the line l_p passing through p and parallel to the normal n_p, calculating the intersection point q between the line l_p and the second point cloud, and calculating the distance between p and q, the distance being indicative of the at least one fit indicator.

17. The system of claim 10, wherein the at least one fit indicator is calculated at each of a plurality of different regions of the first point cloud, each region comprising at least two points, wherein each region is colored according to the at least one fit indicator calculated for the given region.

18. A non-transitory storage medium comprising instructions executable by a computer for executing a method of virtual fitting of a foot with a shoe, the method comprising:

obtaining data informative of a first 3D model associated with the foot, the first 3D model having a first point cloud comprising a plurality of data points representing the surface of the first 3D model;

obtaining data informative of a second 3D model associated with an interior cavity of the shoe, the second 3D model having a second point cloud comprising a plurality of data points representing the surface of the second 3D model;

aligning the first and second point clouds such that the heel extreme point of the second point cloud aligns with the heel extreme point of the first point cloud and the ankle back extreme point of the first point cloud aligns with the top back point of the second point cloud;

calculating for at least one point p in the first point cloud a value of at least one fit indicator indicative of a fit between the foot and the shoe at the point p, the value of the at least one fit indicator determined based on a distance between the at least one point p and a corresponding point q in the aligned second point cloud; and

outputting, to a display device, a graphical representation of the first 3D model colored in accordance with the calculated at least one fit indicator, wherein the color at p is indicative of the fit calculated at p,

wherein aligning the first and second point clouds comprises calculating a Principal Component Analysis (PCA) of each point cloud, aligning the centers and the axes of each point cloud, detecting the plane of each point cloud, calculating the residual rigid transformation to make the two planes coincide, and refining the results of the transformation to make the heel extreme point of the second point cloud align with the heel extreme point of the first point cloud and the ankle back extreme point of the first point cloud align with the top back point of the second point cloud.