System and method for body scanning and avatar creation

• US 10,628,729 B2
• Filed: 06/13/2011
• Issued: 04/21/2020
• Est. Priority Date: 06/08/2010
• Status: Active Grant

• Current Assignee: Styku, LLC
• Original Assignee: Styku, LLC
• Inventors: Sareen, Raj
• Primary Examiner(s): Perungavoor, Sath V
• Assistant Examiner(s): Dobbs, Kristin
• Application Number:
  US13/159,401
• Publication Number:
  US 20120086783A1
• Time in Patent Office: 3,235 Days
• Field of Search: 345420, 700132
• CPC Class Codes: G06N 3/006 based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2210/16 Cloth
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

A method and apparatus is disclosed for scanning a body. The system comprises a processor and a range camera capable of capturing at least a first set of depth images of the body rotated to 0 degrees and at least a second set of depth images of the body rotated to x degrees, wherein x is >0 degrees, and x<360 degrees. A first set of computer instructions executable on the processor is capable of calculating a first set of three dimensional points from the first set of depth images and a second set of three dimensional points from the second set of depth images. A second set of computer instructions executable on the processor is capable of rotating and translating the first and second set of three dimensional points into a final set of three dimensional points. A third set of computer instructions executable on the processor is capable of creating a three dimensional mesh from the final set of three dimensional points.

Claims

1. A system for scanning and measuring a surface of a body of a person, including any clothing worn, comprising:

a processor;

an infrared camera comprising an infrared light emitter that emits infrared light and at least an infrared sensor that reads reflected infrared light;

a first set of computer instructions executable on the processor capable causing the infrared emitter to illuminate the surface of the body with infrared light and to cause the infrared sensor to capture the infrared light reflected off the surface of the body;

a second set of computer instructions executable on the processor capable of capturing and storing the reflected infrared light in at least a first depth image of a surface of a body from at least a first point of view while the body is in a pose and at least a second depth image of the outer surface of the body from at least a second point of view while the body remains in the same pose;

a third set of computer instructions executable on the processor capable of calculating a first set of three dimensional points from the first depth image and a second set of three dimensional points from the second depth image;

a fourth set of computer instructions executable on the processor capable of applying rotational and translational matrix transformations on the first set of three dimensional points to align said first set of three dimensional points with the second set of three dimensional points and then calculating and storing a square of the sum of the differences between the first set of three dimensional points with the second set of three dimensional points as an error;

a fifth set of computer instructions executable on the processor capable of reconstructing the surface of the body by minimizing an error between the first and second set of three dimensional points by repeatedly making iterative and small changes to the matrix transformations and then re-calculating the error until the error is minimized;

a sixth set of computer instructions executable on the processor capable of applying poisson surface reconstruction to construct a mesh;

a seventh set of computer instructions executable on the processor capable of rigging the mesh with a biped bone structure, scaled and sized to appropriately fit the mesh;

an eighth set of computer instructions executable on the processor capable of utilizing the bone structure from within the mesh to identify one or more anthropometric points of measure;

a ninth set of computer instructions executable on the processor capable of extracting anthropometric body circumferences by calculating the convex hull associated with points around a specific region of the body'"'"'s surface.

2. The system of claim 1, comprising avatar processing software capable of creating an avatar from the final combined set of three dimensional points.

3. The system of claim 2, wherein the avatar is capable of display to a user on a network attached device over a network.

4. The system of claim 1, wherein each of the first and second points of view is of a type selected from the group consisting of:

a point of view of the body on a rotating pedestal while one or more infrared cameras remain stationary;

a point of view of the body on a rotating pedestal while one or more infrared cameras are moving to capture each depth image;

a point of view of the body remaining fixed in space while one or more infrared cameras are moving to capture each depth image;

a point of view of the body remaining fixed in space and with two or more infrared cameras remaining fixed in space with each infrared camera having a different point of view with respect to the other infrared camera;

a point of view of the body with one or more infrared cameras moving with respect to the body that moves while capturing the depth images; and

a point of view of the body with one or more infrared cameras moving, stopping to take a depth image, moving and then stopping again to take a depth image; and

a point of view of the body with one or more infrared cameras moving, stopping to take at least one depth image, but also taking yet at least another image while moving.

5. The system of claim 1, wherein the third set of instructions is further capable of minimizing the differences between the first and second sets of three dimensional points.

6. The system of claim 1, wherein the third set of instructions is further capable of rotating and translating the first and second sets of three dimensional points.

7. The system of claim 1, wherein the system is configured for operation in one or more locations selected from the group consisting of:

a home, a commercial kiosk, a retrofitted store fitting room, a gaming machine establishment, and a karaoke room.

8. A method for scanning and measuring a surface of a body of a person, including any clothing worn, comprising:

causing an infrared camera to illuminate the surface of the body with infrared light and to capture the infrared light reflected off the surface of the body, the infrared camera comprising an infrared light emitter that is capable of emitting the infrared light, the infrared camera further comprising an infrared sensor that is capable of reading reflected infrared light;

storing the reflected infrared light in at least a first depth image of a surface of a body from at least a first point of view while the body is in a pose and at least a second depth image of the surface of the body from at least a second point of view while the body remains in the same pose;

calculating a first set of three dimensional points from the first depth image and a second set of three dimensional points from the second depth image;

applying rotational and translational matrix transformations on the first set of three dimensional points to align said first set of three dimensional points with the second set of three dimensional points and then calculating and storing a square of the sum of the differences between the first set of three dimensional points with the second set of three dimensional points as an error;

applying poisson surface reconstruction to construct a mesh;

rigging the mesh with a biped bone structure, scaled and sized to appropriately fit the mesh;

utilizing the bone structure from within the mesh to identify one or more anthropometric points of measure; and

a ninth set of computer instructions executable on the processor capable of extracting anthropometric body circumferences by calculating the convex hull associated with points around a specific region of the body'"'"'s surface.

9. The method of claim 8, comprising creating an avatar from the mesh capable of display to a user on a network attached device over a network.

10. The method of claim 8, wherein each of the first and second points of view is of a type selected from the group consisting of:

a point of view of the body on a rotating pedestal while one or more infrared cameras remain stationary;

a point of view of the body on a rotating pedestal while one or more infrared cameras are moving to capture each depth image;

a point of view of the body remaining fixed in space while one or more infrared cameras are moving to capture each depth image;

a point of view of the body remaining fixed in space and with two or more infrared cameras remaining fixed in space with each infrared camera having a different point of view with respect to the other infrared camera;

a point of view of the body with one or more infrared cameras moving with respect to the body that moves while capturing the depth images; and

a point of view of the body with one or more infrared cameras moving, stopping to take a depth image, moving and then stopping again to take a depth image; and

a point of view of the body with one or more infrared cameras moving, stopping to take at least one depth image, but also taking yet at least another image while moving.

11. The method of claim 8, comprising displaying the measurements over a network on a network attached device.

12. The method of claim 8, wherein each step is performed in one or more locations selected from the group consisting of:

a home, a commercial kiosk, a retrofitted store fitting room, a gaming machine establishment, and a karaoke room.

13. A system for scanning and measuring a surface of a body of a person, including any clothing worn, comprising:

a processor;

an infrared camera comprising an infrared light emitter that emits infrared light to illuminate the surface of the body including any clothing worn, and two or more infrared sensors that capture reflected infrared light resulting from the emitter'"'"'s illuminating of the surface of the body;

a first set of computer instructions executable on the processor capable of causing the infrared camera to illuminate the surface of the body with infrared light and to capture the infrared light reflected off the surface of the body;

a second set of computer instructions executable on the processor capable of capturing and storing reflected infrared light in at least a first depth image of a surface of a body from at least a first point of view while the body is in a pose and at least a second depth image of the surface of the body from at least a second point of view while the body remains in the same pose;

a third set of computer instructions executable on the processor capable of calculating a first set of three dimensional points from the first depth image and a second set of three dimensional points from the second depth image;

a fourth set of computer instructions executable on the processor capable of applying rotational and translational matrix transformations on the first set of three dimensional points to align said first set of three dimensional points with the second set of three dimensional points and then calculating and storing a square of the sum of the differences between the first set of three dimensional points with the second set of three dimensional points as an error;

a fifth set of computer instructions executable on the processor capable of reconstructing the surface of the body by minimizing an error between the first and second set of three dimensional points by repeatedly making iterative and small changes to the matrix transformations and then re-calculating the error until the error is minimized;

a sixth set of computer instructions executable on the processor capable of applying poisson surface reconstruction to construct a mesh;

a seventh set of computer instructions executable on the processor capable of rigging the mesh with a biped bone structure, scaled and sized to appropriately fit the mesh;

an eighth set of computer instructions executable on the processor capable of utilizing the bone structure from within the mesh to identify one or more anthropometric points of measure; and

a ninth set of computer instructions executable on the processor capable of extracting anthropometric body circumferences by calculating the convex hull associated with points around a specific region of the body'"'"'s surface.

14. A method for scanning and measuring a surface of a body of a person, including any clothing worn, comprising:

using an infrared camera system comprising an infrared light emitter that emits infrared light to illuminate the surface of the body including any clothing worn, and an infrared sensor that captures reflected infrared light resulting from the emitter'"'"'s illuminating of the surface of the body;

capturing and storing the reflected infrared light in at least a first depth image of a surface of a body from at least a first point of view while the body is in a pose and at least a second depth image of the surface of the body from at least a second point of view while the body remains in the same pose;

calculating a first set of three dimensional points from the first depth image and a second set of three dimensional points from the second depth image;

applying rotational and translational matrix transformations on the first set of three dimensional points to align said first set of three dimensional points with the second set of three dimensional points and then calculating and storing a square of the sum of the differences between the first set of three dimensional points with the second set of three dimensional points as an error;

applying poisson surface reconstruction to construct a mesh;

rigging the mesh with a biped bone structure, scaled and sized to appropriately fit the mesh;

utilizing the bone structure from within the mesh to identify one or more anthropometric points of measure; and

extracting anthropometric body circumferences by calculating the convex hull associated with points around a specific region of the body'"'"'s surface.

15. A system for scanning and measuring a surface of a body of a person including any clothing worn, comprising:

a processor;

an infrared camera comprising an infrared light emitter that emits infrared light and at least an infrared sensor that reads reflected infrared light;

a first set of computer instructions executable on the processor capable of illuminating the surface of the body with infrared light and capturing the infrared light reflected off the surface of the body;

a second set of computer instructions executable on the processor capable of capturing and storing reflected infrared light in at least a first depth image of a surface of a body from at least a first point of view while the body is in a pose and at least a second depth image of the surface of the body from at least a second point of view while the body remains in the same pose;

a third set of computer instructions executable on the processor capable of calculating a first set of three dimensional points from the first depth image and a second set of three dimensional points from the second depth image;

a fourth set of computer instructions executable on the processor capable of applying rotational and translational matrix transformations on the first set of three dimensional points to align said first set of three dimensional points with the second set of three dimensional points and then calculating and storing a square of the sum of the differences between the first set of three dimensional points with the second set of three dimensional points as an error;

a fifth set of computer instructions executable on the processor capable of applying poisson surface reconstruction to construct a mesh;

a sixth set of computer instructions executable on the processor capable of rigging the mesh with a biped bone structure, scaled and sized to appropriately fit the mesh;

a seventh set of computer instructions executable on the processor capable of utilizing the bone structure from within the mesh to identify one or more anthropometric points of measure; and

an eighth set of computer instructions executable on the processor capable of extracting anthropometric body circumferences by calculating the convex hull associated with points around a specific region of the body'"'"'s surface.

16. A method for scanning and measuring a surface of a body of a person, including any clothing worn, comprising:

using an infrared camera comprising an infrared light emitter that emits infrared light and at least an infrared sensor that reads reflected infrared light, illuminating the surface of the body with infrared light and capturing the infrared light reflected off the surface of the body;

capturing and storing the reflected infrared light in at least a first depth image of a surface of a body from at least a first point of view while the body is in a pose and at least a second depth image of the surface of the body from at least a second point of view while the body remains in the same pose;

calculating a first set of three dimensional points from the first depth image and a second set of three dimensional points from the second depth image;

applying rotational and translational matrix transformations on the first set of three dimensional points to align said first set of three dimensional points with the second set of three dimensional points and then calculating and storing a square of the sum of the differences between the first set of three dimensional points with the second set of three dimensional points as an error;

applying poisson surface reconstruction to construct a mesh;

rigging the mesh with a biped bone structure, scaled and sized to appropriately fit the mesh;

utilizing the bone structure from within the mesh to identify one or more anthropometric points of measure; and

extracting anthropometric body circumferences by calculating the convex hull associated with points around a specific region of the body'"'"'s surface; and

providing a 3D viewer application for viewing the mesh, the surface of the body, and the anthropometric points of measure on one or more displays.

17. A method for scanning and measuring a most outer surface of a body including any clothing worn of a person, comprising:

using an infrared camera comprising an infrared light emitter that emits infrared light and at least an infrared sensor that reads reflected infrared light, illuminating the surface of the body with infrared light and capturing the infrared light reflected off the surface of the body, the infrared camera further capable of stereophotogrammetry for further capturing the surface of the body;

capturing and storing the reflected infrared light in at least a first depth image of a surface of a body from at least a first point of view while the body is in a pose and at least a second depth image of the outer surface of the body from at least a second point of view while the body remains in the same pose;

calculating a first set of three dimensional points from the first depth image and a second set of three dimensional points from the second depth image;

applying poisson surface reconstruction to construct a mesh;

rigging the mesh with a biped bone structure, scaled and sized to appropriately fit the mesh;

utilizing the bone structure from within the mesh to identify one or more anthropometric points of measure; and

extracting anthropometric body circumferences by calculating the convex hull associated with points around a specific region of the body'"'"'s surface.

18. The method of claim 17, comprising providing a virtual fitting room.

19. The method of claim 18, comprising displaying one or more virtual garments on the mesh in the virtual fitting room.

20. The method of claim 17, wherein each step is performed in one or more locations selected from the group consisting of:

a home, a commercial kiosk, a retrofitted store fitting room, a gaming machine establishment, and a karaoke room.