Method and system for markerless motion capture using multiple cameras
First Claim
1. A method for markerless motion capture, comprises the steps of:
- acquiring video sequences of a subject by a plurality of cameras positioned in a surrounding relationship with the subject;
coupling the acquired video sequences to a data processing unit to be processed therein;
in said data processing unit, for each frame of a set of frames of said acquired video sequences,(a) constructing a three-dimensional (3-D) volumetric data (voxel) representation of the subject'"'"'s image, said 3-D voxel representation forming a kinematic structure including a plurality of articulated chains connected at respective joints and composed of said 3-D voxels,(b) transforming said 3-D voxels of said kinematic structure to a Laplacian Eigenspace, each of said 3-D voxels being mapped to a respective one of a plurality of smooth 1-dimensional (1-D) curves in the Laplacian Eigenspace according to a position of said each 3-D voxel in said 3-D voxel representation,(c) segmenting said 1-D smooth curves by 1-D spline fitting in the Laplacian Eigenspace,(d) registering said segmented 1-D smooth curves to a graphical subject model,(e) computing a skeleton curve of the subject from said segmented 1-D smooth curves,(f) estimating a stature of the subject from said skeleton curves computed from said set of frames of said acquired video sequences,(g) constructing an initial skeleton model having skeleton parameters corresponding to the stature of the subject,(h) computing pose parameters of said initial skeleton model, and(i) constructing an initial subject model based on said initial skeleton model parameters and said pose parameters thereof, and displaying said initial subject model at a display unit.
1 Assignment
0 Petitions
Accused Products
Abstract
Completely automated end-to-end method and system for markerless motion capture performs segmentation of articulating objects in Laplacian Eigenspace and is applicable to handling of the poses of some complexity. 3D voxel representation of acquired images are mapped to a higher dimensional space (k), where k depends on the number of articulated chains of the subject body, so as to extract the 1-D representations of the articulating chains. A bottom-up approach is suggested in order to build a parametric (spline-based) representation of a general articulated body in the high dimensional space followed by a top-down probabilistic approach that registers the segments to an average human body model. The parameters of the model are further optimized using the segmented and registered voxels.
-
Citations
20 Claims
-
1. A method for markerless motion capture, comprises the steps of:
-
acquiring video sequences of a subject by a plurality of cameras positioned in a surrounding relationship with the subject; coupling the acquired video sequences to a data processing unit to be processed therein; in said data processing unit, for each frame of a set of frames of said acquired video sequences, (a) constructing a three-dimensional (3-D) volumetric data (voxel) representation of the subject'"'"'s image, said 3-D voxel representation forming a kinematic structure including a plurality of articulated chains connected at respective joints and composed of said 3-D voxels, (b) transforming said 3-D voxels of said kinematic structure to a Laplacian Eigenspace, each of said 3-D voxels being mapped to a respective one of a plurality of smooth 1-dimensional (1-D) curves in the Laplacian Eigenspace according to a position of said each 3-D voxel in said 3-D voxel representation, (c) segmenting said 1-D smooth curves by 1-D spline fitting in the Laplacian Eigenspace, (d) registering said segmented 1-D smooth curves to a graphical subject model, (e) computing a skeleton curve of the subject from said segmented 1-D smooth curves, (f) estimating a stature of the subject from said skeleton curves computed from said set of frames of said acquired video sequences, (g) constructing an initial skeleton model having skeleton parameters corresponding to the stature of the subject, (h) computing pose parameters of said initial skeleton model, and (i) constructing an initial subject model based on said initial skeleton model parameters and said pose parameters thereof, and displaying said initial subject model at a display unit. - View Dependent Claims (2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13)
-
-
14. A system for markerless motion capture, comprising:
-
a plurality of cameras disposed in surrounding relationship with a subject, a data processing unit coupled to said plurality of cameras to receive video sequences therefrom, said data processor unit processing said video sequences and building a subject model, said subject model being presented on a display unit, wherein said data processor unit includes; (a) a voxel constructing unit constructing a 3-dimensional (3-D) volumetric data (voxel) representation of the subject image from said video sequences acquired by said plurality of cameras, said 3-D voxel representation including a kinematic structure composed of a plurality of articulated chains connected at respective joints, (b) a Laplacian Eigenspace (LE) transformation unit mapping the 3-D voxel representation to the LE to form a plurality of smooth 1-dimensional (1-D) curves in the LE, (c) segmenting unit for segmenting the 1-D curves in the LE, (d) model unit constructing a graphical model of the subject, (e) registration unit registering said segmented 1-D curves to the graphical model, (f) skeleton curve computing unit computing skeleton curve of the subject from said segmented 1-D curves, and (g) estimation unit estimating parameters of said subject model and the subject pose parameters. - View Dependent Claims (15, 16, 17, 18, 19, 20)
-
Specification