COMPUTER-IMPLEMENTED METHOD AND APPARATUS FOR TRACKING AND RESHAPING A HUMAN SHAPED FIGURE IN A DIGITAL WORLD VIDEO

• US 20130330060A1
• Filed: 11/29/2011
• Published: 12/12/2013
• Est. Priority Date: 11/29/2010
• Status: Active Grant

• Current Assignee: Max-Planck-Gesellschaft Zur Foerderung Der Wissenschaften e.V. (Max-Planck-Gesellschaft zur Frderung der Wissenschaften e)
• Original Assignee: Thorsten Thormaehlen
• Inventors: Seidel, Hans-Peter, Theobalt, Christian, Thormaehlen, Thorsten, Jain, Arjun
• Granted Patent: US 9,191,579 B2
• Time in Patent Office: Days
• US Class Current: 386/280
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: H04N 5/262 Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects

Abstract

The invention concerns a computer-implemented method for tracking and reshaping a human-shaped figure in a digital video comprising the steps: acquiring a body model of the figure from the digital video, adapting a shape of the body model, modifying frames of the digital video, based on the adapted body model and outputting the digital video.

Claims

1. Computer-implemented method for tracking and reshaping a human-shaped figure in a digital video comprising the steps:

acquiring a body model of the figure from the digital video;

adapting a shape of the body model;

modifying frames of the digital video, based on the adapted body model; and

outputting the digital video.

2. A method according to claim 1, wherein the body model comprises a kinematic skeleton model and a surface mesh.

3. A method according to claim 1, wherein the body model uses a normal skinning approach for modeling pose-dependent surface mesh adaptation.

4. A method according to claim 1, wherein pose and shape parameters of the model are acquired by measuring the misalignment of a silhouette boundary of the reprojected model with a silhouette boundary of the figure.

5. A method of claim 4, wherein acquiring pose and shape parameters further comprises minimizing a misalignment between the figure in a video frame and the body model re-projected onto the video frame.

6. A method according to claim 5, further comprising the step of providing an interface in which a user may explicitly mark additional image points to be tracked and in which broken trajectories may be linked.

7. A method according to claim 1, further comprising the steps of

providing an interface for setting shape attributes of the body model,

wherein the shape of the body model is adapted by mapping shape attributes set via the interface to the shape.

8. A method according to claim 7, wherein the shape attributes of the body model comprise at least one of height, weight, breast girth, waist girth, hips girth, leg length and muscularity.

9. A method of claim 1, wherein the body model is acquired using the digital video and additional sensor equipment.

10. A method according to claim 9, wherein the additional sensor equipment is a depth camera or another scanner or device for capturing 2D or 3D static or time-varying shape, either online or off-line.

11. A method according to claim 10, wherein the additional sensor equipment is an optical motion capture system, an electromagnetic motion capture, any form of motion capture suit or an exoskeleton for pose estimation.

12. A method according to claim 1, wherein the motion of the model is tracked in real-time from the video and/or depth data provided by an additional sensor.

13. A method according to claim 1, wherein the digital video is output in real time.

18. Computer-readable medium, comprising instructions that, when executed, implement a method according to claim 1.

14. Apparatus for tracking and reshaping of a human-shaped figure in a digital video comprising:

a module for acquiring a body model of the figure from the digital video;

a module for adapting a shape of the body model;

a module for modifying frames of the digital video, based on the adapted body model; and

means for outputting the digital video.

15. Apparatus according to claim 14, wherein the module for acquiring a body model comprises a depth camera or another scanner or device for capturing 2D or 3D static or time-varying shape, either online or off-line.

16. Use of an apparatus according to claim 14 for visualizing a human person in a different shape.

17. Use of an apparatus according to claim 14 for warping images of a human person, based on an image of a different human person.