DISTRIBUTED MARKERLESS MOTION CAPTURE

• US 20100285877A1
• Filed: 05/05/2010
• Published: 11/11/2010
• Est. Priority Date: 05/05/2009
• Status: Abandoned Application

• Current Assignee: Mixamo, Inc. (Adobe Inc.)
• Original Assignee: Mixamo, Inc. (Adobe Inc.)
• Inventors: Corazza, Stefano
• Application Number:
  US12/774,689
• Publication Number:
  US 20100285877A1
• Time in Patent Office: Days
• US Class Current: 463/32
• CPC Class Codes: A63F 13/213 comprising photodetecting means, e.g. cameras, photodiodes or infrared cells A63F13/219, A63F13/655 take precedence
• CPC Class Codes: A63F 13/335 using Internet
• CPC Class Codes: A63F 13/35 Details of game servers
• CPC Class Codes: A63F 13/355 Performing operations on behalf of clients with restricted processing capabilities, e.g. servers transform changing game scene into an encoded video stream for transmitting to a mobile phone or a thin client
• CPC Class Codes: A63F 13/428 involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
• CPC Class Codes: A63F 13/52 involving aspects of the displayed game scene
• CPC Class Codes: A63F 2300/1093 using visible light
• CPC Class Codes: A63F 2300/538 for performing operations on behalf of the game client, e.g. rendering
• CPC Class Codes: A63F 2300/6045 for mapping control signals received from the input arrangement into game commands
• CPC Class Codes: A63F 2300/6607 for animating game characters, e.g. skeleton kinematics
• CPC Class Codes: A63F 2300/8082 Virtual reality
• CPC Class Codes: G06T 13/00 Animation

Abstract

Systems and methods for performing remote markerless motion capture to drive 3D animations in real time in accordance with embodiments of the invention are described. One embodiment of the invention includes an optical device connected to a data acquisition device, where the combination of the optical device and the data acquisition device is configured to perform markerless motion capture, and a server system configured to communicate with the data acquisition device via the Internet. In addition, the server system is configured to receive motion capture data from the data acquisition device, and the server system is configured to generate motion data to animate a 3D character model based upon the received motion capture data.

Claims

1. A system configured to perform remote markerless motion capture to drive a 3D character model in real time, comprising:

an optical device connected to a data acquisition device, where the combination of the optical device and the data acquisition device is configured to perform markerless motion capture; and

a server system configured to communicate with the data acquisition device via the Internet;

wherein the server system is configured to receive motion capture data from the data acquisition device; and

wherein the server system is configured to generate motion data to animate a 3D character model based upon the received motion capture data.

2. The system of claim 1, wherein the optical device is a time of flight camera.

3. The system of claim 1, wherein:

the data acquisition device includes a game engine client configured to render 3D animations based upon 3D animation information received from the server system; and

the server system is configured to stream 3D animation information to the data acquisition device including the motion data generated by the server system based upon the received motion capture data.

4. The system of claim 3, wherein the server system is configured to control the frame rate of the generated animation data in response to the frame rate of the received motion capture data and in response to Internet bandwidth constraints.

5. The system of claim 1, wherein:

the server system is configured to match the motion capture data against a set of predetermined command gestures; and

the server system is configured to generate predetermined motion data based upon matching the motion capture data with a command.

6. The system of claim 1, wherein the server system is configured to generate motion data influenced by the received motion capture data.

7. The system of claim 6, wherein the server system is configured to generate motion data by at least retargeting the motion data to a 3D character model.

8. The system of claim 7, wherein the server system is configured to generate motion data by at least generating synthetic motion data influenced by the retargeted motion capture data.

9. The system of claim 6, wherein the server system is configured to generate motion data by at least:

generating synthetic motion data influenced by the received motion capture data; and

combining aspects of the received motion data with aspects of the synthetic motion data.

10. A method of animating a 3D character, comprising:

performing markerless motion capture using an optical device;

providing the markerless motion capture data to a remote server system;

generating motion data using the server system based upon the markerless motion capture data; and

animating a 3D character using the generated motion data.

11. The method of claim 10, wherein the optical device is a time of flight camera.

12. The method of claim 10, wherein the markeless motion capture data is expressed in terms of joint center points and joint rotation parameters.

13. The method of claim 10, further comprising:

matching the markerless motion data using the server system against a predetermined set of commands; and

generating the motion data using a predetermined motion associated with an identified command.

14. The method of claim 10, further comprising generating motion data influenced by the received motion capture data using the server system.

15. The method of claim 10, further comprising retargeting the received motion data to a 3D character model using the server system.

16. The method of claim 15, further comprising generating synthetic motion data influenced by the retargeted received motion data.

17. The method of claim 16, further comprising generating motion data based upon a combination of aspects of the synthetic motion data and aspects of the received motion data.

18. The method of claim 10, further comprising streaming 3D animation information including the generated motion capture data to a rendering engine client located remotely.

19. The method of claim 18, further comprising modifying the frame rate of the animation information streamed by the server system in response to the frame rate of the motion capture data received by the server system and the internet bandwidth constraints.