System and method for database driven action capture

• US 20100309209A1
• Filed: 06/02/2010
• Published: 12/09/2010
• Est. Priority Date: 06/05/2009
• Status: Active Grant

• Current Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Original Assignee: Disney Enterprises Incorporated (The Walt Disney Company)
• Inventors: Hodgins, Jessica, Slyper, Ronit
• Granted Patent: US 8,947,441 B2
• Time in Patent Office: Days
• US Class Current: 345/473
• CPC Class Codes: G06T 13/40 of characters, e.g. humans, animals or virtual beings

Abstract

There is provided a system and method for database driven action capture. By utilizing low cost, lightweight MEMS devices such as accelerometers, a user friendly, wearable, and cost effective system for motion capture is provided, which relies on a motion database of previously recorded motions to reconstruct the actions of a user. By relying on the motion database, calculation errors such as integration drift are avoided and the need for complex and expensive positional compensation hardware is avoided. The accelerometers may be implemented in an E-textile embodiment using inexpensive off-the-shelf components. In some embodiments, compression techniques may be used to accelerate linear best match searching against the motion database. Adjacent selected motions may also be blended together for improved reconstruction results and visual rendering quality. Various perceivable effects may be triggered in response to the reconstructed motion, such as animating a 3D avatar, playing sounds, or operating a motor.

Claims

1. A method for translating the motions of a user into perceptible effects, the method comprising:

receiving a first motion data from a plurality of motion sensing devices based on detecting the motions of the user;

selecting a first recorded motion from a database of recorded human motions by querying the database using the first motion data; and

causing a first perceptible effect in response to selecting the first recorded motion.

2. The method of claim 1, further comprising:

receiving a second motion data from the plurality of motion sensing devices, wherein the second motion data contains a subset of data from the first motion data and a new subset of data more recent in time than the first motion data;

selecting a second recorded motion from the database by querying the database using the second motion data; and

causing a second perceptible effect in response to selecting the first recorded motion and selecting the second recorded motion.

3. The method of claim 1, wherein the first perceptible effect comprises rendering the first recorded motion on a display.

4. The method of claim 2, wherein the second perceptible effect comprises rendering a smooth blending between the first recorded motion and the second recorded motion on a display.

5. The method of claim 1, wherein querying the database using the first motion data is by applying a compression algorithm to the first motion data and conducting a search using the database preprocessed with the compression algorithm.

6. The method of claim 5, wherein the search comprises a linear best match search, and wherein the compression algorithm comprises a Haar wavelet transform preserving approximately 10% of the most significant coefficients from the transform.

7. The method of claim 4, wherein the querying the database using the second motion data is by applying a compression algorithm to the second motion data and conducting a search using the database preprocessed with the compression algorithm, wherein the search further matches positional data from a final frame of the second recorded motion to positional data from a corresponding frame of the first recorded motion.

8. The method of claim 1, wherein the plurality of motion sensing devices are placed on the user by attachment to an article of clothing worn by the user.

9. The method of claim 1, wherein the plurality of motion sensing devices is one of accelerometers, triangulated ultrasonic tags, cameras, a Light Detection and Ranging (LIDAR) system, and sensors based on the heterodyne principle.

10. The method of claim 1, wherein the first perceptible effect is a variation of the first recorded motion modulated according to the first motion data.

11. A system for translating the motions of a user into perceptible effects, the system comprising:

a plurality of motion sensing devices configured to detect the motions of the user;

a database of recorded human motions;

an output device configured to generate the perceptible effects; and

a computing device having a processor configured to;

receive a first motion data from the plurality of motion sensing devices based on detecting the motions of the user;

select a first recorded motion from the database by querying the database using the first motion data; and

cause a first perceptible effect using the output device in response to selecting the first recorded motion.

12. The system of claim 11, wherein the processor of the computing device is further configured to:

receive a second motion data from the plurality of motion sensing devices, wherein the second motion data contains a subset of data from the first motion data and a new subset of data more recent in time than the first motion data;

select a second recorded motion from the database by querying the database using the second motion data; and

cause a second perceptible effect using the output device in response to selecting the first recorded motion and selecting the second recorded motion.

13. The system of claim 11, wherein the output device comprises a display, and wherein the first perceptible effect comprises rendering the first recorded motion on the output device.

14. The system of claim 12, wherein the output device comprises a display, and wherein the second perceptible effect comprises rendering a smooth blending between the first recorded motion and the second recorded motion on a display.

15. The system of claim 11, wherein the processor of the computing device is further configured to query the database using the first motion data by applying a compression algorithm to the first motion data and conducting a search using the database preprocessed with the compression algorithm.

16. The system of claim 15, wherein the search comprises a linear best match search, and wherein the compression algorithm comprises a Haar wavelet transform preserving approximately 10% of the most significant coefficients from the transform.

17. The system of claim 14, wherein the processor of the computing device is further configured to query the database using the second motion data by applying a compression algorithm to the second motion data and conducting a search using the database preprocessed with the compression algorithm, wherein the search further matches positional data from a final frame of the second recorded motion to positional data from a corresponding frame of the first recorded motion.

18. The system of claim 11, wherein the plurality of motion sensing devices are placed on the user by attachment to an article of clothing worn by the user.

19. The system of claim 11, wherein the plurality of motion sensing devices is one of accelerometers, triangulated ultrasonic tags, cameras, a Light Detection and Ranging (LIDAR) system, and sensors based on the heterodyne principle.

20. The system of claim 11, wherein the first perceptible effect is a variation of the first recorded motion modulated according to the first motion data.