×

Joint audio-video facial animation system

  • US 10,586,368 B2
  • Filed: 12/29/2017
  • Issued: 03/10/2020
  • Est. Priority Date: 10/26/2017
  • Status: Active Grant
First Claim
Patent Images

1. A method comprising:

  • accessing audio data and video data at a client device, the audio data comprising a speech signal;

    determining locations of a set of facial landmarks based on the video data;

    identifying a user profile based on the locations of the set of facial landmarks, the user profile comprising a selection of a user avatar;

    generating a weighted finite state transducer (WFST) based on at least the speech signal of the audio data;

    performing a breadth-first search upon an output of the WFST;

    determining a phone sequence based on the breadth-first search;

    generating a first facial model based on the locations of the set of facial landmarks;

    generating a second facial model based on the phone sequence;

    constructing a composite facial model based on the first facial model, the second facial model, and the selection of the user avatar; and

    causing display of the composite facial model at the client device.

View all claims
  • 1 Assignment
Timeline View
Assignment View
    ×
    ×