Good, bad and hopes

What the talk is about

Andrey Sibiryakov talks about what did not work for him, using automatic lip-sync for the lower face as the example. He shows how hand-tuned phoneme settings gave way to scans of an actor, yet machine audio processing still made errors on fast speech, and a script that swallows short phonemes did not fix it. The breakthrough came when he first analysed the actual mechanics of articulation: the jaw drives vowels, the lower lip shapes both vowels and consonants and drops before speech, and the upper lip moves slower and with a delay. Phonemes also need separate poses for open and closed mouth. Writing the rules took half an hour and the prototype a few days. He credits persistence and patience, and recommends resources on facial expressions. Useful for facial animators and technical artists.