Page 1 of 1

Sex or Fighting?

PostPosted: Sun Mar 02, 2014 3:39 am
by brkurt
Here is my first lip synching exercise using my Alpha 8 Gina Carano model:


Here is the video:

It has two immediate flaws: (1) I forgot to use my 'I' viseme, and substituted 'Aah'; (2) there needs to be a rest between 'it' and 'is'.
The second point is important, because in the real world, we let our lips flap between words, where in an animation there should be no mouth breathing.

Here is the second video, now without the rig:

Here is the third video, where I have applied the rule of 'no mouth breathing' for every word.

Another version of the same, this time in YouTube:

Note how much easier it is to read her lips.
Also, note how the 'b' of 'about' and the 'f' of fighting sync up with the model; that's what I meant about inserting the viseme pose 1/10th of a second before peak amplitude of the sound.

After a lot a background reading on the subject, I'm beginning to think that a fully automated approach will never be all that satisfying. But this one is done.
And...I haven't used any advanced techniques. No subsurfing, no shape keys, no cover-it-up UV maps. That means all those tools are still available.
The big breakthrough for me was to realized that the bones must be moved and rotated in all three axes (x, y & z), not just x & y.

Here is the model with UV maps. The flesh tone was supplied by Makehuman; the eyes by 3DXO.

Image OMG She's blonde!

And here she is on Vimeo. Our state-of-the-art network is having problems with 'packet shaping', so I can't upload to YouTube.

Here is the breakdown for the lip sync:

<!-- espeak -f sex_or_fighting_espeak1.txt -g 10 -m -p 60 -ven+f2 -w /home/brkurt/Documents/animatics/voices/sex_or_fighting_espeak1.wav -->

Ha! Are we talking about sex or fighting? Fighting it is, then!

<!-- viseme is located at the peak amplitude, not just the beginning. Zoom in to check.-->
<!-- for example the Ha! begins at 1.15s x 25 fps = 28.75 or roughly frame 28 -->
<!-- ng & r consonants end with a quick eh and then rest -->
<!-- extended phoneme breakdowns 2 frames each -->

Time(s) Frame Phoneme Breakdown
0.0 1 rest
1.15 28 Ha!! [aah]
1.40 35 rest
2.00 50 Are [a & r]
2.25 56 we [w & ee]
2.60 65 tal [t & ah & l]
2.8 70 ki [k & eh]
3.0 75 ng [n & g]
3.1 77 a [aah]
3.2 80 bout [b & o & t]
3.5 87 s [sz]
3.6 90 e [eh]
3.8 95 x [cdgkr & sz]
4.1 102 or [o & r]
4.5 112 figh [f & i]
4.8 120 ting? [t & n & eh]
5.0 125 rest
5.5 137 Ger! [g & r]
6.4 160 Figh [f & i]
6.7 167 ting, [t & eh & n]
7.3 182 it [eh & t]
7.4 185 rest
7.6 190 is, [eh & sz]
8.3 207 then! [th & eh & n]
8.7 217 rest

I'm going to finish off by add six Ekman emotion poses, placing Smile before the lip sync, and Surprise, Disgust, Fear, Sadness, Anger afterwards.

Here is the Vimeo animatic: