Page 1 of 2

Quick and easy lip syncing

PostPosted: Sun Jun 11, 2017 7:09 pm
by blindsaypatten
VscorpianC has a nine-part video tutorial on using Papagayo to get a MakeHuman character to lip sync to some spoken audio. If you just want to play around with it a bit to see if it's something you want to make use of there is a much quicker way to get started. A lot of her tutorial deals with creating new visemes because the ones that are included in the MHX2 runtime are too extreme. What I don't think she realized was that the visemes can just be scaled in the graph editor. This way you can create anything from barely any movement all the way up to exaggerated yelling. You can select and scale particular channels, so you can for example scale the mouth open movements, which move the jaw, down to something appropriate for normal speaking.

The Papagayo 1.2 distribution includes some samples so if you want (near) instant gratification you can install Papagayo, open one of the samples, and export a .dat MOHO file. You can then load that with the MHX2 runtime and voila, you have a talking character. You will most likely want to scale some or all of the channels down in the graph editor unless you want really exaggerated movement. Here's an animation of an excerpt from one of the samples:
https://www.youtube.com/watch?v=DeLCKaiWYso

Papagayo can be downloaded from:
http://www.lostmarble.com/papagayo/

As an interesting aside, I find that still renders during the speech can sometimes add a little sense of movement and life to the character versus a neutral expression.

Re: Quick and easy lip syncing

PostPosted: Sun Jul 09, 2017 9:34 pm
by westli
How do you scale channels down in the graph editor?

Re: Quick and easy lip syncing

PostPosted: Mon Jul 10, 2017 3:09 am
by blindsaypatten
westli wrote:How do you scale channels down in the graph editor?


Select the channels and then type sy to scale the y values.

Re: Quick and easy lip syncing

PostPosted: Tue Jul 11, 2017 10:03 pm
by jujube
Wait, there are visemes in mhx2?

Re: Quick and easy lip syncing

PostPosted: Tue Jul 11, 2017 10:43 pm
by loki1950
jujube wrote:Wait, there are visemes in mhx2?


Yes there are.

Enjoy the Choice :)

Re: Quick and easy lip syncing

PostPosted: Thu Jul 13, 2017 12:45 am
by jorgeo
If I understand correctly, you setup the mp3 and text and then export each lipsync animation individually?

I went another route and used a game engine (Unity) to script mouth animations using audio volume sampling. It's crude (managing rotation just in the jaw), but I can add any audio file in there and it will auto lipsync.

Sample (First 5 seconds).
https://youtu.be/V7_Qi-rMjRM

I assume when I have time I might be able to move other mouth bones accordingly (maybe depending on pitch?) to make it seem more natural. I just don't have much experience in mouth movements.

Re: Quick and easy lip syncing

PostPosted: Thu Jul 13, 2017 7:02 pm
by brkurt
jorgeo wrote:If I understand correctly, you setup the mp3 and text and then export each lipsync animation individually?

I went another route and used a game engine (Unity) to script mouth animations using audio volume sampling. It's crude (managing rotation just in the jaw), but I can add any audio file in there and it will auto lipsync.

Sample (First 5 seconds).
https://youtu.be/V7_Qi-rMjRM

I assume when I have time I might be able to move other mouth bones accordingly (maybe depending on pitch?) to make it seem more natural. I just don't have much experience in mouth movements.


I created an in-depth approach to lip syncing that is very accurate, using a Makehuman model with Blender poses. Rather than Papagayo, I used Audacity, and set up a 'rule of thumb' for viseme movements: 2 frames for consonants; 4 frames for vowels.

This approach allows one to fine-tune the visemes with shape keys, rather than starting with shapes like mhx2.

http://www.makehumancommunity.org/forum/viewtopic.php?f=15&t=10750&p=24541&hilit=myster+of+o#p24541

It works quite well:

https://www.youtube.com/watch?v=R0zJQntwkpw

Re: Quick and easy lip syncing

PostPosted: Wed Nov 14, 2018 3:03 am
by Joshea
I've been doing the lipsync on MakeHuman with shape keys, but would like to try with the visemes. Where would I find them? They're not in my pose library. I'm using Blender 2.79b

Any comments would be appreciated. :D

Re: Quick and easy lip syncing

PostPosted: Wed Nov 14, 2018 4:14 am
by loki1950
The visemes are part of the mhx2 control panel.

Enjoy the Choice :)

Re: Quick and easy lip syncing

PostPosted: Wed Nov 14, 2018 3:41 pm
by Joshea
loki1950 wrote:The visemes are part of the mhx2 control panel.

Enjoy the Choice :)


Thanks :) . I'll post if it works out for me.