Quick and easy lip syncing

If your topic doesn't fit anywhere else, put it here.

Moderator: joepal

Quick and easy lip syncing

Postby blindsaypatten » Sun Jun 11, 2017 7:09 pm

VscorpianC has a nine-part video tutorial on using Papagayo to get a MakeHuman character to lip sync to some spoken audio. If you just want to play around with it a bit to see if it's something you want to make use of there is a much quicker way to get started. A lot of her tutorial deals with creating new visemes because the ones that are included in the MHX2 runtime are too extreme. What I don't think she realized was that the visemes can just be scaled in the graph editor. This way you can create anything from barely any movement all the way up to exaggerated yelling. You can select and scale particular channels, so you can for example scale the mouth open movements, which move the jaw, down to something appropriate for normal speaking.

The Papagayo 1.2 distribution includes some samples so if you want (near) instant gratification you can install Papagayo, open one of the samples, and export a .dat MOHO file. You can then load that with the MHX2 runtime and voila, you have a talking character. You will most likely want to scale some or all of the channels down in the graph editor unless you want really exaggerated movement. Here's an animation of an excerpt from one of the samples:
https://www.youtube.com/watch?v=DeLCKaiWYso

Papagayo can be downloaded from:
http://www.lostmarble.com/papagayo/

As an interesting aside, I find that still renders during the speech can sometimes add a little sense of movement and life to the character versus a neutral expression.
blindsaypatten
 
Posts: 586
Joined: Tue Mar 14, 2017 11:16 pm

Re: Quick and easy lip syncing

Postby westli » Sun Jul 09, 2017 9:34 pm

How do you scale channels down in the graph editor?
westli
 
Posts: 8
Joined: Thu May 22, 2014 3:45 am
Location: United States

Re: Quick and easy lip syncing

Postby blindsaypatten » Mon Jul 10, 2017 3:09 am

westli wrote:How do you scale channels down in the graph editor?


Select the channels and then type sy to scale the y values.
blindsaypatten
 
Posts: 586
Joined: Tue Mar 14, 2017 11:16 pm

Re: Quick and easy lip syncing

Postby jujube » Tue Jul 11, 2017 10:03 pm

Wait, there are visemes in mhx2?
jujube
 
Posts: 404
Joined: Fri Aug 14, 2015 10:46 pm

Re: Quick and easy lip syncing

Postby loki1950 » Tue Jul 11, 2017 10:43 pm

jujube wrote:Wait, there are visemes in mhx2?


Yes there are.

Enjoy the Choice :)
my box::HP Envy i5-6400 @2Q70GHzx4 8 Gb ram/1 Tb(Win10 64)/3 Tb Mint 19.2/GTX745 4Gb acer S243HL K222HQL
Q8200/Asus P5QDLX/8 Gb ram/WD 2Tb 2-500 G HD/GF GT640 2Gb Mint 17.3 64 bit Win 10 32 bit acer and Lenovo Ideapad 320-15ABR Win 10/Mint 19
User avatar
loki1950
 
Posts: 1219
Joined: Thu Dec 18, 2014 6:27 pm
Location: Ottawa,Ontario

Re: Quick and easy lip syncing

Postby jorgeo » Thu Jul 13, 2017 12:45 am

If I understand correctly, you setup the mp3 and text and then export each lipsync animation individually?

I went another route and used a game engine (Unity) to script mouth animations using audio volume sampling. It's crude (managing rotation just in the jaw), but I can add any audio file in there and it will auto lipsync.

Sample (First 5 seconds).
https://youtu.be/V7_Qi-rMjRM

I assume when I have time I might be able to move other mouth bones accordingly (maybe depending on pitch?) to make it seem more natural. I just don't have much experience in mouth movements.
User avatar
jorgeo
 
Posts: 24
Joined: Fri Aug 05, 2016 7:35 am
Location: Austin, TX

Re: Quick and easy lip syncing

Postby brkurt » Thu Jul 13, 2017 7:02 pm

jorgeo wrote:If I understand correctly, you setup the mp3 and text and then export each lipsync animation individually?

I went another route and used a game engine (Unity) to script mouth animations using audio volume sampling. It's crude (managing rotation just in the jaw), but I can add any audio file in there and it will auto lipsync.

Sample (First 5 seconds).
https://youtu.be/V7_Qi-rMjRM

I assume when I have time I might be able to move other mouth bones accordingly (maybe depending on pitch?) to make it seem more natural. I just don't have much experience in mouth movements.


I created an in-depth approach to lip syncing that is very accurate, using a Makehuman model with Blender poses. Rather than Papagayo, I used Audacity, and set up a 'rule of thumb' for viseme movements: 2 frames for consonants; 4 frames for vowels.

This approach allows one to fine-tune the visemes with shape keys, rather than starting with shapes like mhx2.

http://www.makehumancommunity.org/forum/viewtopic.php?f=15&t=10750&p=24541&hilit=myster+of+o#p24541

It works quite well:

https://www.youtube.com/watch?v=R0zJQntwkpw
brkurt
 
Posts: 1100
Joined: Sun Feb 17, 2008 8:49 pm

Re: Quick and easy lip syncing

Postby Joshea » Wed Nov 14, 2018 3:03 am

I've been doing the lipsync on MakeHuman with shape keys, but would like to try with the visemes. Where would I find them? They're not in my pose library. I'm using Blender 2.79b

Any comments would be appreciated. :D
Joshea
 
Posts: 115
Joined: Sat Apr 29, 2017 12:45 am
Location: Ontario, Canada

Re: Quick and easy lip syncing

Postby loki1950 » Wed Nov 14, 2018 4:14 am

The visemes are part of the mhx2 control panel.

Enjoy the Choice :)
my box::HP Envy i5-6400 @2Q70GHzx4 8 Gb ram/1 Tb(Win10 64)/3 Tb Mint 19.2/GTX745 4Gb acer S243HL K222HQL
Q8200/Asus P5QDLX/8 Gb ram/WD 2Tb 2-500 G HD/GF GT640 2Gb Mint 17.3 64 bit Win 10 32 bit acer and Lenovo Ideapad 320-15ABR Win 10/Mint 19
User avatar
loki1950
 
Posts: 1219
Joined: Thu Dec 18, 2014 6:27 pm
Location: Ottawa,Ontario

Re: Quick and easy lip syncing

Postby Joshea » Wed Nov 14, 2018 3:41 pm

loki1950 wrote:The visemes are part of the mhx2 control panel.

Enjoy the Choice :)


Thanks :) . I'll post if it works out for me.
Joshea
 
Posts: 115
Joined: Sat Apr 29, 2017 12:45 am
Location: Ontario, Canada

Next

Return to General discussions about makehuman

Who is online

Users browsing this forum: No registered users and 1 guest

cron