Exporting mhx2 facial animation to Unity

[alex_d]

Hi there,

I'm trying to make a MakeHuman model talk in Unity.

For the lipsync control I want to use Annosoft lipsync, it works by recording the bone locations on 17 frames (each containing different mouthshapes like o, a, e, ...) and then blends those shapes with a script.
More info and annosoft unity package w working fbx model here: http://forum.unity3d.com/threads/lipsyn ... er.123701/

So I was hoping I could use MHX2 facial drivers to create 17 frames with mouth shapes for talking. (like this: http://www.annosoft.com/docs/Visemes17.html)
I used the sliders (checked drivers on import) and right clicked the values to set keyframes. It shows up in the timeline and on the curves panel under 'subAction', but when I import this model in Unity (as a .blend or .fbx) it doesn't show me any animation... Is it because the keyframe info is being lost, since it targets the mhx runtime?

Are those drivers actual bones or are they procedurally controlled or something? I'm still a bit new to MH and 3D in general so I'm a bit lost as to what I can do now.

[ThomasL]

The sliders are object level properties of the armature, which drive the actual shapekey values on the mesh. It was done this way because I wanted it to work with file linking in Blender, where you can access the rig but not the mesh in the scene file. Unfortunately, I don't think Unity will understand drivers and properties. Definitely not if you export with fbx, and probably it will ignore that info when it reads the .blend file as well.

What you can try is to use the face panel. If you check Face Shapes and uncheck Face Drivers (unsure about the exact names), then an option called Face Panel should appear at the very bottom. The shapekeys are now driven by bones instead of properties, which have a better chance to work in Unity. I'm still very unsure if drivers will work at all, though.

If you check neither Face Drivers nor Face Panel, then you can manipulate the shapekeys directly in the mesh context. This will be a bit cumbersome but has the best chance to work with Unity, I think.

[alex_d]

Awesome. Thanks for the reply! I should be able to figure it out now.

[fletcherkildren]

Did you manage to get this to work?

[alex_d]

Sorry for the slow response - I didn't because Annosoft uses bone-driven animation (though I guess with some fiddling the scripts could be adapted) and the drivers in Blender that control the morphs are lost when importing into Unity.

Unity can however import the morph targets themselves, so with some fiddling to the script it might be possible to do it like that, but I didn't opt for that. (If Unity doesn't import your blendshapes, make sure your Mask Modifiers are applied/turned off in blender)

I ended up using the pitchipoy meta-rig which has a good facial bonestructure and solved it that way. Still fiddling around with the rig though, because it's a bit of a pain to get it working properly with Mecanim, but I've finally found a good solution for that.

[duststorm]

The nightly version (1.1) has facial expressions using bones. Perhaps that would work.

[AjayT]

Hi,

I'm trying to create a face animation of my baby son smiling, yawning, sleeping, crying etc. I'd like to do this using python API commands in Blender.

So far I've managed to import my baby in .mhx2 format, and access the Face Components menu,

Code: Select all

bpy.ops.import_scene.makehuman_mhx2( filepath = this_m ) # import baby

bpy.ops.mhx2.add_facerig_drivers()

Now I'm trying to do the face animations, but I'm having problems with how to set the sliders, which drive the actual shapekey values on the mesh. For example,

Code: Select all

bpy.data.objects[ model_ID_str ].MhxFaceRigDrivers.["MfaCheeksPump"] = 1

gives the error

Code: Select all

SyntaxError: invalid syntax

I know it's possible to do something like this as there's a Youtube video of the MakeHuman baby in the 1.1 nightly build here [MakeHuman 1.1 new expression system][https://www.youtube.com/watch?v=upzQfgWA8Sw]

Really appreciate any help you could give?

Thanks, Aj


PS, I can import the test.bvh faceshift file from [http://support.faceshift.com/support/solutions/articles/37249-test-bvh-c3d-sequence]

Code: Select all

bpy.ops.mhx2.load_faceshift_bvh( filepath="/home/../BlenderProjects/tbo_test.bvh" )

and take works fine?

[AjayT]

The nightly version (1.1) has facial expressions using bones. Perhaps that would work.

Hi Duststorm,

are the facial expressions in the nightly version (1.1) the same as in the file,

Code: Select all

makehuman/data/poseunits/face-poseunits.bvh

I'd like to reproduce them in Blender for the baby MakeHuman, as in the YouTube video, https://www.youtube.com/watch?v=upzQfgWA8Sw

I tried to do this, but I could not import the .bvh file using the mhx2 addon?

Regards,

Aj

[ThomasL]

Code: Select all

bpy.data.objects[ model_ID_str ].MhxFaceRigDrivers.["MfaCheeksPump"] = 1

There is a dot between MhxFaceRigDrivers and ["MfaCheeksPump"].

It is not necessarily a good idea to try to script the mhx2 importer. It might work, but the code was not designed with that in mind. If it does not work, you are on your own.

PS, I can import the test.bvh faceshift file from [http://support.faceshift.com/support/solutions/articles/37249-test-bvh-c3d-sequence]

Code: Select all

bpy.ops.mhx2.load_faceshift_bvh( filepath="/home/../BlenderProjects/tbo_test.bvh" )

and take works fine?

I don't think so. Faceshift support in mhx2 only works if you use the default rig exported from MH. Moreover, it only works with the default rig from last summer, not the present one, because things have changed. Primarely the roll angles are different now which completely screws up the old animations. Faceshift support would be cool though, either with shapekeys or with face bones. If it happens within MH I will see if the mhx2 format can be modified so it can be exported to Blender.

[AjayT]

HI Thomas,

thank you very much for taking the time to reply - I appreciate it

Unfortunately I could not get the rig drivers to work from a Python script, and I think it's probably best to just use basic bone rotations, as that would be more portable?

I did manage to get the Faceshift bvh file working, though I can't remember how I created the rig that it works with ?

I posted a video on youtube on here,

https://www.youtube.com/watch?v=inAr0AutHyA

It's part of a project I'm working on to infer facial expressions and movements directly from pictures and video data, rather than having to use shape keys and bvh file's. If that sounds interesting to you I'd be very happy to explain more if you wanted?

Kind regards,

Aj