How to use FaceIt on MakeHuman models?

[MrColz]

Hello,

I have done the whole process of assigning vertices and aligning the face templates (the teeth were a pain haha) but when the rig is generated the face really doesn't want to move. The only key that worked perfectly were the eyes. I left some pictures here. Any ideas on how to fix?

https://imgur.com/a/EQMoD0C

[joepal]

I have no idea. It would be hard to give an answer since I don't own FaceIt.

Maybe you already had a rig in the character, such as the default skeleton? I can see how it could be a problem if you try to mix two armature modifiers in the same mesh.

[DrHas]

I'm working on it as we speak. The issue seems to be Arkit and finding a universal way to record facial mocap for retargeting. I.E Apple vs Android. I don't use Crapple, terrible digital dictators imo. I've had success in rigging and getting data transferd, but the bones are an issue. This could be a mistake at my end though and my lack of knowledge. I've started trying to use Audio2face NVIDA and Omniverse to create mocap, which Faceit also can interpret...I'm still struggling with retargeting data to model. This may be an Omniverse issue and again, my complete lack of knowledge using said software. I'm playing with it, but it will take time to work through. Audio2face is still a work in progress too. Faceit can also do FBX - there's a free Mocap app for android, but again I haven't been able to transfer data. My feeling is, it's only a matter of time, that it works but I have to figure out what I'm doing wrong or indeed not doing right! Big difference. One thing I am doing and trying is importing model into Blender with no rig. I then use Facit and Rigify. I'm very knew to rigging...it's the most boring part of the whole process lol. My point being, I'm knee deep in this stuff and bogged down. I'm having better luck with BVH files and the CMU with face rig in Blender.

[DrHas]

I forgot, I'm goinb back to Blender 2.93 LTS to see if this helps. I will be continuing these experiments over the coming weeks. I'm working on other animation projects at the moment and working on technical difficulties. I tend to start trouble shooting...get bored. Start work or trouble shooting something else, get bored and return to what I was doing before. Like voice cloning lol. I do get things done though.

[DrHas]

I've got it working. There's a couple of ways to create facial mocap using Faceit. The issue is, you need an Iphone for one of the ways. Although, there is an addon, on Blender market, but it doesn't record the full 52 arc shapes. So, I use NVidia Audio2face which is free. I will, because I have been working on some how to vid on this very topic, but I will try and finish the Audio2face vid today for u. The processes of Audio2face are complicated, until u know the process. Once you have it though, it's really quick.

There's also BlendAr, which is free again, and this time uses Rigify as the rig. Faceit isn't needed. And better still, the facial mocap can be captured via an app - both Android and iOS. I've almost finished a how to vid on this too. The issue with this way is Rigify, parenting, teeth and the tounge. I've almost got there with this too. With some jiggery pokery, I almost fixed the teeth issue the other night. You have to reparent stuff, change the type of parent and do a bit of weight painting. All of which, isn't my speciality, but I'm getting there.

I really need to pull my finger out, I've been working on this shit for months, along with the skin shader, which I've improved by combining a paid for addon and combining the best bits of the Makehuman shader. This is on top of transferring my 'real world' photography skills into Blender. I know lol. I'm a bored, mad bad and dangerous to know polymath with ADHD speeding through the universe lol. Don't take me too seriously...I don't. I will get right onto how to vid.

Ben

[Hawas2001]

Hey, it sounds like you've made great progress! For the face movement issue, make sure the facial bones and weight painting are properly assigned. Sometimes, the face can be tricky if the rigging isn't fully adjusted. Try checking the influence of the face bones and tweaking the vertex weights a bit. Let me know if that helps!