Animation retargeting - anyone still interested?

[InezDias]

Hello everyone,

Somewhere I read that animation retargeting would be something that people could use help with, but as a newcomer it is hard to distinguish which information is up to date and which is not. Also, would that be about the core MakeHuman application or for MPFB? I think it would be quite doable as a vanilla Blender Plugin, so I assume that could be also integrated into MPFB...

But I have no idea if anyone is still interested in the topic or where even to ask the question (I am not on facebook)

Cheers

Inez

[tomcat]

Hello everyone,
Somewhere I read that animation retargeting would be something that people could use help with, but as a newcomer it is hard to distinguish which information is up to date and which is not.

I'm using a translator and may not fully understand the question. It is possible to assign animations using the Differeomorphic BVH and FBX Retargeter. A fresh version works with the new programs.

The tutorial is relevant for:

• Godot 4.4 beta 3
• Blender 4.3.2
• MakeHuman 1.3.0
• Diffeomorphic BVH and FBX Retargeter 4.2.1
• MPFB 2.0.8

[InezDias]

Oh I see, thank you for your reply. I had been thinking about creating a retarger built into MHFB itself, but I guess it is not needed then.

[joepal]

This said, integrating retargeting functionality within MPFB will make sense at some point.

My math skills are way too sub par though. I'd need help from someone who understands the relevant calculations.

[InezDias]

This said, integrating retargeting functionality within MPFB will make sense at some point.

My math skills are way too sub par though. I'd need help from someone who understands the relevant calculations.

But that is precisely what I am offering. The basic retargeting between armatures with the same rest pose I have already done; currently working on converting from T-Pose to A-Pose and vice vesa, which is the tricky part.

And then some automated detection of which bone is which (like the Mixamo website does), plus a gui for the user to refine that mapping, similar to what game engines have. Where I am lacking skils is really the gui part in Blender, but with the help of tutorials I can probably manage.

[RobBaer]

It seems to be gone now, but there used to be a MakeWalk plugin (Thomas Larrson) with some of this functionality didn't there?

[tomcat]

It seems to be gone now, but there used to be a MakeWalk plugin (Thomas Larrson) with some of this functionality didn't there?

Actually Differeomorphic BVH and FBX Retargeter is the evolution of this plugin. As far as I understand, Thomas used to work with MH team? Maybe it makes sense to contact him directly?

[joepal]

But that is precisely what I am offering. The basic retargeting between armatures with the same rest pose I have already done; currently working on converting from T-Pose to A-Pose and vice vesa, which is the tricky part.

And then some automated detection of which bone is which (like the Mixamo website does), plus a gui for the user to refine that mapping, similar to what game engines have. Where I am lacking skils is really the gui part in Blender, but with the help of tutorials I can probably manage.

Sounds great.

I'm able to provide the less complicated parts of the solution, such as the UI stuff, a relevant place to put the logic in the code etc. If you want, I can create a code branch and set up an outline with a panel and a button and stubbed code for where the actual retargeting should ideally end up.

I'm planning on making a tutorial on developing for MPFB, but it's a bit down the list as I'm prioritizing the getting started and basic operations videos atm.

[InezDias]

Ok, nice.
I have a first version working, with pose conversion; but no automated bone matching yet. I will post some screenshots tomorrow so you can see what problem I have with the gui (basically I would like to dispay bones in a tree view like Blender does in the outliner. But I have no idea how to make a proper tree view (found a tutorial that does it with buttons but that look rather crude).

[InezDias]

Sorry, I got delayed by some RL business. But here are now some screenshots:

I have two sekletons, one of which is in A-Pose and the other in T-Pose:

For the makhuman skeleton, I am applying a T-Pose. The other one has an animation:

In the timeline, the animation can be previewed:

The retargeting will occur from the active armature to the first selected armature (if there was more than one target armature selected, I could copy the animation to all of them, just did not yet get around to implementing that)

Here I am actually unsure - the normal Blender paradigm seems to be "selected to active" instead of "active to selected". But I was thinking that there is alway only on source armature, while there can be multiple targets - just as there is only one active, but multiple selected objects.

Now you need to define the mapping of bones in the source sekeleton to bones in the target sekeleton (this will eventually be (semi-) automated):

The right side of the list (where you select the source bones) is quite nice - with the Eyedropper tool you can select bones from the Outliner rather conveniently. The left side is what i am struggling with: currently it is a UIList, with some buttons for expanding the items, but the programming behind it is quite complicated plus the result looks ugly. What I would like is a tree representation like Blender itself has in the outliner, but I don't know how to do that.

Anyway, once all the mappings are done, push "retarget" and voilà:

(Never mind the fingers, I haven't mapped them - doing this manually when the bones don't even have sensible names that indicate which is right and which is left is quite a lot of work.) Of course the retargeting will only be as good as the poses you start with match; there is still some room for improvement.

Well, this is what i have so far - next steps are
- including the global transforms of the armatures into the calucaltion (right now I have cheated a little by applying all transforms up front)
- adding root motion (the lack of which is why the left character in the obove picuture has its feet in the air while the right one stays on the ground)
- automating the bone mapping
- lots of tests with different armatures

Any comments or suggestions for improvement?