To transfer poses from MH to the Blender pose library

Tech support and suggestions forum. If you only have a basic question on how to get started, please use the "newbies" forum in the community section.

Moderator: joepal

To transfer poses from MH to the Blender pose library

Postby tomcat » Mon May 31, 2021 8:50 am

Objective: to transfer poses from MH to the Blender pose library (for further use in the game engine).

When directly importing the model, everything is fine, but such pose cannot be used in the library:

Image

  1. The character in pose A (None) from MH is transferred to Blender;
  2. A pose is specified in the MH character;
  3. Skeleton Sync "Sync with MH" is used in Blender;
  4. Result/Comparison:

Image Image

If the character is given "automatic weights" in Blender before the pose transfer, the situation is slightly better, but not perfect (outline model with automatic weights):

Image

Is there any way to fix this?

Translated with http://www.DeepL.com/Translator (free version)
Foreigners' reactions to Russian "Bird's Milk" candies
— Are your birds being milked?
— In Russia everyone is milked. Here even the zucchini is used to make caviar.
User avatar
tomcat
 
Posts: 336
Joined: Sun Sep 27, 2015 7:53 pm
Location: Moscow (Orcish Stan), The Aggressive Evil Empire

Re: To transfer poses from MH to the Blender pose library

Postby joepal » Tue Jun 01, 2021 5:58 am

Unfortunately, the short answer is probably "no". The sync functionality is a best effort, but won't be exact. There will be many circumstances where it will be slightly off, or even produce strange results.

This is a known problem, which boils down to differences in how blender and MH see the bone rotation transforms. It is theoretically fixable, but so far the math involved has been beyond us.
Joel Palmius (LinkedIn)
MakeHuman Infrastructure Manager
http://www.palmius.com/joel
joepal
 
Posts: 4465
Joined: Wed Jun 04, 2008 11:20 am

Re: To transfer poses from MH to the Blender pose library

Postby tomcat » Tue Jun 01, 2021 11:47 am

Thank you for your reply. The situation is sad.

In theory, what would it take to solve this problem? What the math involved?

Translated with http://www.DeepL.com/Translator (free version)
Foreigners' reactions to Russian "Bird's Milk" candies
— Are your birds being milked?
— In Russia everyone is milked. Here even the zucchini is used to make caviar.
User avatar
tomcat
 
Posts: 336
Joined: Sun Sep 27, 2015 7:53 pm
Location: Moscow (Orcish Stan), The Aggressive Evil Empire

Re: To transfer poses from MH to the Blender pose library

Postby loki1950 » Tue Jun 01, 2021 1:04 pm

All 3D modelling is based on matrix mathematics which use very heavy computational algorithms. One wrong step in the algorithm trashes the final result finding that wrong step is not a trivial process.

Enjoy the Choice :)
my box::HP Envy i5-6400 @2Q70GHzx4 8 Gb ram/1 Tb(Win10 64)/3 Tb Mint 19.2/GTX745 4Gb acer S243HL K222HQL
Q8200/Asus P5QDLX/8 Gb ram/WD 2Tb 2-500 G HD/GF GT640 2Gb Mint 17.3 64 bit Win 10 32 bit acer and Lenovo Ideapad 320-15ABR Win 10/Mint 19
User avatar
loki1950
 
Posts: 1219
Joined: Thu Dec 18, 2014 6:27 pm
Location: Ottawa,Ontario

Re: To transfer poses from MH to the Blender pose library

Postby punkduck » Tue Jun 01, 2021 9:24 pm

A few sentences:

Automatic weights generate also weights for bones which are not used in the standard-weight file. The root bone is one of the bones and the facial bones are mostly without weights.

Since I am working on a new skeleton atm ... I analyzed the old one and added the document to the chapter basemesh and skeleton:
http://www.makehumancommunity.org/w/images/8/8d/Facebones.png

The black bones are just helpers to pose the yellow ones ...

So this will be always different.

Second problem: one cannot always copy poses from one character to the other. So even the library has to be adjusted. Just an example: unfortunately the root bone is tilted, when the vertex at the back of the character which defines the root bone is moved by e.g. shaping the buttocks :(

Of course this angle can be calculated and be corrected ... but one character should be the default then.

For this problem one idea is to place the root bone inside the ground cube ... it does not have to be connected and the ground cube is always at (0.0.0).
User avatar
punkduck
 
Posts: 1216
Joined: Mon Oct 17, 2016 7:24 pm
Location: Nuremberg, Germany

Re: To transfer poses from MH to the Blender pose library

Postby tomcat » Wed Jun 02, 2021 7:54 am

punkduck wrote:Second problem: one cannot always copy poses from one character to the other. So even the library has to be adjusted.

I take it that this is the problem that prevents the full use of MH in games?

Example: you can't assign a uniform animation to multiple different characters (in games like The Sims)?

Since I am working on a new skeleton atm ...


Rig MH is probably great for individual work, but for mass animation production it is better to have as few bones as possible.

Cascadeur new animation program (Russian development, free version)

Mode "Tail", rig CMU/MH:

Image

Translated with http://www.DeepL.com/Translator (free version)
Foreigners' reactions to Russian "Bird's Milk" candies
— Are your birds being milked?
— In Russia everyone is milked. Here even the zucchini is used to make caviar.
User avatar
tomcat
 
Posts: 336
Joined: Sun Sep 27, 2015 7:53 pm
Location: Moscow (Orcish Stan), The Aggressive Evil Empire

Re: To transfer poses from MH to the Blender pose library

Postby punkduck » Wed Jun 02, 2021 7:56 pm

Hi

I take it that this is the problem that prevents the full use of MH in games?

Example: you can't assign a uniform animation to multiple different characters (in games like The Sims)?


Yes and no. You have to keep the characters a little uniform then. The problem with the root bone is ugly and it could be fixed. Thomas Larrson's diffeomorphic retargeter can recalculate one bone through one animation. Game engines then calculate the height of the character so it always touches the ground.

I cloned it to my repository, updated the code (Thomas is on bitbucket, but has an old version in github) ... updated it to the latest version (some time ago) and added default-poses for our skeleton. I can load bvh files now and use this as a base for our skeleton. Maybe it helps.

https://github.com/black-punkduck/retarget-bvh

I only examined OpenSim and wrote an article here:
http://www.makehumancommunity.org/forum/viewtopic.php?t=16927

OpenSim tends to use predefined sizes in my opinion. I accidentally upload the female proxy and it created o-shaped legs, too many vertices in crotch area.
If you check Avastar, they seem to use certain character defaults. At least when you compare the blonde character (with the wings), they look different in proportions between blender and OpenSIM.


Using the same animation: you should also think about different arm lengths etc. They will simply intersect with body parts in certain positions like crossing the arms in front of the body or in the demo appended she would scratch her face with longer arms, Sitting on the floor looked like an accident with the blonde character in OpenSim :lol:

Rig MH is probably great for individual work, but for mass animation production it is better to have as few bones as possible.


Not only bones, vertices also. The less geometry has to be created, the faster the game is. Yes bones cannot be replaced by additional level of details (LODs), this only works for vertices.
As always it is a problem to keep it detailed and realistic but also use only a low amount of geometry and memory. The reason for the use of normal maps and 100s of other tricks ... ;)

Depending to your animation: Some of the bones are simply designed different. The upperleg02, lowerleg02 and upperarm02, lowerarm02 bones are mostly used for twisting only (rotation in local y direction).

Just an example what I am really not happy about: Shoulder rotation in reality is a rotation + movement ... and it is hard to animate that correctly ... a typical fashion pose (arms behind head) often looks ugly.

https://www.youtube.com/watch?v=LF7oST34r4s

The Sims/OpenSim are limited to the game-engine or Bento skeleton. Unity I have no experience.

Unreal engine works with our own skeleton as well. They accept it as an fbx with all we have ... and for a game? ... well it depends what you want to create: for a multiplayer environment it is certainly the best to reduce the skeleton.

For a world with 2 or 3 characters it looks different. Both worlds exist. More common is: people want to use ready made animations from e.g. Mixamo and this demands a special skeleton ....

At the moment I only tested one of my characters in UE. A default-no toes skeleton with breast bones "uncoupled" to allow some jiggle.
I did all animations myself, partly derived from natural walks. Just for information: for UE the character walks in place normally (depending on the player-type) ... it is pushed forward a specific distance with the game controller. My movement was two steps and I wrote a blender plugin to place the root bone (it is NOT fixed) around a center point to allow hip movement.

It uses the face bones to speak, phonemes I figured out with papagayo ... (no sound added because I "stole" her voice, sorry), this was created with an animation overload starting from the neck bone, I can zoom to her face, so detailed bones are needed.

But I don't know why she needs ten tongue bones ... (well in reality there might be a reason so :mrgreen: ) ... so yes my skeleton will have less bones in some areas but especially the bones for volume (buttocks when sitting, belly when bending forward) will be added but then also be animated with constraints ... deform versus pose bones ...

By the way: if you prefer shape-keys for the face: a shape-key is a duplication of the character in memory, that is not necessarily better when you have a lot of characters. Nevertheless they work in UE and they can also be bound to a bone.
Attachments
ue_demo.mp4
(4.01 MiB) Downloaded 395 times
User avatar
punkduck
 
Posts: 1216
Joined: Mon Oct 17, 2016 7:24 pm
Location: Nuremberg, Germany

Re: To transfer poses from MH to the Blender pose library

Postby tomcat » Thu Jun 03, 2021 9:45 am

Thank you for a fine detailed and comprehensive answer.

I'm sorry, I probably should have mentioned exactly what this is for.

I'm not ready to tell you about the project in full yet… but the main features are:

It is assumed, approximately speaking, an alternative to The Sims. Like a mix of The Sims 3 (open world) and SimsSity. With thousands of characters in the future.

The game engine used is Godot. I'm still only learning it, full-fledged work will start with the release of version 4.

This is not a one game, but rather a project that will be released more and more complicated programs. Not so much a game. A Socio-EConomic Simulator (SEC[K]S) :D . But it can be played as a game. :)

You can make multiplayer based on it, but it's mostly single-player.

punkduck wrote:You have to keep the characters a little uniform then.

That's not valid. The point is to implement growing up, smooth growth of the characters. MH is an exclusive character generator that allows you to do this.
"With all the richness of choice there is no other alternative" (c) Russian TV advertising.

https://github.com/black-punkduck/retarget-bvh

Oh, that's much better than the old version. Thank you! Dziekuje bardzo!

Using the same animation: you should also think about different arm lengths etc. They will simply intersect with body parts in certain positions like crossing the arms in front of the body or in the demo appended she would scratch her face with longer arms, Sitting on the floor looked like an accident with the blonde character in OpenSim

Usually for this use "adaptive animation", when two options are made on the extreme points (for a long arm and a short arm) and the game is selected intermediate option on the actual length.

What if you use Inverse Kinematics?

Just for information: for UE the character walks in place normally (depending on the player-type) ... it is pushed forward a specific distance with the game controller.
Yep, that's a typical solution for gamedev.

I want to try using the rig from Cascadeur, but I don't like the understated number of fingers and lack of facial bones.

Image

ue_demo.mp4

Great! Not perfect, but I'd like to achieve similar in Godot to begin with.

Yes bones cannot be replaced by additional level of details (LODs)

But you can turn off animation of body parts at a great distance: face, hands, feet. Not many people can make out a facial expression from a verst (a mile, a kilometer) away.

Translated with http://www.DeepL.com/Translator (free version)
Foreigners' reactions to Russian "Bird's Milk" candies
— Are your birds being milked?
— In Russia everyone is milked. Here even the zucchini is used to make caviar.
User avatar
tomcat
 
Posts: 336
Joined: Sun Sep 27, 2015 7:53 pm
Location: Moscow (Orcish Stan), The Aggressive Evil Empire

Re: To transfer poses from MH to the Blender pose library

Postby punkduck » Fri Jun 04, 2021 3:12 pm

woodcat wrote:The game engine used is Godot. I'm still only learning it, full-fledged work will start with the release of version 4.

Ah ... yes, I really like to read some of their arguments ...
  • Small download (around 30 MB), and you are ready to go. (UE source code for Linux is huge and compilation is a 3-4 hour job even with parallel computing). Windows comes with a installable packet, Linux is source code.
  • Scene instancing makes teamwork a breeze. Every team member can focus on their own scene, be it a character, level, etc., and edit without stepping on each other's toes. (UE import and export to work with others is tricky and badly documented, although parts can be imported. But really to share a project seems only be possible with workspace somewhere in the net, then it depends on your internet connection ... I watched a video where a few guys are talking for an eternity just the exchange a new texture for an asset :(, it looks like a half-baken concept )
  • Full C++ support without needing to recompile the engine using GDNative ( a real weakness. For an empty game the complete engine was re-compiled. What are makefiles for? )

woodcat wrote:Oh, that's much better than the old version. Thank you! Dziekuje bardzo!

Thomas is unfortunately working more or less for Daz only now. His blender importer is better than the official one. But the code of the retargeter itself is derived from old makewalk. Without his work there wouldn't be some tools today, like mhx2 exporter etc.

woodcat wrote:Usually for this use "adaptive animation", when two options are made on the extreme points (for a long arm and a short arm) and the game is selected intermediate option on the actual length.

What if you use Inverse Kinematics?

Sitting on the ground was a buildin-animation. For OpenSim I only added the skeleton. We tried small animations to prove mouth and fingers are working. It was not that much.

Inverse kinematics is supported in UE. But for an animation built in Blender it is not necessarily needed, because "visual locrotscale" will save the keyframes for the "normal" bones as well. Otherwise the skeleton with the complete ik has to be exported. Kinematic bones as helpers in a game engine nevertheless give other opportunities (like correct foot position on a steep hill) but are also "more expensive" with CPU.

woodcat wrote:I want to try using the rig from Cascadeur, but I don't like the understated number of fingers and lack of facial bones.


The base skeleton is not bad, 3 fingerbones are sufficient for a lot of gestures, except for piano players of course ;). I like the calculation by curves and inverse kinematics they use for some movements, it looks rather natural. But yes, especially when you do games with social aspect, a face should show expressions.

woodcat wrote:Great! Not perfect, but I'd like to achieve similar in Godot to begin with.

Yes, this is a demo, there are problems, first the idle movements are interrupted by a timer (which looks like she is hesitating) and the slow 90 degree turn I tried myself ... since the root bone has to be adjusted after the turn I am also not that happy with that. Normally she flips in a game engine like a whirl-wind but that looks even more comical. When she should sit down it is even worse. Until now she is pushed to the location, where sitting is "allowed", then sits down by animation and is bounced up again --- depending on how bad I designed the collision box :(
It will take a while. But as I wrote: it is a test. I also need to understand which way is the best, for a lot of problems there are more ways to do it.


woodcat wrote:But you can turn off animation of body parts at a great distance: face, hands, feet. Not many people can make out a facial expression from a verst (a mile, a kilometer) away.

oh yes, there are other ways of optimization. There is also a limit when it is no longer displayed at all.

At least your ideas look rather interesting. But for the first step: keep it always small to see if it works. The idea of 100s of characters in one scene might need a little work :mrgreen: or ... to say it with a smile: start with a village in North Siberia and try Moscow a week later earliest :D

Greetings to Russia from a small country ...
User avatar
punkduck
 
Posts: 1216
Joined: Mon Oct 17, 2016 7:24 pm
Location: Nuremberg, Germany

Re: To transfer poses from MH to the Blender pose library

Postby tomcat » Sat Jun 05, 2021 6:23 am

punkduck wrote:Ah ... yes, I really like to read some of their arguments ...

A great advantage of Godot is its simplicity and clarity. It's very easy to learn and its GDScript language is perfect to start teaching programming (I'm trying to get teachers at school interested in it).

Inverse kinematics is supported in UE.

IK is also supported in Godot.

I also need to understand which way is the best, for a lot of problems there are more ways to do it.

There are different ways to adapt the animation. Motion Matching, for example.

But for the first step: keep it always small to see if it works. The idea of 100s of characters in one scene might need a little work

Yes, I will start with one person, then a family, then a small village. Novgorod Republic. When Russia was still a democracy. Yes, it was… 600 years ago.

By the way, for this phrase, if it is published on a Russian forum, I could go to jail. I'm quite serious. The other day in Russia adopted the "Law on educational activities" and no one has the right to publish historical facts without special state certification. And given that I'm making the game based on historical events…

Greetings to Russia from a small country ...

Oh, a very small state. :lol:

Large size, in and of itself, does not make a state great.

When I have something to show, I will post my developments here on the forum. I suppose it will take a few months. The game is supposed to be free.

Translated with http://www.DeepL.com/Translator (free version)
Foreigners' reactions to Russian "Bird's Milk" candies
— Are your birds being milked?
— In Russia everyone is milked. Here even the zucchini is used to make caviar.
User avatar
tomcat
 
Posts: 336
Joined: Sun Sep 27, 2015 7:53 pm
Location: Moscow (Orcish Stan), The Aggressive Evil Empire

Next

Return to Bugs, problems and feature requests

Who is online

Users browsing this forum: No registered users and 1 guest