Where are hidden the weight matrices in a mhx2 file?

Discussions about MHX2 and other plugins that are developed outside the scope of MakeHuman. Note that bug reports should go to the respective code projects and might go unseen here.

Moderator: joepal

Where are hidden the weight matrices in a mhx2 file?

Postby jeanmilost » Sat May 29, 2021 12:54 am

Hello,

For a learning purpose, I'm writing a small reader for mhx2 files in c++ and OpenGL (project available here: https://github.com/Jeanmilost/MakeHuman/)

My final purpose is to animate the model by rigging the skeleton with an animation read from a Filmbox (.fbx) file. Currently I may draw the model successfully, I'm able to build the skeleton (I can also draw it, e.g by pressing the [1] key in my above project), and I already extracted the weights data, which are ready to be used.

However I noticed that a data is missing in order to successfully achieve my project. Indeed, I cannot find any weight matrix in the whole .mhx2 file.

For example, any Filmbox (.fbx) file contains a such matrix in its deformers (see the TransformLink matrix) ...:
Code: Select all
   Deformer: 74303680, "SubDeformer::calf_l_laure_source_mh", "Cluster" {
      Version: 100
      UserData: "", ""
      Indexes: *360 {
         a: 4146,9055,9057,9058,[...]
      }
      Weights: *360 {
         a: 0.00510000018402934,0.0082999998703599,0.0135000003501773,0.010499999858439,[...]
      }
      Transform: *16 {
         a: 0.999574184385344,-0.000827508388654136,-0.0291679352521862,0,0.000720479628384557,-0.998593092084758,0.0530211329493891,0,-0.0291707750657101,-0.0530195703534821,-0.998167276351283,0,-10.9260759398066,46.6885032726829,-3.22593307514371,1
      }
      TransformLink: *16 {
         a: 0.999574184417725,0.000720479583833367,-0.0291707757860422,0,-0.000827508436385407,-0.998593151569135,-0.0530195720540331,0,-0.0291679363642472,0.053021134410156,-0.998167336074704,0,10.8659648895264,46.8017349243164,-1.06333863735199,1
      }
   }


...as well as any DirectX (.x) file (see the last line, which is a 4x4 matrix):
Code: Select all
    SkinWeights {
     "Bip01_Spine3";
     360;
     1050,
     1051,
     1052,
     1054,
     1057,
     [...]
     1.000000,
     1.000000,
     1.000000,
     0.394761,
     0.394761,
     [...]
     0.000000,-0.000002,1.278853,0.000000,1.100796,0.223046,0.000001,0.000000,-0.291970,1.440953,0.000002,0.000000,28.876726,-65.051079,-0.142296,1.000000;;
    }


So where may be read a such matrix from a MakeHuman (.mhx2) file? Where it is hidden?

Regards
jeanmilost
 
Posts: 7
Joined: Thu Jul 03, 2014 2:07 pm

Re: Where are hidden the weight matrices in a mhx2 file?

Postby loki1950 » Sat May 29, 2021 2:58 am

That maybe because the weights data is in a file of it's own with the .mhw suffix. BTW the mhx2 file is a json formatted file which makes it easier to import into blender.

Enjoy the Choice :)
my box::HP Envy i5-6400 @2Q70GHzx4 8 Gb ram/1 Tb(Win10 64)/3 Tb Mint 19.2/GTX745 4Gb acer S243HL K222HQL
Q8200/Asus P5QDLX/8 Gb ram/WD 2Tb 2-500 G HD/GF GT640 2Gb Mint 17.3 64 bit Win 10 32 bit acer and Lenovo Ideapad 320-15ABR Win 10/Mint 19
User avatar
loki1950
 
Posts: 1219
Joined: Thu Dec 18, 2014 6:27 pm
Location: Ottawa,Ontario

Re: Where are hidden the weight matrices in a mhx2 file?

Postby jeanmilost » Sat May 29, 2021 3:53 am

When I export a model to mhx2, I only see a .mhx2 file and a textures dir containing the textures in .png format, but no .mhw file is created. Can you please explain a little more about this file? How should it be created? Also, about the weights, the .mhx2 file however contains the weights with their indices, as shown below:
Code: Select all
            "mesh" : {
                "vertices" : [[1.455,-7.69,-0.14134],[1.4655,-7.6794,-0.14334],[...]
                "faces" : [[0,6,7,1],[1,7,8,2],[...]
                "uv_coordinates" : [[0.5655,0.5909],[0.5372,0.5882],[...]
                "uv_faces" : [[0,1,2,3],[3,2,4,5],[...]
                "weights" : {
                    "calf_l" : [[0,0.034125],[1,0.049072],[2,0.064068],[...]
                    "calf_r" : [[2931,0.034125],[2932,0.049072],[2933,0.064068],[...]
                    "foot_l" : [[0,0.96587],[1,0.95093],[2,0.93593],[...]
                    "foot_r" : [[2931,0.96587],[2932,0.95093],[2933,0.93593],[...]
                    "ball_l" : [[1372,0.082863],[1373,0.0639],[1374,0.12654],[...]
                    "ball_r" : [[4303,0.065825],[4304,0.067966],[4305,0.0638],[...]
                }
           [...]


As you can see all the weights data are here, except the weight transform matrix. So this matrix is provided in a separate file, right?
jeanmilost
 
Posts: 7
Joined: Thu Jul 03, 2014 2:07 pm

Re: Where are hidden the weight matrices in a mhx2 file?

Postby punkduck » Sat May 29, 2021 8:23 am

Hi

Yes these lines are the bone-names followed by an array of vertex-number + weights.

The .mhw file is already used for this ;) Because the values of the mhx2 file are exactly these values appended. At least for the body. For clothes the values are calculated by makehuman but they appear the same way.

This is enough information for blender ... these are the weight-values for the vertex-groups. Unreal Engine also works with that, and they use fbx. It looks like the algorithm how the weights will influence skin driven by the bones is more or less the question, when you speak about a transformation matrix .. I guess it is a matrix (or some of them) but it could be any algorithm as well,

Both systems, Blender and Unreal Engine, have these methods build into their software and both know different methods to handle it. Like "keep volume" in blender, when you e.g. twist the lower arm and the volume does not disappear and a dual-quaternion algorithm is used.

A little demo of both methods may help to understand what I am talking about:

https://www.youtube.com/watch?v=2s3uToONDHg

Unfortunately I don't have the "deforming" algorithm for that and my mathematical knowledge is not so good that I have these algorithms available. The standard algorithm used by blender and UE is linear blend skinning, I guess. Found some code ... but of course did not test it:

http://graphics.cs.cmu.edu/courses/15-466-f17/notes/skinning.html

For dual quaternion check in the net. When you implement one of them: be aware that both methods have their weaknesses ... for a human character you will lose volume in knee, arm and buttocks region with the linear one. Dual quaternion creates a hulk when the character poses hands above head.
User avatar
punkduck
 
Posts: 1216
Joined: Mon Oct 17, 2016 7:24 pm
Location: Nuremberg, Germany

Re: Where are hidden the weight matrices in a mhx2 file?

Postby jeanmilost » Thu Jun 03, 2021 1:31 pm

Thank you very much for these info, they helped me to resolve my issue.

The missing weight matrix I searched for was just the inverse of the bone global matrix, as explained in this document: https://veeenu.github.io/blog/implement ... animation/

Furthermore there was a bug in my source code, which caused the weights to be computed incorrectly.

If someone is interested by the result: https://github.com/Jeanmilost/MakeHuman

Once again thanks to all who took the time to answer my questions.
jeanmilost
 
Posts: 7
Joined: Thu Jul 03, 2014 2:07 pm

Re: Where are hidden the weight matrices in a mhx2 file?

Postby tomcat » Thu Jun 03, 2021 2:45 pm

jeanmilost wrote:If someone is interested by the result: https://github.com/Jeanmilost/MakeHuman

Sorry for the stupid question. Does this program need to be compiled?

If Visual Studio 2019 is not installed, you can't run it?
Foreigners' reactions to Russian "Bird's Milk" candies
— Are your birds being milked?
— In Russia everyone is milked. Here even the zucchini is used to make caviar.
User avatar
tomcat
 
Posts: 336
Joined: Sun Sep 27, 2015 7:53 pm
Location: Moscow (Orcish Stan), The Aggressive Evil Empire

Re: Where are hidden the weight matrices in a mhx2 file?

Postby loki1950 » Thu Jun 03, 2021 10:08 pm

Yes this just the source code for a C++ program it needs to be compiled into an executable binary file either using Visual Studio on Windows or gcc under any Linux distro. BTW .mhx2 files are json formatted files which is an other programming language.

Enjoy the Choice :)
my box::HP Envy i5-6400 @2Q70GHzx4 8 Gb ram/1 Tb(Win10 64)/3 Tb Mint 19.2/GTX745 4Gb acer S243HL K222HQL
Q8200/Asus P5QDLX/8 Gb ram/WD 2Tb 2-500 G HD/GF GT640 2Gb Mint 17.3 64 bit Win 10 32 bit acer and Lenovo Ideapad 320-15ABR Win 10/Mint 19
User avatar
loki1950
 
Posts: 1219
Joined: Thu Dec 18, 2014 6:27 pm
Location: Ottawa,Ontario

Re: Where are hidden the weight matrices in a mhx2 file?

Postby jeanmilost » Fri Jun 04, 2021 1:18 am

Does this program need to be compiled?

Yes it does. I didn't compiled it because I think that the source code is the most interesting part of this project, and on the other hand, a such executable would not be very useful, as it actually just load and show a mhx2 model from a hardcoded file name.

However if you're interested, I may compile and release it on my Github. I may also add a small menu to be able to do several actions like open any .mhx2 file. But once again I think the most interesting part is the code itself, which you may get and modify as you want.

Also if you don't want or need to install a heavy compiler like VS2019, you should be able to compile this code with any other compiler, e.g. it should be compiled without problems with Code::Blocks. A compiler like Embarcadero RAD Studio may also be used, but the code handling the main Windows form should be adapted a little in this case. I also compiled a similar code on MacOS/xCode recently, with success. Of course I needed to adapt the rendering part, and get rid of the Windows SDK, but it was not so painful at the end. I wrote the code in a high scaling, cross-platform way, the Windows API specific code remained on a high level, in classes which should be easily overridable.

Please let me know if you want a compiled exe, or any help I may provide.
jeanmilost
 
Posts: 7
Joined: Thu Jul 03, 2014 2:07 pm

Re: Where are hidden the weight matrices in a mhx2 file?

Postby tomcat » Sat Jun 05, 2021 5:08 am

jeanmilost wrote:I didn't compiled it because I think that the source code is the most interesting part of this project

You are absolutely right! The source code is really the most interesting part… for those who know programming. Unfortunately not everybody knows it yet.

I, for example, am just beginning to learn programming. I am currently learning GDScript — the Godot Engine language. I'm still a long way away from C++.

But I am interested in everything related to MH. My interest is related to my project.

However if you're interested, I may compile and release it on my Github. I may also add a small menu to be able to do several actions like open any .mhx2 file. But once again I think the most interesting part is the code itself, which you may get and modify as you want.

If you do, that would be excellent. I'm sorry, but I'm not yet qualified to edit C++ code. :cry:


Translated with http://www.DeepL.com/Translator (free version)
Foreigners' reactions to Russian "Bird's Milk" candies
— Are your birds being milked?
— In Russia everyone is milked. Here even the zucchini is used to make caviar.
User avatar
tomcat
 
Posts: 336
Joined: Sun Sep 27, 2015 7:53 pm
Location: Moscow (Orcish Stan), The Aggressive Evil Empire

Re: Where are hidden the weight matrices in a mhx2 file?

Postby jeanmilost » Fri Jun 11, 2021 3:04 am

If you do, that would be excellent. I'm sorry, but I'm not yet qualified to edit C++ code.

So I released a compiled version of the source code, you may find it here: https://github.com/Jeanmilost/MakeHuman ... s/tag/v1.0

But be aware that Godot uses instead the Filmbox (.fbx) format for its assets. If you're interested, I also wrote a small c++ reader for this format, available here: https://github.com/Jeanmilost/Demos/tre ... OpenGL/FBX

Hope this will help :)
jeanmilost
 
Posts: 7
Joined: Thu Jul 03, 2014 2:07 pm

Next

Return to MHX2 and other plugins developed outside MakeHuman

Who is online

Users browsing this forum: No registered users and 1 guest