Transformation Pipeline

MakeHuman python API, python plugins, etc

Moderator: joepal

Transformation Pipeline

Postby Kalkas » Fri Oct 02, 2015 9:36 am

I am currently thinking about using makehuman in my project but even after browsing the code for some time now i have still issues understanding some core concepts. More specific it concerns the pose system. I have the following questions:

1) How is the length of the bones calculated from the deformation of the mesh? In the *.mkskel files in the joints section there are IDs listed. I guess they are estimated from these referenced vertices.

2) In the new pose system, how is the position of each vertex position in the result calculated? Right now i am guessing the pipeline is:
starting model => morphed model with targets => estimate joints and skeleton => use predefined weights for skinning of mesh to bones => apply pose.

I dont know if there is a better place to ask these questions, but the scripting topic was the closest i could find.
Kalkas
 
Posts: 5
Joined: Fri Oct 02, 2015 9:10 am

Re: Transformation Pipeline

Postby duststorm » Mon Oct 05, 2015 2:46 pm

Kalkas wrote:1) How is the length of the bones calculated from the deformation of the mesh? In the *.mkskel files in the joints section there are IDs listed. I guess they are estimated from these referenced vertices.

The joints are part of the basemesh, and are modeled along with it. A bone is always between two joints, thus the length of the bone is simply the euclidian distance between those two joints.

Kalkas wrote:2) In the new pose system, how is the position of each vertex position in the result calculated? Right now i am guessing the pipeline is:
starting model => morphed model with targets => estimate joints and skeleton => use predefined weights for skinning of mesh to bones => apply pose.

Yes, that's it generally. The basemesh is modeled in rest/bind pose, the skeleton is updated, and then the pose is applied. While dragging sliders I apply a hack to make it work more real-time though: I transform the morph offsets of the vertices with the (stale) pose matrices. This works well for a quick preview of moderate changes to the mesh, however it's what causes the extreme deformations when the model is changed very radically. Releasing the slider solves these deformations because it recalculates the skeleton and its bind matrices.


Interesting, do you want to share some more about this project? :)
MakeHuman™ developer
User avatar
duststorm
 
Posts: 2569
Joined: Fri Jan 27, 2012 11:57 am
Location: Belgium

Re: Transformation Pipeline

Postby Kalkas » Tue Oct 06, 2015 2:42 pm

Thank you for your helpful reply.

In my research i try to fit dynamic 3D-models of humans to images. Before training a dynamic model from data only i wanted to try out an existing dynamic model so i can focus on the fitting problem first.

As far as i can tell i am actually talking to the source, so I have some additional questions. How did you generate the poses in the system? Did you use an automatic way to "retarget" them from an existing pose database? Or were they carefully posed by hand?
Kalkas
 
Posts: 5
Joined: Fri Oct 02, 2015 9:10 am

Re: Transformation Pipeline

Postby duststorm » Tue Oct 06, 2015 10:57 pm

They were posed by hand.
Although I am still planning to create a motion database for MH characters using retargeted and tweaked mocap data.
MakeHuman™ developer
User avatar
duststorm
 
Posts: 2569
Joined: Fri Jan 27, 2012 11:57 am
Location: Belgium

Re: Transformation Pipeline

Postby Patator » Thu Nov 05, 2015 4:44 pm

duststorm wrote:They were posed by hand.
Although I am still planning to create a motion database for MH characters using retargeted and tweaked mocap data.


Thank you so much. I use MH in my project too trying to generate a database of images to train machine learning model. This is just exactly what I need. My goal is to do everything in MH, from modeling to animation to rendering. As for now I use Blender to do the animation and rendering but I'm not really satisfied (There are some images of that in my PhD thesis :) ). I'm really interested in contributing on the animation/posing aspect of MH. I'll try to make some time to do it. A tool to convert the CMU database into MH compatible BVH files would be wonderful. Do you think it's hard to do ? How should I start doing that ? by looking at the makewalk plugin ? Should I revitalize the animation plugin first ?

More generally, I see MH as a very useful tool for researchers and it's a shame it's a bit difficult to generate good databases with it even though it's very promising.
Patator
 
Posts: 10
Joined: Mon Aug 20, 2012 4:43 pm

Re: Transformation Pipeline

Postby duststorm » Tue Nov 10, 2015 3:16 pm

Patator wrote:I use MH in my project too trying to generate a database of images to train machine learning model.

I did a similar thing a while back, you can find the paper here http://jonas.ejibe.net/papers/dhm2013_submission_54.pdf
This was done with version A7 of MakeHuman and the posing code was never really finalized for production.
I did use a lot of it to design the new posing system in MakeHuman, though, but optimizing it for realtime animation playback or retargetting of motions were never done.
The old code is still available on kbuys and my github repositories I think.

Patator wrote:I'm really interested in contributing on the animation/posing aspect of MH. I'll try to make some time to do it. A tool to convert the CMU database into MH compatible BVH files would be wonderful. Do you think it's hard to do ? How should I start doing that ? by looking at the makewalk plugin ? Should I revitalize the animation plugin first ?

Cool :)
The tool we have that comes closest to what you need, namely retargetting mocap animations to the MakeHuman rig, is the MakeWalk blender plugin.
It probably needs some updating, as I especially think that it does not work well with the most recent "default" rig that MakeHuman uses. This is something we have not tackled yet completely, but would greatly benefit the software.

We do have some other tools for importing bvh animations on different body types of a MakeHuman rig, which are in the makehuman-utils repository: https://bitbucket.org/MakeHuman/makehuman-utils
The retarget script there could be of interest.

Patator wrote:More generally, I see MH as a very useful tool for researchers and it's a shame it's a bit difficult to generate good databases with it even though it's very promising.

MakeHuman is by nature not that simple, which is probably the reason why making such tools that are easy to use is not an easy task.
But perhaps with the help of other researchers that use MH for a particular purpose a better toolset could be constructed.
MakeHuman™ developer
User avatar
duststorm
 
Posts: 2569
Joined: Fri Jan 27, 2012 11:57 am
Location: Belgium

Re: Transformation Pipeline

Postby Patator » Thu Nov 12, 2015 2:17 pm

Thanks a lot for your answer. Also I skimmed through your paper. I did the exact same thing during my PhD. Except I generated all the training images in Blender. But I also used Random Forests and everything ! I worked alone on the project and struggled a little bit to make it all work as I wanted to. Your solution is so much better. I need to reuse the pipeline I made for a new project for another application but I'm not quite happy with it. My idea was to do everything in MH in order to extend the diversity of training images by adding clothes and changing some parameters on the fly for each image. It seems you made everything before me...

P.S : Also I wonder. How did you manage to fit 180k images in your random forest ?! Was it on a normal computer ?
Patator
 
Posts: 10
Joined: Mon Aug 20, 2012 4:43 pm

Re: Transformation Pipeline

Postby duststorm » Fri Nov 13, 2015 9:45 am

Patator wrote:P.S : Also I wonder. How did you manage to fit 180k images in your random forest ?! Was it on a normal computer ?

The machine learning algorithms were run on the amazon cloud, in the github repository you should find the code that ran on the clusters using hadoop fs.
MakeHuman™ developer
User avatar
duststorm
 
Posts: 2569
Joined: Fri Jan 27, 2012 11:57 am
Location: Belgium

Re: Transformation Pipeline

Postby Kalkas » Fri Nov 13, 2015 9:48 am

Very impressive indeed.

Aside from the retargeting code is there also a paper the retargeting is based on? Being able to retarget the CMU Mocap data on the current rig would help me a lot.
Kalkas
 
Posts: 5
Joined: Fri Oct 02, 2015 9:10 am

Re: Transformation Pipeline

Postby Patator » Fri Nov 13, 2015 3:37 pm

I have no problem doing that in blender. I use MHX2 However.
Patator
 
Posts: 10
Joined: Mon Aug 20, 2012 4:43 pm

Next

Return to Python scripts

Who is online

Users browsing this forum: No registered users and 1 guest