Updating mesh after bone rotation

MakeHuman python API, python plugins, etc

Moderator: joepal

Updating mesh after bone rotation

Postby AndyM » Fri Jun 05, 2015 12:38 pm

Hi,

I am writing a plugin which modifies the bone rotations of a skeleton. This code seems to work:

Code: Select all
gui3d.app.selectedHuman.skeleton.getBone("root").setRotation([10,10,10])

The problem is to get the mesh updated from the new skeleton configuration. applyAllTargets() is not working. Any suggestions?
Attachments
MH.png
AndyM
 
Posts: 9
Joined: Fri Jun 05, 2015 12:12 pm

Re: Updating mesh after bone rotation

Postby duststorm » Fri Jun 05, 2015 2:04 pm

User skeletons (which is what the GUI draws) are updated lazily, because they are only used for visualization and export.
Access the skeleton through
Code: Select all
human.getSkeleton()

instead of simply accessing the skeleton member, this will update the user skeleton (fit the rest pose to fit that of the base skeleton).

Furthermore, after changing bone.matPose (which is what setRotation does), you will have to call bone.update() (or skeleton.update() after changing a bunch of bone poses) to get the skeleton to actually update. Anyway, from your screenshot it appears like you might have already done that.

Note that skeleton management in MakeHuman is not that straightforward anymore since we have implemented the two-skeleton approach. You basically have human.getBaseSkeleton() and human.getSkeleton(), the base skeleton is the only one that's used internally, it's used for all posing. getBaseSkeleton() will always give you the default skeleton, getSkeleton() returns whatever you have chosen in the skeleton library (can be None if nothing was selected). This allows us to pose the mesh, even if no skeleton was chosen, and guarantees that the basemesh is optimally posed in its rest pose, regardless of the chosen skeleton.

It is only the base skeleton that has a pose in MH (its rest pose is always the A pose), whatever other skeleton you have chosen from the skeleton library is simply fitted inside the joint locations of the posed mesh (thus, contrary to human.getBaseSkeleton(), human.getSkeleton() will always be in rest pose, unless you manually add a pose to it).

You can change the pose of user skeleton, though. After updating the skeleton, its pose should be drawn (note, GUI tabs like skeleton library or skeleton debug -- the second one is in utilities and is a developer tool that can be enabled through Settings > Plugins and a restart of MH, you might like it -- only update the skeleton drawing when they get an onShow() event, of course you can invoke that event yourself, or have a look at the drawing code that is linked to it)
The skeleton drawing in skeleton library renders the user skeleton, though (however everytime you update the user skeleton its rest pose is fitted to that of the base skeleton, so visually there will be no difference as long as you select that same default skeleton as user skeleton). But you can easily create a plugin that renders the base skeleton.

Note though that changing the pose of the user skeleton will not modify the pose of the human, as it is skinned by the base skeleton only.
Depending on what you want to do, you might want to change the pose of the base skeleton instead.
If you are looking to skin the human by posing the user skeleton, then I recommend you clone the human mesh, grab the vertex weights of the user skeleton, and create a new AnimatedMesh object that links the user skeleton (or a copy of it) to the clone of the human mesh. Cloning to a separate object that you have full control of is easier and you don't run the risk of interfering with other MakeHuman mechanisms.
To see how you can clone the mesh and grab the user skeleton vertex weights for it, have a look at the Ogre3D exporter, it is the most straightforward implemented of the exporters and a good example of how the API works.

You can read more details about this implementation here http://bugtracker.makehuman.org/issues/817
Have a look at the commits and the commit messages linked to this issue for details.
MakeHuman™ developer
User avatar
duststorm
 
Posts: 2569
Joined: Fri Jan 27, 2012 11:57 am
Location: Belgium

Re: Updating mesh after bone rotation

Postby AndyM » Sat Jun 06, 2015 12:05 pm

To be more exactly. The skeleton is not the goal. I want the skinned human to follow the modified skeleton.
So, I changed the code to work with the update functions you mentioned:
Code: Select all
human = gui3d.app.selectedHuman
skeleton = human.getBaseSkeleton()
bone = skeleton.getBone("root")
bone.setRotation([10,10,10])
bone.update()
skeleton.update()

bodyWeights = human.getVertexWeights(skeleton)

So I grab the weights, but did not see a way to create an AnimationMesh from that point. Which seems to be the object which I need to bring the skinned human to the new skeleton pose? The Orgre3D exporter did not create one, as far as I can see...

There is also the Pose class in animation.py which seems to be the right class for my problem. But how to create the needed poseData from the modified skeleton or the weights?
AndyM
 
Posts: 9
Joined: Fri Jun 05, 2015 12:12 pm

Re: Updating mesh after bone rotation

Postby duststorm » Sat Jun 06, 2015 1:53 pm

AndyM wrote:So I grab the weights, but did not see a way to create an AnimationMesh from that point.

It's in shared/animation.py
The human objects inherits from it and manages it internally. Basically you need to add your mesh and weights to it as bound mesh, set a pose and skin the mesh.
But if you want to manipulate the pose on the skeleton and then skin directly, without using a separate animation track, you could go via the skeleton directly and not even need the AnimatedMesh. Have a look at skeleton.skinMesh()

Note: skinning is currently not that fast in MH, certainly skeleton.skinMesh. It's not fast enough for animation playback (I get about 10 FPS max out of it).

AndyM wrote:The Orgre3D exporter did not create one

No, because it does not need to skin the mesh.
MakeHuman™ developer
User avatar
duststorm
 
Posts: 2569
Joined: Fri Jan 27, 2012 11:57 am
Location: Belgium

Re: Updating mesh after bone rotation

Postby AndyM » Sat Jun 06, 2015 5:23 pm

I brought finally this one to work (which do exactly what I want):
Code: Select all
anim = animation.Pose("Test", skeleton.getPose())
human.addAnimation(anim)
human.setActiveAnimation(anim.name)
human.setToFrame(0)
For each pose change I have to create a new animation, right? Or is there a better way to do that all?
AndyM
 
Posts: 9
Joined: Fri Jun 05, 2015 12:12 pm

Re: Updating mesh after bone rotation

Postby duststorm » Sat Jun 06, 2015 8:51 pm

That's a sane solution, if performance is not very important.
MakeHuman™ developer
User avatar
duststorm
 
Posts: 2569
Joined: Fri Jan 27, 2012 11:57 am
Location: Belgium

Re: Updating mesh after bone rotation

Postby AndyM » Sun Jun 07, 2015 8:27 am

At a certain point, for sure, it will. But that could be a long way... Thanks so far!
AndyM
 
Posts: 9
Joined: Fri Jun 05, 2015 12:12 pm

Re: Updating mesh after bone rotation

Postby AndyM » Tue Jul 21, 2015 3:55 pm

After working with this code for a while I figure out a strange behavior. If I call this code twice I get different pose each time. Any idea?

Code: Select all
anim = animation.Pose("Test", human.getBaseSkeleton().getPose())
human.addAnimation(anim)
human.setActiveAnimation(anim.name)
human.setToFrame(0)
AndyM
 
Posts: 9
Joined: Fri Jun 05, 2015 12:12 pm

Re: Updating mesh after bone rotation

Postby AndyM » Fri Jul 24, 2015 3:09 pm

Further investigation...
The getBaseSkeletion().getPose() gives me a list of local rotation matrixes (bone.matPose), but the new created Pose object, which calls setPose, seems to assume that this is a list of global rotation matrices.
Is that right? What is the global orientation of the bones?
AndyM
 
Posts: 9
Joined: Fri Jun 05, 2015 12:12 pm

Re: Updating mesh after bone rotation

Postby AndyM » Fri Jul 24, 2015 4:36 pm

This part in skeleton.setPose() cause me trouble. It takes the local pose and transform it to another orientation. Which pose matrix is the right one for that function?

Code: Select all
# Calculate rotations
bone.matPose[:3,:3] = poseMats[bIdx,:3,:3]
invRest = la.inv(bone.matRestGlobal)
bone.matPose = np.dot(np.dot(invRest, bone.matPose), bone.matRestGlobal)
AndyM
 
Posts: 9
Joined: Fri Jun 05, 2015 12:12 pm

Next

Return to Python scripts

Who is online

Users browsing this forum: No registered users and 1 guest