Difference between revisions of "Technical notes on MakeHuman"

Revision as of 14:54, 27 January 2016

Vertex weights

With vertex weights we intend a weighted mapping of vertices of a mesh to a bone of a skeleton. The weight is a value between 0 and 1, the sum of these weights for each vertex should be normalized, which means they should sum to 1 (note: no euclidian distance, just simple addition of all weights). A weight map in MakeHuman is expressed as a bone name (becomes a bone index internally, using the index of the bone in a breadth-first list relative to skeleton structure), a vertex name and a floating point weight between 0 and 1. This weighted vertex to bone mapping is used for skinning (posing and animating, using vertex skinning, smooth skinning or linear blend skinning algorithm). For the skinning algorithm to work correctly it’s required that the weights are always normalized, otherwise you will see vertices move out of place disproportionally (though the weight loading code in MakeHuman ensures that this normalization always happens, so this is not a concern to the artist that defines the weights). The vertex weights can be exported to external tools as well using formats such as collada (DAE), FBX, Ogre3D meshes or MD5 quake meshes.

Vertex weights and skeletons

Vertex weights are defined for the default “master” skeleton or rig. They are defined in a json file with jsonw extension. They link each vertex of the basemesh including helpers to a named bone of this master rig, together with a certain weight. The weights loader automatically removes doubles and normalizes the weights (so that they sum to 1). It also reports what the maximum number of bone weights per vertex is, additionally it allows compiling the weights to a reduced set with n number of bone weights, and automatically renormalizes those weights so they, again, sum to 1. When loading weights, any vertex that receives no weighting is assigned to the root bone, as this is needed for skinning to prevent all those vertices jumping to the origin (0*pos == 0). Alternative skeletons can define their own weights, by referencing a weights file just like the master rig, or they can choose to derive their weights from the master rig, through so called reference bone mappings. Skeletons will derive their weights from the master rig if they do not specify a weights file explicitly. Custom vertex weights in alternative rigs override those defined on the master rig, except in the case where a proxy defines its own vertex weights. See section Vertex weights for proxies for more details.

Reference bones for vertex weights and poses

Reference bones are defined on alternative rigs, they are a list of names of bones contained in the master rig. Every bone of an alternative rig can have such a list of reference bones. An alternative rig can also omit these references if it uses the same name as bones in the master rig, in which the mapping is implicit. Having a list of bone references always overrides the implicit mapping. Reference bone mappings are required for two purposes. They are used for mapping poses defined on the master rig to other rigs. All poses are defined as BVH files that pose the master rig, so they reference only the bone names of the master rig. Poses are applied to all other rigs by using the reference bone mapping: all the rotations that the pose applies to the referenced bones are combined (in the order in which the references were specified -- note: rotations are NOT commutative) and are applied to the alternative rig bone. Reference bone mappings can also be used for determining vertex weights of custom rigs. If the alternative rig does not explicitly define its own vertex weights, the mapping of reference bones is used to infere a set of bone weights for each bone: All the weights of the reference bones on the master rig (with the weights definition of the default/master rig) are combined and apply to the bone of the alternative rig. Doubles are merged, and weights renormalized. Any vertex that is not weighted will be assigned to the root bone. Watch for messages in the log to verify this.

Vertex weights for proxies

Generally proxies are mapped to basemesh geometry, that means either to the body or to helpers, or a combination of both (though this is unusual). Proxies derive their vertex weights from the vertices on the basemesh on which they are fitted. A proxy defines its fitting as a 3d offset from one or three vertices on the basemesh, allowing it to deform as the human deforms. This mapping is used to transfer the vertex weights from basemesh to proxy: a proxy vertex referencing three basemesh vertices gets the bone weights of those three vertices. Doubles are merged and the weights are normalized, so that the influence factors are averaged or smoothed. In this case the weights applied to proxies are always those defined on the skeleton: either the weights explicitly defined on the skeleton, or the weights obtained from remapping the master rig weights to this skeleton using the reference bone mapping mentioned in the previous sections. There is an exception to this, which is explained in the next paragraph. A proxy can explicitly define its own vertex (to bone) weights. In this case it overrides the weights that would be obtained through the proxy mapping, and replaces them by its own custom definition of weights. These vertex weights are always defined to reference the bones of the master rig (default skeleton), so they need to be remapped to other skeletons using reference bone mapping. This also means that the custom weights of a proxy override any custom weights mapping that might be loaded with an alternative rig (but only for that proxy item). Custom proxy weights should be avoided if possible, but are in some cases required, eg. for properly rigging shoes which are rigidly fitted to feet, or an alternative topology which defines very accurate rigging on the face. The general advice for custom rigs is therefore to stay as close to the master rig as possible, reusing as much bones identically as possible, to achieve the best results for proxies with custom weights. In case of custom weight, the mhclo file must contain a line to indicate the weight file to use, using the keyword “vertexboneweights_file”, with the syntax: vertexboneweights_file filename.jsonw

Targets and modifiers

MakeHuman applies targets on the human by means of modifiers. Such a modifier has a value (usually between [0, 1] or [-1, 1]) that determines the effect of it on the human mesh. A modifier can control one or more targets and in apply the mixed sum of them on the mesh. Some modifiers.

Organization of targets

To make sure targets are linked to the right modifier, or to make managing targets easier, a certain discipline has to be observed in managing them. Targets are ASCII files in the data/targets or data/custom folder (in program or user folder) with a file extension of .target. The file name and folder targets are in implicitly determines their group name. A group is identified by a sequence of tags or tokens, which are taken from the file and folder name of each target, and which can be delimited by “-” symbols, to include multiple tokens in a single file or folder name. The lib/targets.py module takes care of loading all the names of the available targets and assembles them into groups. All targets with the same tokens end up in the same group. This allows easy querying and requesting of targets. The creator is pretty much free to decide what groups to create for their targets. By passing the name of the target group to a modifier, they can make sure that modifier controls the targets belonging to this group. It does not matter whether you use folders or - delimited file names for creating groups. An example: data/targets/cartoon/eyes-large.target is categorized in group (cartoon, eyes, large) data/targets/cartoon-eyes-large.target is also categorized in group (cartoon, eyes, large) Most simple modifiers work by specifying a group (for example cartoon-eyes) and a set of two tokens to append at the end, depending on the position of the modifier slider (eg. small, large which would allow controlling the targets cartoon-eyes-small and cartoon-eyes-large by moving the slider to the left or right).

Reserved group tokens

Some of the tokens in the file or foldername of targets are linked to values that variables of the human can have. Such variables are for example age, weight, muscle, gender. This table lists all the reserved tokens, and the variables they belong to.

Note that it is not a good idea to use simply “max” or “min” as a reserved token name, because they need to be unique (otherwise it would not be possible to determine whether they are a minimum for, for example, weight or height). These reserved tokens are declared in the lib/targets.py module. The variable can be set and queried on the human object, for example human.getAge(), human.setMuscle(0.7), human.getProportion(). The reserved tokens are NOT included in the group name, so you should at least include some other, non-reserved tokens in the file or folder names to assign your target to some meaningful group. The inclusion of these reserved tokens in your target names or paths makes it convenient to create macro modifiers.

Macro targets and modifiers

Macro modifiers are modifiers whose effect (the combination of targets they apply on the human) is influenced by macro variables set on the human (eg. human.set/getAge()). Therefore, macro modifiers often control a large set of targets (20 or even more). It’s also common for multiple modifiers to control exactly the same set of targets, for example the standard age/gender/race sliders all control the set of targets in the ‘macrodetails’ group. Each slider modifies a different variable (eg. human.setAge()) controlling the set of targets, and then reapplies a combination of the same set of targets on the human.

Macro modifier dependencies

There are other macro modifiers that can also depend on the same variables age, gender, race that is controlled by the main macro modifiers, as discussed above. For example the macro modifiers that influence the weight and muscle mass of the human, controls the targets group “macrodetails-universal”, the universal token is added by prepending “universal-” to all the file names of related targets, which all reside in the folder “macrodetails” (thus macrodetails-universal ends up being the group name). They could have been included all in the “macrodetails” group, but are kept separate for performance reasons, so that the “macrodetails” group of targets (which already contains some 40 targets or so) can be applied in realtime while moving the slider. Of course a change in the variable that these age/race/gender modifiers control will have an effect on the macrodetails-universal modifiers (because they control targets that depend on age -- their targets include -young- -old- etc. tokens in their filenames). This means there are inter-dependencies between macro modifiers, if one is updated, another one that depends on it should be refreshed. These dependencies can be descibed as follows: one macro modifier controls exactly one macro variable (such as age). It (or rather, the targets it controls) can however depend on multiple macro variables. The trivial case is where a macro modifier only depends on the macro variable that it controls itself. The tracking of these dependencies and deciding which modifiers must be updated when another is changed happens automatically. Dependencies are automatically tracked as soon as you add a modifier to the human (by calling modifier.setHuman(human)). To make this more performant in real-time, the propagation of updates to modifiers is limited while sliders are dragged (see realtimeDependencyUpdates in apps/humanmodifier.py). As you might have understood, creating modifiers that depend on human variables is almost automatic. The only thing you need to do is create some targets that resolve to the same group name, add variations for all the macro variables you want and add the proper reserved tokens to the file or folder name (refer to the left column in the table above), and create a modifier with this group name. Once added to the human, the dependencies are tracked automatically.

Organization of modifiers

We already discussed how we organized targets in groups, to make it easier to connect them to a modifier. Now we will discuss how we can do a similar thing for modifiers. Modifiers provide an easy way to manage targets. They allow reducing many of hundreds of targets to a more manageable number of controllable properties. But even then, we still have about 100 modifiers, which is still quite a lot. Therefore we group them in groups of categories, much like we do in the GUI. This makes it easy to get a grouped overview of all modifiers when using an alternative interface, think of commandline or scripting. The way in which you declare the group and other properties of a modifier determines what target groups it gets its targets from. Modifiers are declared with a group name, some intermediary “target name” and a number of tail tokens (2 or 3 usually, eg min/max). The group of targets controlled by these modifiers is composed as: groupname-targets_name-tail. Macro modifiers are declared with a group name, and a controlled variable. The groupname must be the same groupname of the targets they control.

Declarative modifiers

Instead of hard coding what modifiers you want to instantiate, MakeHuman offers a data-based format to declare what modifiers should be created, and how they should be shown in a user interface with sliders. A full declaration requires three files, all stored in data/modifiers. The first is the declaration of the modifiers (eg. modeling_modifiers.json). This format lists a modifier group name, and all the modifiers with their macro variable (if they are macro modifiers) or target name and 2 or 3 (if they are a modifier with a center point) tail tokens. The second file adds a description to the modifiers declared in the first file. It basically contains a key-value set, referencing the modifier name, and a textual description of what the modifier controls. A modifier name is constructed as follows: group/targets_name-all|the|tails and for a macro modifier: group/variable The third file defines how these modifiers should be shown in the user interface as sliders. It lists a set of categories, and the sliders to show in them, and how their values have to be saved in a mhm file (mainly a feature for legacy compatibility).

Your own plugin with modifiers, sliders and targets

Using the declarative data format described in the section above, it’s really simple. Have a look at the plugin plugins/0_modeling_0_modifiers.py. Notice it’s barely 5 lines of code. Creating your own modeling plugin is as simple as that, pointing it to your own modifier data files.

Binary targets

Because loading these ASCII target files, and iterating all target files and folders can be slow (especially on windows) in build versions, we compile all targets in the data/targets system folder into a single binary file, that furthermore acts exactly the same as multiple target files. The binary targets file is stored in the system data/ folder and is called targets.npz, it is created by running the compile_targets.py script, which is also automatically run in the build_prepare.py build routine. The file structure of the targets files is maintained in this binary file so that targets can be loaded from it in the same way as you would do with regular files. A change in the .target files, or newly added target files allow a recompilation of the targets npz file before the change is applied.

Material format

Introduction

MakeHuman material files have extension .mhmat and are simple ASCII definitions of the material settings to use for rendering. These material settings are primarily intended for configuring the material system within MakeHuman, but they are usually carried out to other software as well by the exporters. Not all properties will be supported for all given combinations of shaders, nor is there a guarantee that the exporters will honor all the properties in these files.

The Material System

The MakeHuman material system is closely based on the [standard rendering equation] as is used in OpenGL and DirectX forward rendering pipelines. MakeHuman supports shaders and the material format contains the most commonly used extensions in real-time rendering. As a consequence of this, users need to be aware of the implications of setting values such as ambient too bright, or diffuse too dark. All usual rules about rendering materials apply.

At the highest level, the material supports setting the most basic material properties, such as diffuse, specular and ambient color. These are supported both by shaders as well as the fixed function shading pipeline (which is used when no shader is specified). Fixed function also supports the diffuse texture option.

At a second level, a shader can be chosen. This refers directly to the filename of a [GLSL shader] as found in the data/shaders/glsl folder of MakeHuman. The selected shader will determine what material options will be available.

Various additional settings become available, depending on whether or not the chosen shader has support for these features, such as bump or normal mapping, ambient occlusion, specular mapping, etc. For most of these, a texture map and a scalar property between 0 and 1 can be set, the latter determining the intensity or influence of the effect.

Finally, MakeHuman also contains a built-in offline renderer that supports some additional properties such as subsurface scattering, for which some special properties are available.

Shaders

The most powerful functionality of the material system is unlocked when using shaders.

MakeHuman shaders are intended to be written in a modular way, so that shading features can be enabled or disabled using the material format. Examples of such features are bump mapping, normal mapping, specular mapping, diffuse texturing, ambient occlusion and displacement mapping.

MakeHuman includes a basic introspection function that allows it to determine what features a shader supports. It accomplishes this by looking for #ifdef and #ifndef statements. If a shader supports a certain technique, it is supposed to encapsulate this in #ifdef NORMALMAP ... #else ... #endif statements.

The material system manages these shading functions by defining specific pre-processor keywords when their corresponding shaderConfig property is set to true. Shaders are recompiled when needed and stored in a shader cache for efficiency.

MakeHuman passes scene and material information to the shaders by certain uniform parameters. Depending on which shaderConfig properties were enabled, additional uniforms could be exposed to the shader. The shader does not have to declare all these uniforms, however. MakeHuman binds only those uniforms present in the shader if they have the right corresponding names. Other properties are available from the OpenGL default uniforms or properties.

The following table lists these defines and the shaderConfig properties in the material they relate to:

Additionally, MakeHuman exposes the vertex tangent as a vertex property, which can be accessed by the shader.

 attribute vec4 tangent

Currently, [Lengyel’s Method] for tangent calculation is used, but plans are to change this to mikktspace http://bugtracker.makehumancommunity.org/issues/87 in the future.

Shader examples for MakeHuman can be found here:

• Litsphere shader: [[1]] [fragment]
• Phong shader: [[2]] [fragment]
• Normal map shader (wip): [[3]] [fragment]
• X-ray shader: [[4]] [fragment]

Basic pointers

• You can test the influence of material properties using the Material Editor plugin (which you can enable in Settings > Plugins, then restart)
• Using a diffuse texture? You probably want to set the diffuse color to 1 1 1 (pure white)
• Never set ambient color too bright, or your model will look completely white. 0.1 0.1 0.1 is a good value to start from.
• Emissive is usually set to black, unless you want a radioactive character
• Higher shininess means smaller and sharper highlight, lower shininess makes the highlight larger and more blurred
• Reduce specular color to reduce the specularity, make it black for a rough and non shiny object
• opacity 1 means fully opaque, 0 means fully transparent (invisible)

Scene definition

To complete the rendering equation, you also need to define the scene. To be more precise, the positions, types, color and intensities of the lights need to be set. By default, MakeHuman only has a default scene with only one pointlight. In MakeHuman, lights do not move if the camera is moved, or rather, the model is rotated while the camera and lights remain static. So light positions are defined relative to the camera position.

MakeHuman scenes are defined in .mhscene files, which are binary py-pickled files. They can be created or modified using the Scene Editor plugin (can be enabled in settings) and can be selected from the Scene library in the Render tab.

Fixed function shading pipeline supports up to 8 lights, specific light support depends on the shader used, some shaders only support a certain type or number of lights, other ignore the chosen scene altogether (for example the litsphere shader that uses pre-baked environment lighting). The built-in offline renderer also uses the specified scene.

Scenes are not exported along with the model.

Example

This is a full and complete (but fictional) example of a material file for MakeHuman, it shows all the available properties.

   # Example MakeHuman Material definition
   
   name Material_name
   tag some_tag
   tag another_tag
   description material description text here
   
   ambientColor 0.11 0.11 0.11
   diffuseColor 1.0 1.0 1.0
   specularColor 0.0470588235294 0.0470588235294 0.0470588235294
   shininess 0.96
   emissiveColor 0.0 0.0 0.0
   opacity 1.0
   translucency 0.0
   
   shadeless False
   wireframe False
   transparent True
   alphaToCoverage True
   backfaceCull False
   depthless False
   
   autoBlendSkin False
   
   castShadows True
   
   receiveShadows True
   
   diffuseTexture diffuse_texture.png
   bumpmapTexture bumpmap.png
   bumpmapIntensity 1.0
   normalmapTexture normalmap.png
   normalmapIntensity 1.0
   displacementmapTexture norm_displacement.png
   displacementmapIntensity 1.0
   specularmapTexture specular.png
   specularmapIntensity 1.0
   transparencymapTexture alpha.png
   transparencymapIntensity 1.0
   aomapTexture ambient_occlusion_map.png
   aomapIntensity 1.0
   
   # Sub-surface scattering parameters
   sssEnabled True
   sssRScale 5.0
   sssGScale 2.5
   sssBScale 1.0
   
   shader shaders/glsl/litsphere
   
   shaderParam litsphereTexture litspheres/lit_hair.png
   
   shaderConfig ambientOcclusion True
   shaderConfig normal False
   shaderConfig bump True
   shaderConfig displacement False
   shaderConfig vertexColors True
   shaderConfig spec True
   shaderConfig transparency True
   shaderConfig diffuse True
   
   shaderDefine ALPHA_MAP
   
   uvMap alt_uv_coords.mhuv

This is another example, this time a real one. It is the X-ray material, used for example when showing the skeleton:

   # Material definition for XrayMaterial
       
   name XrayMaterial
   ambientColor 0.1 0.1 0.1
   diffuseColor 1.0 1.0 1.0
   specularColor 0.3 0.3 0.3
   shininess 0.1
   emissiveColor 0.0 0.0 0.0
   opacity 0.1
   translucency 0.0
       
   transparent True
   backfaceCull True
   shadeless False
   wireframe False
   depthless False
       
   shader data/shaders/glsl/xray
       
   shaderConfig diffuse False
   shaderConfig transparency False
   shaderConfig normal False
   shaderConfig bump False
   shaderConfig spec False
   shaderConfig vertexColors False
   shaderConfig displacement False