Separate texture channels

This forum is aimed at user contributions, in the form of assets, side projects, code patches and similar.

Moderator: joepal

Separate texture channels

Postby yousry » Sat Mar 14, 2015 11:38 am

In addition to the color and normal map it would be useful to have additional maps for better render results:

AO (Ambient Occlusion) map from the high resolution model.
A Specular and Gloss map for a better environment light response.
A Curvature map could be used for physical events like rain / sweat etc.

I could donate (or we could update your actual) shader if you can support openGL 3+.
yousry
 
Posts: 7
Joined: Wed Feb 05, 2014 9:46 am

Re: Separate texture channels

Postby duststorm » Sat Mar 14, 2015 4:23 pm

The problem with OpenGL3 is that not everyone is able to run it.
There are a lot of unfortunate MakeHuman users that have Windows on their machines, where modern OpenGL support is painful, to say the least.

OpenGL3 support can be added, but at any cost MH must remain able to run on OpenGL 1.0 or 1.2 drivers, so a backward compatible solution must be provided. After all, having nice graphics is cool, but having an application that only a limited audience can use because of its graphics requirements is not a good idea. The most important aspect of MH remains the modeling part.

But we have considered adding AO to the integrated rendering engine in the past, so any contributions for this are welcome. Development of the graphics pipeline in MH has just not gotten a lot of attention lately.


Also note that some of MH's shaders already contain the feature to use a static AO map.
The material system supports specular and environment maps, but the shaders have not yet been updated to use them.

If you want to help us create a good uber shader, to replace the current phong shader, that would be awesone. It has been on the agenda, but we just haven't gotten around to it.
Also the normal mapping in our litsphere shader is not as accurate as we would want it to be, so it could use some attention too.

If you want to help us with the shaders, all the help is welcome :)
MakeHuman™ developer
User avatar
duststorm
 
Posts: 2569
Joined: Fri Jan 27, 2012 11:57 am
Location: Belgium

Re: Separate texture channels

Postby yousry » Sun Mar 15, 2015 1:10 pm

duststorm wrote:The problem with OpenGL3 is that not everyone is able to run it.
There are a lot of unfortunate MakeHuman users that have Windows on their machines, where modern OpenGL support is painful, to say the least.

OpenGL3 support can be added, but at any cost MH must remain able to run on OpenGL 1.0 or 1.2 drivers, so a backward compatible solution must be provided. After all, having nice graphics is cool, but having an application that only a limited audience can use because of its graphics requirements is not a good idea. The most important aspect of MH remains the modeling part.


All this is understandable and I will check what is possible with this setup. Nevertheless I'm using Mesa as minimum requirement (OpenGL 3.2) for my sources and some techniques like separable subsurface scattering, displacement mapping and the complete post process (aa, ssao, bloom, two-pass ray-tracing, depth-field , order independent transparency) will not be transferable.

duststorm wrote:If you want to help us create a good uber shader, to replace the current phong shader, that would be awesone. It has been on the agenda, but we just haven't gotten around to it.


Without support for GLSL includes, subroutines and partial compilation such uber-shader should be avoided. You will end up in an if else labyrinth which is unfavorable for GLSL.

duststorm wrote:Also the normal mapping in our litsphere shader is not as accurate as we would want it to be, so it could use some attention too.
If you want to help us with the shaders, all the help is welcome :)


If I understand you correctly you would prefer per-pixel normals instead of vertex normals?
I started with this task and you can give some early feedback . You can find the patches as attachment including the shader compile-trees. (A validation against the khronos spec)

For physically based shading we would need access to additional uniforms.

    * The model-matrix for world coordinates.
    * A light array.
Optionally:
    * A HDR map
    * A shadow buffer/map.

Whoops: I could not attach the diffs ?! Do you have a GitHub clone? (DropBox: https://www.dropbox.com/s/8aoatesd8w9ojv8/ForMakeHuman.zip?dl=0)


Code: Select all
diff -r 7920e0f723ac makehuman/data/shaders/glsl/litsphere_vertex_shader.txt
--- a/makehuman/data/shaders/glsl/litsphere_vertex_shader.txt   Sat Mar 14 18:46:05 2015 +0100
+++ b/makehuman/data/shaders/glsl/litsphere_vertex_shader.txt   Sun Mar 15 13:48:56 2015 +0100
@@ -32,37 +32,38 @@
 #ifdef NORMALMAP
     // Inputs
     attribute vec4 tangent;
-
-    // Output
-    varying mat3 tbnMat;    // Tangent to view space conversion matrix
-#else
-    // Output
-    varying vec3 vNormal;   // Vertex normal in view space
 #endif
 
+// Output
+varying vec3 vNormal;   // Vertex normal in view space
+varying vec3 vTangent;   // Vertex tangent in view space
+varying vec3 vBitangent;   // Vertex bitangent in view space (btw. binormal != bitangent)
+
+#ifdef debug
+   varying vec3 vDebugColor; // signal color for debugging purposes
+#endif
 
 void main() {
-    gl_Position = ftransform();
-    gl_TexCoord[0] = gl_MultiTexCoord0;
-    #ifdef NORMALMAP
-        vec3 vNormal = vec3(0, 0, 0);
-    #endif
+
     vNormal = normalize(gl_NormalMatrix * gl_Normal);  // Vertex normal in view space
 
-    #ifdef DIFFUSE
-        gl_TexCoord[0] = gl_MultiTexCoord0;
-    #endif
-
     #ifdef NORMALMAP
-        // Tangent and binormal in view space
-        vec3 tang = normalize(gl_NormalMatrix * tangent.xyz);
-        //normal = normalize(gl_NormalMatrix * gl_Normal);
-        vec3 binormal = normalize(cross(vNormal, tang));
-
-        tbnMat = mat3(tang, binormal, vNormal);   // TBN matrix, converts from Tangent space in view space
+        // Tangent and binormal in model inverse transpose
+        vTangent = normalize(gl_NormalMatrix * tangent.xyz);
+        vBitangent = normalize(cross(vNormal, vTangent));
     #endif
 
     #ifdef VERTEX_COLOR
         gl_FrontColor = gl_Color;
     #endif
+
+
+    // check that vs fs correspondent
+#ifdef debug
+    vDebugColor = vec3(1,0,0);
+#endif
+
+    gl_TexCoord[0] = gl_MultiTexCoord0;
+    gl_Position = ftransform();
+
 }



And the corresponding AST:
Code: Select all
Shader version: 120
0:? Sequence
0:46  Function Definition: main( (void)
0:46    Function Parameters:
0:48    Sequence
0:48      move second child to first child (3-component vector of float)
0:48        'vNormal' (smooth out 3-component vector of float)
0:48        normalize (3-component vector of float)
0:48          matrix-times-vector (3-component vector of float)
0:48            'gl_NormalMatrix' (uniform 3X3 matrix of float)
0:48            'gl_Normal' (in 3-component vector of float)
0:66      move second child to first child (4-component vector of float)
0:66        direct index (smooth 4-component vector of float)
0:66          'gl_TexCoord' (smooth out implicitly-sized array of 4-component vector of float)
0:66          Constant:
0:66            0 (const int)
0:66        'gl_MultiTexCoord0' (in 4-component vector of float)
0:67      move second child to first child (4-component vector of float)
0:67        'gl_Position' (gl_Position 4-component vector of float)
0:67        Function Call: ftransform( (4-component vector of float)
0:?   Linker Objects
0:?     'vNormal' (smooth out 3-component vector of float)
0:?     'vTangent' (smooth out 3-component vector of float)
0:?     'vBitangent' (smooth out 3-component vector of float)
0:?     'gl_TexCoord' (smooth out implicitly-sized array of 4-component vector of float)



The Frag diff:

Code: Select all
diff -r 7920e0f723ac makehuman/data/shaders/glsl/litsphere_vertex_shader.txt
--- a/makehuman/data/shaders/glsl/litsphere_vertex_shader.txt   Sat Mar 14 18:46:05 2015 +0100
+++ b/makehuman/data/shaders/glsl/litsphere_vertex_shader.txt   Sun Mar 15 13:48:56 2015 +0100
@@ -32,37 +32,38 @@
 #ifdef NORMALMAP
     // Inputs
     attribute vec4 tangent;
-
-    // Output
-    varying mat3 tbnMat;    // Tangent to view space conversion matrix
-#else
-    // Output
-    varying vec3 vNormal;   // Vertex normal in view space
 #endif
 
+// Output
+varying vec3 vNormal;   // Vertex normal in view space
+varying vec3 vTangent;   // Vertex tangent in view space
+varying vec3 vBitangent;   // Vertex bitangent in view space (btw. binormal != bitangent)
+
+#ifdef debug
+   varying vec3 vDebugColor; // signal color for debugging purposes
+#endif
 
 void main() {
-    gl_Position = ftransform();
-    gl_TexCoord[0] = gl_MultiTexCoord0;
-    #ifdef NORMALMAP
-        vec3 vNormal = vec3(0, 0, 0);
-    #endif
+
     vNormal = normalize(gl_NormalMatrix * gl_Normal);  // Vertex normal in view space
 
-    #ifdef DIFFUSE
-        gl_TexCoord[0] = gl_MultiTexCoord0;
-    #endif
-
     #ifdef NORMALMAP
-        // Tangent and binormal in view space
-        vec3 tang = normalize(gl_NormalMatrix * tangent.xyz);
-        //normal = normalize(gl_NormalMatrix * gl_Normal);
-        vec3 binormal = normalize(cross(vNormal, tang));
-
-        tbnMat = mat3(tang, binormal, vNormal);   // TBN matrix, converts from Tangent space in view space
+        // Tangent and binormal in model inverse transpose
+        vTangent = normalize(gl_NormalMatrix * tangent.xyz);
+        vBitangent = normalize(cross(vNormal, vTangent));
     #endif
 
     #ifdef VERTEX_COLOR
         gl_FrontColor = gl_Color;
     #endif
+
+
+    // check that vs fs correspondent
+#ifdef debug
+    vDebugColor = vec3(1,0,0);
+#endif
+
+    gl_TexCoord[0] = gl_MultiTexCoord0;
+    gl_Position = ftransform();
+
 }



And the corresponding AST:

Code: Select all
Shader version: 120
0:? Sequence
0:71  Function Definition: main( (void)
0:71    Function Parameters:
0:85    Sequence
0:85      Sequence
0:85        move second child to first child (3-component vector of float)
0:85          'normal' (3-component vector of float)
0:85          'vNormal' (smooth in 3-component vector of float)
0:88      Sequence
0:88        move second child to first child (3-component vector of float)
0:88          'shading' (3-component vector of float)
0:88          vector swizzle (3-component vector of float)
0:88            Function Call: texture2D(s21;vf2; (4-component vector of float)
0:88              'litsphereTexture' (uniform sampler2D)
0:88              Construct vec2 (2-component vector of float)
0:88                add (3-component vector of float)
0:88                  component-wise multiply (3-component vector of float)
0:88                    vector swizzle (3-component vector of float)
0:88                      'normal' (3-component vector of float)
0:88                      Sequence
0:88                        Constant:
0:88                          0 (const int)
0:88                        Constant:
0:88                          1 (const int)
0:88                        Constant:
0:88                          2 (const int)
0:88                    Constant:
0:88                      0.495000
0:88                      0.495000
0:88                      0.495000
0:88                  Constant:
0:88                    0.500000
0:88                    0.500000
0:88                    0.500000
0:88            Sequence
0:88              Constant:
0:88                0 (const int)
0:88              Constant:
0:88                1 (const int)
0:88              Constant:
0:88                2 (const int)
0:101      move second child to first child (3-component vector of float)
0:101        vector swizzle (3-component vector of float)
0:101          'outColor' (4-component vector of float)
0:101          Sequence
0:101            Constant:
0:101              0 (const int)
0:101            Constant:
0:101              1 (const int)
0:101            Constant:
0:101              2 (const int)
0:101        vector swizzle (3-component vector of float)
0:101          'shading' (3-component vector of float)
0:101          Sequence
0:101            Constant:
0:101              0 (const int)
0:101            Constant:
0:101              1 (const int)
0:101            Constant:
0:101              2 (const int)
0:105      move second child to first child (float)
0:105        direct index (float)
0:105          'outColor' (4-component vector of float)
0:105          Constant:
0:105            3 (const int)
0:105        Constant:
0:105          1.000000
0:114      move second child to first child (4-component vector of float)
0:114        vector swizzle (4-component vector of float)
0:114          'gl_FragColor' (fragColor 4-component vector of float)
0:114          Sequence
0:114            Constant:
0:114              0 (const int)
0:114            Constant:
0:114              1 (const int)
0:114            Constant:
0:114              2 (const int)
0:114            Constant:
0:114              3 (const int)
0:114        vector swizzle (4-component vector of float)
0:114          'outColor' (4-component vector of float)
0:114          Sequence
0:114            Constant:
0:114              0 (const int)
0:114            Constant:
0:114              1 (const int)
0:114            Constant:
0:114              2 (const int)
0:114            Constant:
0:114              3 (const int)
0:?   Linker Objects
0:?     'vNormal' (smooth in 3-component vector of float)
0:?     'vTangent' (smooth in 3-component vector of float)
0:?     'vBitangent' (smooth in 3-component vector of float)
0:?     'vDebugColor' (smooth in 3-component vector of float)
0:?     'litsphereTexture' (uniform sampler2D)
0:?     'AdditiveShading' (uniform float)
0:?       0.000000

Attachments
MakeHumanPerPixel.jpg
yousry
 
Posts: 7
Joined: Wed Feb 05, 2014 9:46 am

Re: Separate texture channels

Postby duststorm » Sun Mar 15, 2015 2:17 pm

yousry wrote:Without support for GLSL includes, subroutines and partial compilation such uber-shader should be avoided. You will end up in an if else labyrinth which is unfavorable for GLSL.

That's true. That's why I propose that OpenGL3 shaders should be a separate file, and should not be mixed with OpenGL1.2 compatible shaders.
For the simplicity of the 1.2 shaders, the IFDEF solution is manageable, for OpenGL3 shaders a different mechanism can be used if needed. In fact I have already implemented a mechanism in our material system for managing shader options to be compiled in. The shader system in MH automatically manages and caches compiled shaders and compiles then when needed. The shading options to enable can be set in the Material object, as well as the texture maps that serve as input for these shaders. Generic shader arguments are also accepted.

You can see this system at work when you enable the "Material editor" plugin in Settings (then restart MH). It gives you a visual editor that allows configuring the material. Shaders with custom input parameters will make new input fields pop up in the editor. Radio boxes allow enabling or disabling shader options, which controls the DEFINE pre-processor statements.

I have not yet had the time to document the shader mechanism, but you should be able to understand it pretty quickly by looking at the existing shaders in the data/shaders/glsl folder and playing around with the material editor.


yousry wrote:If I understand you correctly you would prefer per-pixel normals instead of vertex normals?
I started with this task and you can give some early feedback . You can find the patches as attachment including the shader compile-trees. (A validation against the khronos spec)

Great! I'll have a look :)
Yes, we definitely want per-pixel normals. Now, I might be wrong as it's been a while since I looked at the shader code (most of the current shader code in MH is written by me, still) but I thought we were already using per-pixel normals. But no guarantees that they are actually correct.


yousry wrote:For physically based shading we would need access to additional uniforms.

As mentioned above, chances are that the current material system already offers what you need.
If I remember correctly, I still need to complete the task of allowing arbitrary 4x4 or 3x3 matrix uniforms in the material system, but floats and textures already work.

Do you have a GitHub clone

We're currently only hosted on bitbucket. I have planned to create a read-only github copy, but haven't gotten around to it yet.
Creating it is nothing, but I need to set something up that automatically keeps it in sync.
MakeHuman™ developer
User avatar
duststorm
 
Posts: 2569
Joined: Fri Jan 27, 2012 11:57 am
Location: Belgium

Re: Separate texture channels

Postby yousry » Sun Mar 15, 2015 6:32 pm

Yes, we definitely want per-pixel normals. Now, I might be wrong as it's been a while since I looked at the shader code (most of the current shader code in MH is written by me, still) but I thought we were already using per-pixel normals. But no guarantees that they are actually correct.


Yes, the normal was already calculated in the fragment shader.
I don't remember, when exactly the re-normalization of the texture/tangent space is necessary (between shader stage interpolations), but I think I used a working solution. It is also only a minor change in the code.

(Errata: I copied the same diff twice please use the dropbox link.)
yousry
 
Posts: 7
Joined: Wed Feb 05, 2014 9:46 am


Return to User contributions

Who is online

Users browsing this forum: No registered users and 1 guest