Sync mesh for blender

Works in progress and technical screen shots.

Moderator: joepal

Re: Sync mesh for blender

Postby joepal » Sun Jun 12, 2016 6:54 pm

I've started to refactor the communication protocol now, to allow for more verbose commands and responses. In practice the idea is to start sending json envelopes in both directions and include keys for error messages etc. I'm planning to allow a custom "data" key too, which can be any allowed json type (hash, array, scalar..).

However, this caused me to run into some trouble since the python "json" library is crippled bordering on useless since it doesn't support several core python data types (notably the python array) when encoding. I've mostly solved these problems, but it's soon to commit.

I guess this might be good to know in case there's a large commit in the pipeline concerning something else.
Joel Palmius (LinkedIn)
MakeHuman Infrastructure Manager
http://www.palmius.com/joel
joepal
 
Posts: 4465
Joined: Wed Jun 04, 2008 11:20 am

Re: Sync mesh for blender

Postby MTKnife » Sun Jun 12, 2016 8:58 pm

Pandas has a method to convert DataFrames to JSON. Is there not an equivalent for Numpy?
MTKnife
 
Posts: 311
Joined: Tue Sep 01, 2015 5:22 am

Re: Sync mesh for blender

Postby joepal » Mon Jun 20, 2016 9:06 am

Ok, I've now committed a new version to github.

* All communication is managed via JSON objects. Error messages are included in the object under the "error" key. Data (if any) is included under the "data" key.
* Moved the scripts to subdirectories in anticipation of having wider variety
* Renamed mh_sync_mesh to mh_sync

A few thoughts/discoveries:

Syncing the mesh when the character has a non-rest pose in MH leads to results looking like something from HR Giger.

The behavior with having to repeat the sync pose is strange, but hypothesis: Maybe we're working with *global* rotations in blender where we actually mean *local*? In that case the parent bone will influence the rotation of the child bone. Thus, if child bone b rotates before parent bone a, the child bone b rotation becomes messed up. However, on the next iteration parent bone a was already rotated properly, meaning child bone b doesn't get rotated an extra time around.

If anyone with some blender scripting insight wants to take a look, the sync code is on lines 39-43 here https://github.com/makehumancommunity/c ... perator.py

Anyway, a JSON communication object might now look like this:

Code: Select all
{
  "function": "getCoords",
  "error": "",
  "params": {},
  "data": null
}


And the response is either the same json object returned but with the data key set, or with the error key set.
Joel Palmius (LinkedIn)
MakeHuman Infrastructure Manager
http://www.palmius.com/joel
joepal
 
Posts: 4465
Joined: Wed Jun 04, 2008 11:20 am

Re: Sync mesh for blender

Postby jcpalmer » Mon Jun 20, 2016 2:04 pm

I did progress some more about a week back. When you said you were doing something about args, I just worked on another part of my pipeline. I am primarily a Blender exporter, and WebGL framework BabylonJS importer.

What I have done in makehuman server, is change to a method to get the rest matrix. The first attempt just wrote it out as is, and I used that pretty unreadable code to transform it. I borrowed that from MHX2 in desperation, and it did not work either. Then I hit the button twice in frustration, and noticed it transformed some more.

The main part of the server is now:
Code: Select all
rmat = bone.getRestMatrix('zUpFaceNegY')
pose += '\n"' + bone.name + '":[' + arrayAsString(list(rmat[0,:])) + ', ' + arrayAsString(list(rmat[1,:])) + ', ' + arrayAsString(list(rmat[2,:])) + ', ' + arrayAsString(list(rmat[3,:])) + "]"

I figured that if the 'zUpFaceNegY' was passed from the blender client that if you wanted to make another client, e.g. 3DS Max, it could pass whatever did the most work on the server.

On the client, it uses the matrix as passed by the server. It is still not perfect, and Collada imports really have problems. I ended up putting a select all operation in the matrix assignment loop to get the final result in 1 go. Expressions seem to work completely. Poses are close.

Will adapt the call to your changes and PR/accept in the next few days.
jcpalmer
 
Posts: 115
Joined: Tue Dec 16, 2014 4:14 pm

Re: Sync mesh for blender

Postby Aranuvir » Wed Jun 22, 2016 11:58 am

Git-confusion?
Personally I totally agree using a subfolder structure on github to organize different projects on scripts. The actual structure might be useful for someone who wants to use the plugin-repository as a whole. However, having a mh-script and its potential blender counterpart in different subdirectories could turn out confusing if someone wants to use only a specific script, especially if the repository should be continuing growing.
Aranuvir
 
Posts: 1314
Joined: Sun Oct 12, 2014 2:12 pm

Re: Sync mesh for blender

Postby joepal » Wed Jun 22, 2016 12:10 pm

Unfortunately there is no 1:1 relation between the scripts, especially not after what I added today. 8_socket_server just provides a means to access MH from the outside. This can be used by a variety of different things, not only the blender mesh sync thingy.

For example, today I added a few command-line scripts which also communicates with the socket server. The idea is to provide command line tools for things such as setting a target and rendering an image of the viewport.

So I think that in the longer run it'll be less confusing to separate these things into sub folders and be clear with which dependencies exist. Exactly how the folder structure should look is just how I happened to place it right now. There might be a more logical structure.

Currently the dependency graph is this:

Code: Select all
                          mh_sync
                         /
1_mhapi - 8_socket_server
                         \
                          CLI tools
Joel Palmius (LinkedIn)
MakeHuman Infrastructure Manager
http://www.palmius.com/joel
joepal
 
Posts: 4465
Joined: Wed Jun 04, 2008 11:20 am

Re: Sync mesh for blender

Postby joepal » Wed Jul 06, 2016 5:35 am

I've now made a more coherent demo of the new features and posted it at https://youtu.be/FIxdOvHDAmQ
Joel Palmius (LinkedIn)
MakeHuman Infrastructure Manager
http://www.palmius.com/joel
joepal
 
Posts: 4465
Joined: Wed Jun 04, 2008 11:20 am

Re: Sync mesh for blender

Postby Rhynedahll » Sun Sep 18, 2016 11:31 am

joepal wrote:Unfortunately there is no 1:1 relation between the scripts, especially not after what I added today. 8_socket_server just provides a means to access MH from the outside. This can be used by a variety of different things, not only the blender mesh sync thingy.

For example, today I added a few command-line scripts which also communicates with the socket server. The idea is to provide command line tools for things such as setting a target and rendering an image of the viewport.
CLI tools
[/code]



This is very interesting!

I am particularly interested in the command line usage.

I've watched your video, but it did not make it clear to me whether it would be possible in future to send commands through the socket server to create a mesh from scratch -- as in a sequence of slider parameters -- and retrieve that mesh through the socket server.

I understand that none of this functionality is present now, but are there any limitations that would prevent this sort of modification?
Orphan, an epic fantasy by H. Jonas Rhynedahll
The Key to Magic: Book One
Available on Kindle at Amazon: The Key to Magic
Rhynedahll
 
Posts: 195
Joined: Sat Nov 28, 2009 1:23 pm

Re: Sync mesh for blender

Postby Rhynedahll » Sun Sep 18, 2016 4:02 pm

The socket plugin has an error on load.

Code: Select all
Could not load 8_server_socket
Traceback (most recent call last):
  File "c:\jenkins\workspace\Windows_release_11x\buildscripts\win32\build\makehuman\out00-PYZ.pyz\mhmain", line 539, in loadPlugin
  File "plugins/8_server_socket\__init__.py", line 129, in load
    taskview = category.addTask(SocketTaskView(category))
  File "plugins/8_server_socket\__init__.py", line 60, in __init__
    self.modops = SocketModifierOps(self)
  File "plugins/8_server_socket\modops.py", line 18, in __init__
    self.api = G.app.mhapi
AttributeError: 'MHApplication' object has no attribute 'mhapi'


I'm using MH 1.1.0 and the latest plugin from Github.

Any suggestions?
Orphan, an epic fantasy by H. Jonas Rhynedahll
The Key to Magic: Book One
Available on Kindle at Amazon: The Key to Magic
Rhynedahll
 
Posts: 195
Joined: Sat Nov 28, 2009 1:23 pm

Re: Sync mesh for blender

Postby Aranuvir » Sun Sep 18, 2016 6:31 pm

Just quickly from my memory (not sure if I'm right). IIRC the install process has changed since the video was created. You need to install the plug-in 1_mhapi, too. (https://github.com/makehumancommunity/c ... /makehuman)
Aranuvir
 
Posts: 1314
Joined: Sun Oct 12, 2014 2:12 pm

PreviousNext

Return to WIP (Work In Progress)

Who is online

Users browsing this forum: No registered users and 1 guest