Some Comments on Community Assets Downloading

Tech support and suggestions forum. If you only have a basic question on how to get started, please use the "newbies" forum in the community section.

Moderator: joepal

Some Comments on Community Assets Downloading

Postby MTKnife » Sun Mar 17, 2019 4:54 am

After a great deal of effort, I've finally downloaded all the community assets for the new, Py3 version (Alpha 2?). A couple of things....

Even though I copied over all the assets from the old version, I had to re-download dozens of them. This was probably because I had manually created directories for the old ones, and the new, automatically generated directories didn't match. Fair enough, but I ran into a couple of other problems. First, it takes forever to download everything because I have to do them one at a time: I can't just select a bunch of them (via checkboxes or multiple selection) and download them at once, and it literally took me weeks, because I did it while waiting on stuff in other windows (since I'm not gonna sit here for hours just downloading). For a person coming to Makehuman new, that's going to be extremely intimidating/frustrating/not worth the effort.

Second, I've also got dozens of materials that are being dropped into the master "data/materials" folder, rather than into the assets they belong to, and therefore not showing up unless I copy them manually to the right place. This is even though the materials in question are (in most cases, I think) properly registered in the database as belonging to the correct clothes. Not sure what's going on here.
MTKnife
 
Posts: 311
Joined: Tue Sep 01, 2015 5:22 am

Re: Some Comments on Community Assets Downloading

Postby wolgade » Sun Mar 17, 2019 10:45 am

MTKnife wrote:Even though I copied over all the assets from the old version, I had to re-download dozens of them. This was probably because I had manually created directories for the old ones, and the new, automatically generated directories didn't match.

Yes, this is annoying. You'll end up with duplicate assets and it's very hard to keep track on updated assets.
MTKnife wrote:First, it takes forever to download everything because I have to do them one at a time:

It might be a good idea to bundle the most popular assets in a big zip file.
MTKnife wrote:Second, I've also got dozens of materials that are being dropped into the master "data/materials" folder, rather than into the assets they belong to, and therefore not showing up unless I copy them manually to the right place.

This issue should be resolved. The whole idea of the asset downloader is to make sure that assets end up at the right place. Right now it's pretty pointless to use the asset downloader to get materials.
wolgade
 
Posts: 795
Joined: Wed Jan 30, 2013 6:50 pm

Re: Some Comments on Community Assets Downloading

Postby joepal » Sun Mar 17, 2019 4:35 pm

That it's not easy to download everything is by design. It's a service free of charge, but that also means that there's no way we'd be able to afford a hosting solution surviving a download-everything kind of bashing. As it stands, the asset downloader is pushing the borders of what we're capable of hosting as it is.

The user contributed assets repo is about 6 gb large right now. There are thousands of users using it. If even a fraction of the users started to download everything, we'd rapidly reach traffic volumes counted in terabytes.

All in all, I would very much prefer that people do not download everything. If mass downloading starts to become a phenomenon, it's possible I will be forced to put restrictions in place on the server to throttle excessive donwloading from individial IPs.

For now I'm hoping that the nuisance of having to click through assets one by one is enough to keep the traffic volumes within a sensible range.

The asset downloader should handle materials correctly if the uploaded material correctly specifies what asset it belongs to. However, this is a relatively new feature and not all that tested. I'm hoping that at least third-part materials belonging to core assets will be treated in the correct way now.
Joel Palmius (LinkedIn)
MakeHuman Infrastructure Manager
http://www.palmius.com/joel
joepal
 
Posts: 4465
Joined: Wed Jun 04, 2008 11:20 am

Re: Some Comments on Community Assets Downloading

Postby MTKnife » Sun Mar 17, 2019 6:52 pm

Actually, after posting I realized that I'd already done away with the manually named folders, after the old version of the asset downloader was put in place. Without looking at the code, I suspect the problem is that the directory names changed between the Py2 and Py3 version of the asset downloader, and that's resulted in some materials being tied to assets that now have different directory names.
MTKnife
 
Posts: 311
Joined: Tue Sep 01, 2015 5:22 am

Re: Some Comments on Community Assets Downloading

Postby MTKnife » Thu Apr 04, 2019 3:54 am

OK, an update on misplaced materials: no, it's not caused by old assets' directory names being stored in the database. Last night I downloaded Elv's male kilt and the plaid material for it; both assets are new, and the material is properly pointed to the matching asset. However, the asset downloader drops the material into its own directory in the "material" directory, where it does nobody any good.

I still suspect the naming pattern for directories changed between 1.1 and 1.2 (specifically, how it handles spaces). Beyond that, one of two things must be happening (and yes, I could dig into the code, but it's likely to be weeks before I have the time to do that):

The asset uploader is still using the 1.1 directory-naming algorithm.
The community assets plugin is using different directory-naming algorithm for clothes and for materials.
MTKnife
 
Posts: 311
Joined: Tue Sep 01, 2015 5:22 am

Re: Some Comments on Community Assets Downloading

Postby joepal » Thu Apr 04, 2019 9:37 am

This sounds like a bug to me.

If you have time/energy, any chance you could summarize the problem at https://github.com/makehumancommunity/c ... oad/issues and include a list with examples of assets that get maltreated by the downloader?
Joel Palmius (LinkedIn)
MakeHuman Infrastructure Manager
http://www.palmius.com/joel
joepal
 
Posts: 4465
Joined: Wed Jun 04, 2008 11:20 am

Re: Some Comments on Community Assets Downloading

Postby MTKnife » Sat Apr 06, 2019 3:51 am

OK, done.
MTKnife
 
Posts: 311
Joined: Tue Sep 01, 2015 5:22 am


Return to Bugs, problems and feature requests

Who is online

Users browsing this forum: No registered users and 1 guest

cron