A Server For Blender

[Ricardo2020]

I have a new machine, that is, new to me. It is actually quite old from a hardware generation perspective. I named it the Duke of URL (not to be confused with the timeless classic song entitled "Duke of Earl").

In Decenber of 2024, I took possession and ownership of a used and dirty Hewlett Packard Enterprise DL360 G9 single rack space server. It was being decommissioned due to age and the fact that the RAID battery went dead, resulting in it not being able to boot. I could power it on, but I could not get past the ILO screen. I cleaned up the iron to like new and set about enabling this poor neglected and unwanted beast. I saved it from being parted out and thrown away as trash.

HPE DL360 G9 with lid off

When I got the box, there were two baby Xeons inside, each of which was a quad-core eight-thread part. It could do 16 threads at 2.4GHz on two LGA2011V3/4 sockets. Of the twenty-four DDR4 DIMM slots, eight were populated with 16GB ECC sticks clocked at 2166MHz totaling 128GB. There were four 1.2TB Seagate Constellation 10K RPM SAS spinners on hot swap sleds with room for four more. A DVD-ROM filled the ODD slot. Four gigabit and one ILO ports, several USB3 ports, a serial and a VGA port stuck out the back. It had two hot swap 500watt PSUs.

I wanted more. In went two monster Xeons with 14 cores and 28 threads each, for 56 threads that could idle at 2.4GHz and throttle to 3.5GHz. Holy balls. The RAM is still at 128GB, but I will eventually put in 256GB more for a total of 384GB. The machine can actually run twice that amount using standard ECC. Using load reduced sticks, one can take it to 3.4TB!

I put a PNY Quadro P600 in the primary riser 16x PCIe slot. It is a little card, but I plan to do CPU rendering rather than GPU based runs. However, that little Quadro lets me review the results at 4K on two screens. I only have one 1080p screen hooked up, but it is IPS and lovely to look at.

I put a Creative Labs Sound Blaster Audigy FX in the primary riser 8X PCIe slot. This has eight channels out or in. Dolby THX 7.1 is possible and the sound is flawless without any audible noise or distortion, both of which are barely measurable. A good pair of headphones yields studio quality playback. I have yet to send the output to my control room reference speakers.

I swapped out the DVD-RAM for a DVDRW. Eventually, this will give way to a Bluray MDisc writer...

After messing with the hardware, I got the RAID array to work. This meant removing the battery and resetting the controller to work without it. Going for RAID 5 across four drives, I ended up with a 3.4TB fault tolerant volume. I plugged a USB stick with Linux Mint Cinnamon v22.1 (Xia) into the right front USB port and booted into the OS live. From there, I ran the installer and configured the server as a machine with two roles, one being a server and the other a really powerful workstation.

I installed all my tools:

The OS - Linux Mint Cinnamon 22.1
The Updates - NVidia flatpack runtimes
The Drivers - they were in the OS already
MS Fonts - so Libre and Microsh*t can be friends
The GIMP - the GNU Image Manipulation Program
Blender with MPFB - we know what these are for
Brasero - Optical Disc Writer
VLC with FluidSynth - Media Playing Swiss Army Knife
Genral MIDI SoundFont - the larger Roland Canvas for VLC
Audacity - Two-track Recorder
Ardour - Multi-track Recorder
Calf Plugins - Audio Tools for Recorders
Hydrogen - Drum Machine and Sequencer
Krita - Vecter Imaging Suite
Darktable - Virtual Photography Dark Room and Light Table
Filezilla - Server to Peer File Transfer Environment
Firefox - Web Browser with de-enshitification plugins
Virtual Box with Extentions - run virtual machines with other OSs
DOS Box - for running vintage DOS wares such as DOOM, etc...

The browser starts up on a black local page, loads black new tabs when needed, blocks ALL ads, blocks social gleems, and has download helpers. It sports a dark theme. All the needed URLs live on the bookmark bar. It is blazing fast and secure enough to make the web safe and pleasantly usable.

HPE DL360 G9 with lid off

I have more parts coming, including a four-way M.2 PCIe16X card for the secondary riser, bigger GPU, more RAM, larger storage drives and a dual 10Gb FLOM card.

So....

Does this Duke of URL have enough power to do what we do here at the MakeHuman Community?

Will this old but trusty server Blend?

In short, yeppers. It will do it while idling...

The ugrades planned will make it bigger better faster more, that's all.

In cunclusion, I say get to know an e-waste person who has IT to clean up after. These servers are about ten years old but still good enough for most jobs today. Many are coming off lease and cheap to attain. They make great workstations when the right parts are fitted. And if anybody asks, they run Linux Mint just fine. Plus, they are excellent and very rugged servers well worth getting.

Thanks for reading and please feel free to comment...

[tomcat]

It's always interesting to read about computer archaeology. Although it goes against my ideology — I usually have a serious workload and take new components.

Render on CPU or GPU is quite a controversial issue. But I managed to get PNY Quadro RTX 5000 at a very big discount.

What kind of RAID controller do you have? I have an ASUS Hyper M.2 and Intel VROC — there is no battery. RAID5 is certainly more reliable than RAID0 — it has already failed me.

[Ricardo2020]

Howdy, Tom.

Normally I do not say too much about the hardware I use except some general descriptions, but when I found this old beast and got it working, I had to share the news.

About that RAID controller, it is the one that goes in the mezzanine slot directly on the mobo rather than taking up a riser slot, of which there are only three since this a 1U chassis. The controller can run without a battery. The battery is a 95 watt device designed to keep the spinners going until the cache has been written if the server falls out of power. There is a 2GB cache on the controller. I run it RAID 5. There are four drives, so the system has what we like to call a hot spare. There is a total capacity of 4.8TB, but once the RAID spans those platters, there is 3.4TB of usable space.

I have eight sleds. Four have blanks. Four are filled. There drives are rather small at only 1.2TB each. I am currently looking for larger spinners. 2.5inch SAS units are hard to find. They are also pricey when one is contemplating eight with sleds. I am planning on having two arrays of four drives each and booting from an M.2 on a PCIe card. Currently, I am booting from the RAID. It ain't an F1, but it gets the job done.

In the near future, I want to get a Quadro RTX single slot solution. I have seen them with 16GB of VRAM, but they are wallet flatteners and eye waterers. I must find a deal. Here in the US, those deals are rare since workstation GPUs at a fair price are in short supply. America has gone ape shit bananas...

[Ricardo2020]

I am familiar with the Asus Hyper part in your box. You must have the four stick version with the cover and that tiny flat fan. It is a 16-lane device that supposedly dual bifurcates the slot to act as four four-lane channels since each stick can use four lanes.

The card I am installing into the server is a bare metal card that depends on the mobo to do the bifurcation. The DL360G9 can bifurcate the primary riser but I am not sure about the secondary. Also, the BIOS flash is quite old and needs updating, but the files are behind a license wall.

If the card, made by a Chinese company known as GLOTRENDS, fails to work, I will scale back to a single M.2 stick on a different adapter card. I just hate wasting all those juicy PCIe lanes. If I can get the BIOS updated, I stand to have better luck. We must wait and see... I am running ILO 4, and it works OK.

The Xeon E5-2697v3 is a very powerful CPU. Yes, I know it is a fossil, but it can still say howdy to today's parts. The cores are performance level and they hyperthread. With two of these chips yielding 56 threads, the system idles even when you tell it to do something big. I have yet to see Cycles on Blender ramp up those seven front fans.

[Ricardo2020]

The M.2 card arrived today and was fitted into the server.

Now all I have to do is set up dual bifurcation in the secondary slot in the BIOS and populate the card. I will probably go with Samsung EVO 990 2TB sticks, at 130 bucks each. 8TB on four cards running RAID5 for about 6TB of flash storage. I do not know whether it will boot from this stack, but if not, it will be a hella front line hot file tier.

More on that as my wallet reinflates...

Also, this will be software RAID, and it might also support JBOD. We shall find out when the sticks go in...

[tomcat]

Howdy, Tom.
The battery is a 95 watt device designed to keep the spinners going until the cache has been written if the server falls out of power.

That's an interesting solution. I use UPS, which seems more reliable to me.

In the near future, I want to get a Quadro RTX single slot solution. I have seen them with 16GB of VRAM, but they are wallet flatteners and eye waterers. I must find a deal. Here in the US, those deals are rare since workstation GPUs at a fair price are in short supply.

My GPU was sold with virtually no warranty. I'm lucky it works fine.

America has gone ape shit bananas...

The whole world is like that now.

The Xeon E5-2697v3 is a very powerful CPU. Yes, I know it is a fossil, but it can still say howdy to today's parts.

Yeah, it's powerful. It is only slightly inferior to my newer one, but it is a server CPU. I got a workstation CPU.

And my motherboard only supports one CPU.

[Ricardo2020]

American Power (APC) makes a rackable UPS for this server, rated at 2KW. The server has two 500watt PSUs that draw a kilowatt at boot but settle down to around 200watts total most of the time. The extra oomph in the UPS makes it more robust and provides the spinners and other faff plenty of time to get down. A 1KW UPS gives about five minutes, wherein the bigger one will go for ten. Spinners are slow when writing, so that extra juice is nice. The price is not and it is quite out of my reach. This box will not be on that much since I do not need an on-prem cloud 24/7/365. I will fire it up when a job is big enough to need that much iron. The stuff it does is not mission critical since there are other 'stations sharing the work.

I have found the little brother to your GPU, the RTX4000, currently the most powerful single slot 16GB card of that family. I can own it for about 300 skins. It runs on 75 watts, so it can power off the PCIe slot. HP was kind enough to provide a supplemental power socket if needed to get the PCIe slot power combined with the cable on the mobo to around 125 watts. For the RTX4000, this wire is not needed. It is a very able card that sips the juice. Think of it as half of what you have. However, I am thinking it will do fine in the server. I cannot fit anything bigger in there... I know it will blend. It is enough for me.

When I was trying to figure out which CPUs to socket, it boiled down to price and available power supply. The E5-2697v3 is a 140 watt part. Two pulling 280watts are OK for those two 500 watt PSUs, and since I have the smaller heat sinks without the extenders, putting in the 16core upgrade would have exceeded power and thermal limits under load. I opted for a cooler and quiter system at the expense of four threads, but I got higher clocks that more than made up for that tradeoff. I got the chips for 20 dollars each. They were used. When the package arrived, the chips were spotless clean and ran perfect after ihe part swap.

I also learned today that both 16x slots support bifurcation and dual bifurcation. Thus, I should be able to toggle that on in the BIOS and get the server to see four M.2 sticks. What I do not know is how HPE Intelligent Provisioning will handle the sticks as a RAID5 volume. If I can get that working, the combined throughput would be crazy, like 32GBsec, which means any stick I put in there will be a bottleneck because the card is much faster than the sticks. It matters not since there are narrower bottlenecks elsewhere, such as the GPU, etc...

[Ricardo2020]

Comparing one W chip against two E chips...

One of these is a little less powerful than two of the E chips, but they run hella lot faster overall, so jobs are likely to finish sooner. Also, that RTX5000 will move things along nicely, reducing my Duke of URL to a slug in comparison, not to mention the single W draws half the power of two E chips. It is a wonderful platform you have there...

LGA2066 is a nice socket. It works out a lot of what bothered the 2011 stuff. Overall, it is a faster platform that is twice as efficient power wise compared to mine. The only upside to my box is the huge 56-thread count. That is a lot of CPU worker bees... Oh, and the 3.4TB RAM limit...

Another minor consideration is the two PCIe16x slots. Each CPU has one of these and they work independently. Thus, the GPU will not get lanes stolen by the M.2 card. Servers usually have goms of PCIe lanes, and in Duke's case, there are 40 CPU lanes and 8 PCH lanes.

There are three RAM channels per CPU and each channel can have four DIMMs. That's six channels on twenty-four sticks. The bandwidth is astronomical, more than making up for the faster modern stuff. It is ECC and can operate with load reduced sticks. I went with standard ECC since it has faster CAS, LAS and RAS figures.

Progress...

[tomcat]

American Power (APC) makes a rackable UPS for this server, rated at 2KW.

I have given up on products from this company.

Spinners are slow when writing, so that extra juice is nice.

I no longer have classic HDDs, only SSDs. But they also have a cache and take time to shut down.

I have found the little brother to your GPU, the RTX4000, currently the most powerful single slot 16GB card of that family.

It only has 8GB of memory. I was originally planning to get this model, but a more advanced one came along at a lower price, albeit with a questionable warranty. However, in our conditions no warranty looked reliable, so I took the risk.

[Ricardo2020]

My bad on the RTX4000 VRAM count. Also, this card needs a power connection my server does not have. I am needing a bus powered card that is only one slot thick and no more than standerd full length. The longer and bigger cards simply will not fit. And so, off I go in search of something that will work in there without being wimpy or bank breaking...

Trip Lite makes equivilent UPS, and there are others like Belkin, etc... I cannot afford them for alas, I am poor. I am also old...

As for drives, I use SSDs when speed matters, and HDDs when durability matters. If an SSD fails, it really is gone. If a spinner fails, it might be recoverable. Also, cost per GB matters. HDDs are still cheaper, and my wallet is so thin the light shines through...

Since I am not wealthy, finding this old beast got me excited since I have always wanted a dual socket system because of the massiive memory stack that can be fitted.

I have also always believed that a machine should be twice as powerful and four times as durable as is actually required to get the job done. This way, it never overheats and is much less prone to failure.

I am typing this reply on a workstation I built in 2014. Back then, it was the cat's meow. Today, it is a slug compared to the new stuff. However, Linux Mint runs very well and is plenty fast enough for what it does, which is mostly research, light rendering, and hot file storage. It also plays media nicely. I do not use it for games...

Duke of URL is the most powerful machine I have ever messed with. Being my first real server, I spent about three weels boning up and learning about how to deal with the thing. I had to go back to school. I watched a ton of videos and read a pile of tech papers. It was not hard, just time consuming. Plus, it already had 128GB in the DIMM slots. Nice.

Once I figured the unit out, it made sense and was easy to get going. I was fortunate in finding those two Xeons. The guy who sold them must have taken a farktard of the chips as e-waste from a data center decommissioning haul, refurbed the lot, put them in the little plastic tray packs, and priced them to move. Since I got my pair, they have gone up to 28 bucks each. The 16 core varient sels for 700 dollars each and they are also refurbs. Ouch...

I learned a lot about HPE. The DL360G9 is regarded by many as being one of the best servers they ever made. It is built well and super easy to work on. Every single part, right down to the last tiny screw, can still be had and the price is right if one knows where to look. Cloud Ninjas and Server Supply come to mind... These guys carry everything HPE, both old and new.

I was able to get the secondary riser with cage, a 170 buck part, for a more sensable 40. It came packaged well and looking new. It gave me that all important second PCIe16X slot which will allow the four M.2 sticks to be fitted. Saving the dollars is gonna take a while. It currently costs around 500 smackers to get the set new and I ain't buying used SSDs...