contact Me 


Winnipeg, Manitoba
Canada

​Offering 3D services for film, advertising, web, corporate and birthday parties

Blog

this is a blog.

IKEA cheap renderfarm update: Remember that thing i did? Maybe don't do that thing

Patrick Letourneau

 

 

So, I stuffed a bunch of computers in tiny drawers with stock coolers. As it turns out, Intel don't equip stock heatsink setups with much overhead for low ventilation areas. Who would have thought? Time for the cheap render farm, part 2

Initial tests with the helmer renderarm were within parameters, hitting 79c on the hottest core on average. Thermal management kicks in at 100c, so i had some headroom.... some.

Eventually though, temps on long term renders were hitting the low-mid 80s C. This left me a very small margin should anything go wrong, and made it hard to sleep during overnight renders. I realize that thermal throttling should technically prevent damage, but do i really want to take any chances? Not really. I also assumed that a thermal event halt any render in progress, which is not cool on client jobs.  I looked into low profile coolers, but none of them are made for 100% CPU usage for 100+ hours at a time.

So i built some spreadsheets, looked at moving to ITX cases,  which would mean new motherboards and fresh OS installs. No thanks. Eventually i found the Fractal Core 1000 case, which is small, made for Micro ATX, and can fit the big fat tower coolers i so desperately need.

What i end up with is surprisingly close to the Helmer in volume, almost as small in footprint, and very friendly to the coming GPU rendering revolution. I'm down from 3 fans to a single 120mm in each unit, so i can't even tell when they're on, and they haven't gone above 60c in render. Cost per node was only about $60 as the cases were nice and cheap at the time. I now render with impunity. In the end, i spent a few dozen hours grinding metal and making a huge mess for nothing. But now you wont have to in order to setup a good, cheap renderfarm.

 

Banana for scale

Banana for scale

Not bad

Not bad

 

 

Twice the power of a 12 core Mac Pro, for a quarter of the price*

Patrick Letourneau

*Power refers to ray tracing rendering. You can't play Crysis on these things

4 quad core CPUs that turbo up to 3.9ghz

EDIT: HELMERS ARE WARM 

I've noticed a trend among motion graphics/C4D users of fawning over the Mac Pro for its rendering power.  While i can agree that the Mac Pro has some strong points, its price/performance ratio for 3d work is bad. 

Before you accuse me of fanboyism, know that I work for an iOS developer, I love my iPhone, I've converted my girlfriend's entire family to Macbooks and iPads, and I own a bunch of Apple stock. I love a lot about Apple

Around the same time as the Mac Pro fever I read yet another post online about a small 3d shop stuffing a ton of consumer grade CPUs into an IKEA Helmer cabinet. I had been planning one for years, tweaking a spreadsheet with prices and Cinebench render scores to find the most power for the least money. After considering the 4770s low power CPU, and even purchasing and testing a Intel 4670, I settled on the nicely priced 4770 (non K) CPU. I sold my 2 space PCs on Reddit for much less than i should have, in order to rush into purchasing the parts for this little Helmer render farm. 

I won't go too far into the build detail, as that's been covered many times before, by quite a few sites, one of the best being CGCookie. I will add that I've laid all my parts down on a lining of cork board. Cork is a super insulator and should keep my nodes safe from static events.

PARTS

  • Intel 4770 CPU
  • 8GB of consumer RAM. Anything will do. Room for up to 16GB, which i will need...
  • Garbage 2.5 inch Laptop drives. We all have these laying around, and HD performance has a negligible impact on long renders
  • MSI H81M P33 motherboard. At $54, it's a pretty good motherbaord
  • Antec Basiq VP450 (not pictured above). Non-modular, but pretty compact and very cheap, while still being over 80% efficient (despite not being 80plus)
  • Momentary switch, bezel cables, 2 fans (COOLING MATTERS), motherboard standoffs, cork board

All tolled, I spent about $2000 on 4 nodes, totaling 16 very highly clocked cores and 32 threads.

PERFORMANCE

 

Performance is awesome. In rough terms, a render that took my Core i7 3930k 6 core workstation 15:00 to complete, rendered in 4:10. This is on a still image, with tons of network overhead and inefficiencies. Rendering animations, where each node can run at 100% at all times, yields even better results. In a slightly more synthetic test, Cinebench, each 4770 node scores about 750 points. My $2000 6 core workstation scores just shy of 1000, a $5599 8 core Mac Pro scores just over 1200. I consider Cinebench as close to real-world as it gets, considering the scores are just a measure of time it takes to render a real 3d test image, loading all cores.

 

Here's a quick and dirty video of what the Helmer render farm feels like

 

VALUE. DEAR GOD, THE VALUE

Price/performance is why I'm writing this post. This Helmer render farm is over 6x faster per dollar than a Mac Pro at ray tracing in Cinema 4d. How? Well, the Mac Pro uses Intel Xeon chips. Xeons give you access to more than the 64GB cap of Intel Core i7 CPUs, and let you use ECC memory. While the ram helps if you're working on The Hobbit, most 3d artists never touch 16GB of RAM on a render. On top of that, the core clock goes down as the core count goes up. The 12 core Xeon in the 7000 dollar Mac Pro is clocked at 2.7GHZ (vs 3.4), with a max turbo freq of 3.5 (vs 3.9). This means that a 12 core CPU is not 2x as fast as a 6 core, and that single threaded operations that are common in the prep stages of a render are slowed down. Contrary to the beliefs of some, there's nothing inherently faster about Xeons. a 6 core i7 CPU is just as fast as a 6 core Xeon CPU.

The Mac Pro can edit video, play games, use the viewport, surf Facebook, do lots of stuff my farm can't, but it isn't a dedicated render machine. Dedicated nodes are becoming an absolute must for a full time 3d artist. If you can't work on a client's project because another client's shots are rendering on your machine, you're doing 3d wrong. YOUR WORKSTATION SHOULD NOT BE THE ONLY THING YOU RENDER ON. This is what irks me about the Mac Pro fetish. It's a fast box, but your workstation being all of your rendering resources is a recipe for disaster.

For those seeking out a better setup than they currently have, i say skip the Mac Pro entirely if you aren't married to OS X. For the cost of the $4000 6 core Mac Pro, you can build an equivalent 6 core Windows box AND a 16 core Helmer render box. That's 4 times the render power for the same price. The 6 core PC workstation above is as follows. I'll avoid prices as they fluctuate from place-to-place

  • Intel 4930k CPU
  • 32GB of 1600 MHZ RAM (Vs Mac Pro's 16)
  • Asus p9x79 Motherboard with support for 64 GB of RAM
  • Thermaltake 700 watt PSU
  • 2x 256GB ADATA SSDs in RAID 0 to neatly match the performance of the Mac Pro's blistering PCIe 'SSD
  • 1 GTX 770 GPU with 2 or 4 GB of VRAM. This card beats the Mac Pro's underclocked FirePros in Cinema 4D's viewport benchmark
  • Cheap case

This build combined with a Helmer computer cluster will make your work much easier than having a single $4000 Mac Pro, as you can offload the work entirely to the farm, or just have it kick in to speed up your renders along with your workstation. If you do build one, i cannot stress airflow enough. I alternated the sides i placed the cooling fans on my nodes, in the name of sexy fashion. What i ended up with were 2 nodes where the fans blew directly over the CPU, and 2 where the fans blow right by the CPU, raising temps about 10C. Do not. Skimp. On. Airflow.

 

I AM CREATOR OF PLANETS, or whatever

Patrick Letourneau

I made Earth, and I've uploaded the 2 gigapixel diffuse texture for you wonderful internet people.

Nothing but water and poisonous wildlife in this shot.

Nothing but water and poisonous wildlife in this shot.

For a recent project, a client required a high res earth model... quickly. I set about sourcing everything needed to make it happen, starting with the best textures i could find. The diffuse here is sourced from a 4 gigapixel image i put together from 6 NASA blue marble tiles (86,000x42,000 or thereabouts) , with 42,000px cloud textures, city lights and spec map. Just how sharp is this thing? Here's Cyprus

click to expand

more.

more.

I recommend 16GB of ram or more for use of this one. Have fun and happy rendering!

power bottom.jpg