I was going off of the customer Q&A listed here: https://www.amazon.ca/gp/product/B07M9X68DS/ref=ppx_yo_dt_b_asin_title_o04_s00?ie=UTF8&psc=1
That's the adapter that was reccomended by the Tesla K80 seller. Are you saying that I can just use an 8 pin CPU male to male connection, directly to my PSU?
This is what I've used before (not this exact one but similar) https://www.amazon.com/COMeap-2-Pack-Graphics-030-0571-000-Adapter/dp/B07M9X68DS I think but can't be certain that just using a ATX power cable marked for CPU instead of GPU might work
Sure.
GPU
So the basic driver was that the K80. Slow but has the best VRAM/money factor. I want to run large models later on. I don't mind if inference takes a 5x times longer, it still will be significantly faster than CPU inference.
K80s sell for about 100-140 USD on Ebay. I got mine for a little less than that because i bought batches of 4, however since I am in Europe I had to pay for shipping and taxes.... meh. Cooling: Forget about all these 3d-printed gizmos trying to emulate a server airflow: super-loud, doesn't work very well, plus it's expensive.. Just tape two 80 / 90 mm fans on with aluminium tape (see link above). Cards do not get hotter than 65° Celsius, which is perfectly fine.
Mainboard/CPU/RAM
Next thing was to identify a mainboard, and there are not many useful ones that support many PCI 3.0 x16 cards. I then found this blog post https://adriangcoder.medium.com/building-a-multi-gpu-deep-learning-machine-on-a-budget-3f3b717d80a9
I got a bundle with 2 x Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz and 128 GB RAM, which runs fine. A CPU with less cores and higher clock would probably be a better fit for the purpose . I think the minimum RAM requirement for this setup would be 64 GB.
Power supply
Power requirements for the thing are
4 x 300 W for the GPUs
+ 500 W for the rest (if you just go for a standard SSD, no spinning drives etc.)
= 1700 W roughly
PSU are expensive, so I went for el-cheapo mining ATX PSUs. 1800 Watt at first (was enough for 3 GPUs) then upgraded to 2000 Watt. The exact model I use is named "sdgr-2000eth".
Build/Frame
Since a 19" rack with server airflow was out of the question, I got inspiration from mining rack designs. Ready made racks are not ideal, because the server mainboards usually do not fit. So I built mine from 20x20 aluminium profiles, i bought pre-cut online (cost about 70 Euro). The dimension are 40 cm x 60 cm.
I mounted the mainboard on a MDF sheet. The GPUs are attached via 40cm fully-connected riser cables, I found on Ebay for about 15 EUR a piece. They card just lie hovering over the mainboard on the second floor of the aluminum frame.
Cables etc.
You need:
- special adapter cables to power the GPUs via PCIe connectors I used these https://www.amazon.de/gp/product/B07M9X68DS/ref=ppx_yo_dt_b_asin_title_o01_s00?ie=UTF8&psc=1
- a dual CPU power adapter to power the dual CPUs on the mainboard
https://www.amazon.de/gp/product/B08V1FR82N/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8&th=1
- fan splitter cables to power the fans taped to the K80s
https://www.amazon.de/gp/product/B07MW86TBV/ref=ppx_yo_dt_b_asin_title_o00_s01?ie=UTF8&psc=1
Software
I had started with Ubuntu 22.04, and got into a weird NVIDIA driver, CUDA, whatever dependency hell.. Just use Ubuntu 20.04 LTS. Most things will work out of the box.
Hi, what power cable should I buy for my 2080 ti? Is this enough? Lost mine cant find it somewhere.