UPDATED Easy Setup Guide with NO DOCKER NEEDED https://youtu.be/_KwVgipVzWY
One server for the ultimate Homelab, multi 3090 GPU LXC and Docker host has turned into a really cool rig. I show you here the complete step by step setup from Proxmox to Nvidia Drivers to Nvidia Container Toolkit and then we setup a LXC to house a docker that handles GPU passthru with bare metal speeds. Then I install Ollama and Openwebui for us and test out the speeds of llama 3.1 vs bare metal. This tutorial gets you completely setup!
UPDATED article (NO DOCKER NEEDED) https://digitalspaceport.com/how-to-setup-an-ai-server-homelab-beginners-guides-ollama-and-openwebui-on-proxmox-lxc/
AI Hardware Writeup https://digitalspaceport.com/homelab-ai-server-rig-tips-tricks-gotchas-and-takeaways
LOCAL AI QUAD 3090 SERVER https://youtu.be/JN4EhaM7vyw
GPU Rack Frame https://geni.us/GPU_Rack_Frame
Supermicro H12ssl-i MOBO (better option vs mz32-ar0) https://geni.us/MBD_H12SSL-I-O
Gigabyte MZ32-AR0 MOBO https://geni.us/mz32-ar0_motherboard
AMD 7V13 (newer, faster vs 7702) https://geni.us/EPYC_7V13_CPU
RTX 3090 24GB GPU (x4) https://geni.us/GPU3090
256GB (8x32GB) DDR4 2400 RAM https://geni.us/256GB_DDR4_RAM
PCIe4 Risers (x4) https://geni.us/PCIe4_Riser_Cable
AMD SP3 Air Cooler (easier vs water cooler) https://geni.us/EPYC_SP3_COOLER
iCUE H170i water cooler https://geni.us/iCUE_H170i_Capellix
(sTRX4 fits SP3 and retention kit comes with the CAPELLIX)
CORSAIR HX1500i PSU https://geni.us/Corsair_HX1500iPSU
4i SFF-8654 to 4i SFF-8654 (x4, not needed for H12SSL-i) https://geni.us/SFF8654_to_SFF8654
ARCTIC MX4 Thermal Paste https://geni.us/Arctic_ThermalPaste
Kritical Thermal GPU Pads https://geni.us/Kritical-Thermal-Pads
HDD Rack Screws for Fans https://geni.us/HDD_RackScrews
Be sure to 👍✅Subscribe✅👍 for more content like this!
Join this channel https://www.youtube.com/@DigitalSpaceport/join
Digital Spaceport Website https://digitalspaceport.com
Patreon https://www.patreon.com/digitalspaceport
Buy Me a Coffee https://buymeacoffee.com/digitalspaceport
Please share this video to help spread the word and drop a comment below with your thoughts or questions. Thanks for watching!
Chapters
0:00 All in One Homelab Ai Server
1:34 Install Proxmox
4:28 Configure Host
5:49 Proxmox GPU Passthru options
8:42 NVIDIA Drivers for HOST
11:27 Docker LXC Install
13:26 Proxmox LXC GPU Docker Setup
20:58 Install Dockge and OpenwebUI
25:20 Setup Ollama and test llama 3.1
34:13 Wrapping Up
*****
As an Amazon Associate I earn from qualifying purchases.
When you click on links to various merchants on this site and make a purchase, this can result in this site earning a commission. Affiliate programs and affiliations include, but are not limited to, the eBay Partner Network.
*****