A Guy in Tech
Notes from a homelab that runs too hot. Local LLMs, self-hosted
everything, home automation, and the occasional 3D print.
Written while the GPU fans spin up.
Install llama-swap on Ubuntu (the Easy Way)
If you run local models with llama.cpp, you already know the annoyance: llama-server serves one model at a time. Want a different model? Kill it, restart with new flags. llama-swap fixes that. It’s a tiny Go proxy that sits in front of your models, reads the model field from each request, and starts (or swaps…
Why Hermes Agent Is Eating Your Context Window
I have been playing with custom built agents for a while, mostly for learning purposes, that work with small local LLMs (typically 7b to 35b). But recently I decided to give a change to a bigger open-source agent that opens a bigger range of possibilities and doesn’t involve me developing every tool I need, so…
ROCm vs Vulkan on the RX 7900 XTX
I benchmarked both llama.cpp backends on a 24GB RX 7900 XTX. The answer isn’t “pick one” — it depends entirely on whether your model is dense or MoE.