← Back to blog

Treating a Homelab Like Production: CI/CD as the Center of Gravity

·
homelabdevopsci-cdinfrastructure-as-code

Every service in my homelab — from DNS to media management — goes through the same pipeline a real production system would. Here's the architecture, and why the CI/CD layer, not any one tool, is what actually holds it together.

Why a homelab needs a pipeline

It's easy to run a homelab as a pile of hand-built containers: SSH in, apt install, edit a config file, and move on. That works fine — right up until you can't remember why a service is configured the way it is, an upgrade breaks something silently, or a container needs to move to a different host and half its setup doesn't come with it.

I rebuilt mine around the opposite assumption: nothing gets configured by hand, and nothing counts as "done" until it's gone through the same pipeline every other service uses. Over twenty services — DNS, media automation, monitoring, a password-adjacent credential vault, this very website — now live behind one consistent path from a git commit to a running container. The tools involved (OpenTofu, Ansible, an IPAM system) are unremarkable on their own. What makes the whole thing work is that CI/CD sits at the center of it and nothing skips the line.

Homelab CI/CD architecture diagram

The center: a CI runner, not a tool

The architecture above has a deliberate shape. Every other box — the provisioning tool, the configuration tool, the secrets store, the inventory system — connects through the CI runner, not directly to each other. That's the actual design decision: the pipeline is the integration point, not any individual piece of tooling.

Concretely:

  • OpenTofu declares what a piece of infrastructure should look like — a container's CPU, memory, disk, and network — as code, reviewed the same way application code is.
  • Ansible takes a freshly provisioned container from "exists" to "configured": installs the actual service, lays down its config, starts it.
  • OpenBao holds every credential the pipeline needs — API tokens, database passwords, SSH keys — and hands them out fresh on each run instead of letting them sit on a disk somewhere.
  • NetBox is the fleet's source of truth for what's running and where: every container's name, resources, and IP address get recorded automatically, the same way you'd expect a real CMDB to stay current.

None of those four tools talk to each other directly. They're each just a step the pipeline calls in order.

Anatomy of a deploy

Here's what actually happens when a service changes — whether that's standing up something brand new or updating one that's been running for months:

Anatomy of a deploy pipeline diagram
  1. Push. A change lands in that service's own repository — its infrastructure definition, its configuration, or both.
  2. Plan. The pipeline runs a dry-run of the infrastructure change and surfaces exactly what will happen, before anything happens.
  3. Apply. Once reviewed, the container gets provisioned (or updated) on the cluster — CPU, memory, disk, network, all exactly as declared.
  4. Configure. A configuration run installs and configures the actual application on top of the freshly provisioned container.
  5. Register. The container's identity — name, resources, IP — is written into the fleet's inventory system automatically. If it's not in there, as far as the fleet is concerned, it doesn't exist.

The important part isn't any individual step — it's that this is the only path. There's no "just SSH in and fix it quickly" exception, because the exception is where drift creeps in. If something needs to change, it changes through the pipeline, and the pipeline is what everyone (including future-me) can trust to reflect reality.

Secrets that don't sit around

The one piece that took the most care was secrets. Every one of those pipeline steps needs something — an API token, a database credential, an SSH key — and the easy failure mode is letting those pile up as long-lived values scattered across config files and CI variables.

Instead, every credential is fetched from a central secrets store at the moment a pipeline run actually needs it, used immediately, and discarded when the run ends. Nothing persists on a runner's disk between jobs. A credential leak from a stale checkout or an old CI log is a lot less interesting when there's nothing long-lived sitting there to leak.

What this buys me

The payoff isn't really about any single service running more reliably — it's what the consistency enables:

  • Reproducibility. Any container can be destroyed and rebuilt from its repository, in the same shape, without me remembering the manual steps I took eight months ago.
  • An accurate inventory, without maintaining one by hand. The fleet's IPAM/CMDB reflects reality because the pipeline is what writes to it — nobody has to remember to update a spreadsheet.
  • Confidence to change things. Because every service went through the identical path, a pattern fixed in one place (a bug in how containers get named, say, or how an IP gets assigned) gets fixed for the whole fleet, not just the one service where it was noticed.
  • A genuine staging ground for production practices. The patterns here — plan before apply, secrets fetched just-in-time, everything as code, nothing configured out-of-band — are exactly the practices that matter at any scale. A homelab is just a low-stakes place to actually live by them instead of only talking about them.

The tools will keep changing. The shape won't: CI/CD isn't one piece of this architecture, it's the thing that makes the rest of it an architecture at all.