Skip to content

DevOps Practices & Workflow

The meta-skills — how you work, not just what you build.

Theory

Documentation-as-code. Your journal, architecture, and troubleshooting docs live in Git alongside the infrastructure. They're versioned, reviewable, and searchable. This is the opposite of a Wiki that rots.

The Evolution of Infrastructure (Reproducibility & IaC). A major goal of DevOps is creating automation that takes a bare metal server to a ready state without human intervention.

  • Phase 1 (Imperative Scripts): Using bash scripts like prep-node.sh. This is a great start, but scripts can be brittle and often fail if run twice (they lack idempotency).
  • Phase 2 (Declarative IaC): Migrating to Configuration Management tools like Ansible. You declare the desired state, and the tool ensures the system matches that state. If a node dies, you don't panic; you PXE boot Debian and run your Ansible playbook.

Idempotency. The property that an operation can be applied multiple times without changing the result beyond the initial application. This is a core tenet of modern DevOps practices and is why tools like Ansible are preferred over raw bash scripts for complex infrastructure.

Incident-driven learning. The pattern:

Problem hit → Journal entry → Troubleshooting doc → Automation

Example: sudo: command not found → documented in journal → added to troubleshooting.md → will become an Ansible task.

Tmux workflow. A terminal multiplexer lets you maintain persistent sessions with multiple windows. Your dev.sh + Makefile pattern gives you a one-command workspace launch (make dev) with SSH sessions to every node.

Obstacles

  • Over-documenting vs. under-documenting. The balance: document decisions and failures, not every keystroke. If you can't explain why a line exists, it needs a comment.
  • Script rot. Scripts break when the environment changes. Revisit prep-node.sh after each rebuild to keep it accurate.

Implementation

  • dev.sh — Tmux workspace setup for Homelab and other projects
  • Makefile — make dev, make stop - create and destroy the homelab session.
  • journal.md — the living log, incident-driven learning, documentation in Homelab project.

Resources