Introduction
When I started self hosting more critical services like Vaultwarden, I realized that I focused on backups a lot but not on the infrastructure itself. What I started to realise was that if for some reason my server goes down, I would have to manually set up everything again, which would take a lot of time.
I was doing everything manually currently, and it was a pain to keep track of all the changes I made. Also, whenever I mess something up or need to reinstall the server, I have to do everything from scratch.
I needed a way to automate the deployment and configuration of my servers and services. That’s when I found Ansible. In this post, I will share with you how I use Ansible to manage my homelab infrastructure configurations.
Here’s a YouTube video ↗️ of me trying to explain some parts of this post in a more interactive, uncut, unedited and way below any decent production grade standards.
And boy oh boy, I said its going to be a short video but it’s 10 minutes long.
What is Ansible?
Let’s first understand what is Ansible what problem does it solves.
Ansible ↗️ is an open-source automation tool that allows you to automate the deployment and configuration of servers and services. It uses a simple and powerful automation language that allows you to define the state of your infrastructure in a declarative way. You write playbooks that describe the desired state of your infrastructure, and Ansible takes care of making sure that your infrastructure matches that state.
Ansible is also a agentless tool, which means that you don’t need to install any agents on your servers to manage them. You just need to have SSH access to the servers, and Ansible will use SSH to connect to the servers and run the tasks defined in the playbooks which is my opinion is a great feature.
The best part is that all the operations are idempoent, which means that you can run the same playbook multiple times, and it will only make the necessary changes to bring your infrastructure to the desired state. This makes it safe to run the playbooks multiple times without worrying about breaking your infrastructure. (In fact I tend to run the paybooks multiple times over the course of a few days to make sure everything is in the desired state).
Now that we have a basic understanding of what Ansible is, let’s see how I use it to manage my homelab infrastructure.
Setting up Ansible
The first step is to install Ansible on your local machine. You can follow the official installation guide ↗️ to install Ansible on your machine. I installed it via Homebrew on a Mac, but you can read more about the installation process on the official website.
Once you have installed Ansible, you need to set up your inventory file. The inventory file is a simple text file that contains a list of all the servers you want to manage with Ansible. You can define groups of servers and assign variables to them in the inventory file.
This is how my inventory file looks like:
[homelab]sukuna ansible_host={{sukuna.ip}} ansible_port={{ssh_port}} ansible_user={{me.username}} ansible_connection=ssh ansible_ssh_private_key_file={{ssh_private_key}}satoru ansible_host={{satoru.ip}} ansible_port={{ssh_port}} ansible_user={{me.username}} ansible_connection=ssh ansible_ssh_private_key_file={{ssh_private_key}}suguru ansible_host={{suguru.ip}} ansible_port={{ssh_port}} ansible_user={{me.username}} ansible_connection=ssh ansible_ssh_private_key_file={{ssh_private_key}}I have three servers in my homelab: sukuna, satoru, and suguru. I have defined the IP address, SSH port, username, and SSH private key file for each server in the inventory file.
Once you have set up your inventory file, you can start writing playbooks to manage your servers.
Quick Pro Tip!
When you are setting up a new server, you need to ensure that your client machine is able to SSH into the server for ansible to be able to manage it. You can use the ssh-copy-id command to copy your SSH key to the server.
Here is how you would do that:
ssh-copy-id -i ~/.ssh/id_lab username@hostnameWriting Playbooks
Playbooks are the heart of Ansible. They are simple YAML files that describe the desired state of your infrastructure. You define a list of tasks that Ansible should perform on your servers to bring them to the desired state.
Here is an example playbook that performs apt update and upgrade on all the servers in my homelab:
---- name: Update and upgrade packages hosts: all become: true tasks: - name: Update apt cache ansible.builtin.apt: update_cache: true
- name: Upgrade all packages ansible.builtin.apt: upgrade: distHere we are telling Ansible to run two tasks on all the servers (hosts: all) as a root user (become: true) to update the apt cache and upgrade all the packages.
Read more about the apt module here ↗️. In fact I would highly recommend you to read the official documentation to understand the various modules available, this certainly helped me craft my playbooks.
You can run the playbook using the ansible-playbook command:
ansible-playbook playbook.yml --ask-become-passThis will run the playbook on all the servers defined in the inventory file. The --ask-become-pass flag is used to prompt for the sudo password if required (which we do in the example above, since we are updating packages which requires sudo privileges).
My Ansible Setup
So in the example, we saw that we defined the tasks directly in our playbook.yml file, which is fine for simple tasks. But as your infrastructure grows, you will have more complex tasks and you will need to organize your playbooks better.
What I do is I organize my playbooks into roles. A role is a collection of tasks, handlers, templates, and variables that are used to configure a specific part of your infrastructure. For example, I have a system role which runs on all the servers to setup the basic system configuration like updating packages, setting up users, setting up ZSH shell, my dotfiles config and more.
Similarly, I have roles for setting up specific services like Vaultwarden, Nginx, AdGuard Home, Linkding etc.
Here is how my directory structure looks like:
.├── README.md├── ansible.cfg├── dev-requirements.txt├── galaxy-requirements.yml├── group_vars│ └── all│ ├── docker.yml│ ├── hosts.yml│ ├── secrets.yml│ ├── services.yml│ └── vars.yml├── inventory│ └── hosts├── roles│ ├── adguardhome│ │ ├── defaults│ │ ├── handlers│ │ ├── tasks│ │ └── templates│ ├── adguardhome_sync│ │ ├── defaults│ │ ├── handlers│ │ ├── tasks│ │ └── templates│ ├── cadvisor│ │ ├── defaults│ │ ├── files│ │ └── tasks│ ├── cloudflared│ │ ├── defaults│ │ ├── files│ │ ├── tasks│ │ └── templates│ ├── docker│ │ ├── defaults│ │ ├── handlers│ │ └── tasks│ ├── dotfiles│ │ ├── defaults│ │ └── tasks│ ├── grafana│ │ ├── defaults│ │ ├── files│ │ └── tasks│ ├── immich│ │ ├── defaults│ │ ├── files│ │ └── tasks│ ├── linkding│ │ ├── defaults│ │ ├── files│ │ └── tasks│ ├── loki│ │ ├── defaults│ │ ├── files│ │ └── tasks│ ├── nginx│ │ ├── defaults│ │ ├── files│ │ ├── tasks│ │ └── templates│ ├── node_exporter│ │ ├── defaults│ │ ├── files│ │ └── tasks│ ├── podman│ │ └── tasks│ ├── prometheus│ │ ├── defaults│ │ ├── files│ │ └── tasks│ ├── promtail│ │ ├── defaults│ │ ├── files│ │ └── tasks│ ├── rclone│ │ ├── defaults│ │ ├── files│ │ └── tasks│ ├── scripts│ │ ├── files│ │ └── tasks│ ├── ssh│ │ ├── defaults│ │ ├── handlers│ │ ├── tasks│ │ └── templates│ ├── syncthing│ │ ├── defaults│ │ └── tasks│ ├── system│ │ ├── defaults│ │ ├── files│ │ ├── tasks│ │ └── templates│ ├── tgpt│ │ ├── defaults│ │ └── tasks│ ├── umami│ │ ├── defaults│ │ ├── files│ │ └── tasks│ └── vaultwarden│ ├── defaults│ ├── files│ └── tasks├── scripts│ ├── deploy.sh│ ├── lint.sh│ └── setup.sh└── run.ymlI have a roles directory where I keep all my roles. Each role has its own directory with the tasks, handlers, templates, and variables defined in separate files.
I also have a group_vars directory where I define variables that are common to all the servers in my homelab. The run.yml file that includes all the playbooks that I want to run. This is the entry point for running my playbooks.
To avoid repeating same commands again and again, I have some helpful scripts which runs the deployment, linting and setup of my ansible environment on CI for validating my playbook changes.
Now for me since I wanted to learn about Ansible while working on my homelab, I started from scratch and built up my roles. But you can also use Ansible Galaxy ↗️ to find pre-built roles that you can use in your playbooks. This is a great way to get started quickly and learn how to write playbooks.
I took references from many of the existing roles and tweaked them to suit my needs. The one that I am most proud of is setting up the Nginx reverse proxy. I have a detailed guide on what my Nginx setup looks like, so check that out. This is what it does:
-
It installs Nginx package.
-
It creates the self-signed SSL certificates using a CA that I have setup and copies them to the correct location.
-
I have a very simplied config of all my services that I want to route via Nginx.
The config looks like this:
nginx_sites_available:- name: sugurudomain: "{{ suguru.domain }}"ip: "{{ suguru.ip }}"records:- subdomain: "{{ adguardhome.subdomain }}"port: "{{ adguardhome.port }}"ssl: "{{ adguardhome.ssl }}"- subdomain: "{{ syncthing.subdomain }}"port: "{{ syncthing.port }}"ssl: "{{ syncthing.ssl }}"Here I just have to define the subdomain, port and if I want to use SSL for the service and the role will take care of the rest.
-
It copies all the relevant snippets to the Nginx config directory.
-
It creates the Nginx config file for each service and enables the site.
So now whenever I introduce a new service in my homelab, I just have to add the details to the nginx_sites_available variable and run the playbook. The Nginx role will take care of setting up the reverse proxy for the new service.
Another than this, I also have a system role which does the basic setup of my servers like updating and upgrading packages, setting up the user, creating SSH keys for the user and uploading them to GitHub as well as adding the SSH keys to the authorized keys file of all the other servers so they can communicate with each other, setup GPG keys and mark them as trusted, setup ZSH shell, setup my dotfiles and more.
Some of the things were pretty straight forward however some were quite painful to setup like GPG keys and SSH keys. But I learned a lot from setting them up and it feels great to have them setup now automatically.
Overall, I feel that using Ansible to manage my homelab infrastructure has been a great learning experience. It has allowed me to automate the deployment and configuration of my servers and services, and it has made it easy to keep track of all the changes I make to my infrastructure.
Whats Next?
I am planning to explore more about Ansible and learn how to write more complex playbooks. I want to learn how I can do testing in a much better way because what I am doing currently is running the playbooks on my homelab servers and checking if everything is working as expected which is not the best way to do it.
And ofcourse improve my roles and make them more robust and reusable. I am also planning to write a few more posts about how I use Ansible to manage my homelab infrastructure, so stay tuned for that.
Finally I would love to give a shoutout to Jake Howard a.k.a RealOrangeOne ↗️. This whole idea of using Ansible was something I got the inspiration from him when I saw his response on one of my Reddit posts and checked out his setup and how he uses Ansible to manage his homelab. So thank you Jake for the inspiration. He also has very good content on his site ↗️ so do check it out.
While learning Ansible was not easy and something that I had decided to explore this year as I mentioned in my 2023 journey blog but I am glad that I did and I am looking forward to learning more about it and keep improving my homelab infrastructure.
Conclusion
In this post, I shared how I use Ansible to manage my homelab infrastructure. I explained what Ansible is, how to set it up, and how to write playbooks to manage your servers and services. I also shared my directory structure and how I organize my playbooks into roles.
I hope this post has given you some insights (and hopefully some inspiration) into how you can use Ansible to manage your homelab infrastructure. If you have any questions or suggestions, feel free to reach out to me on Twitter ↗️ / Reddit ↗️.
See you in another one! 👋