Running OpenStack at home

I recently found a 50% off deal on rack servers at Lenovo, and picked one up for a pretty good price. It’s 100% overkill, but I’ve always wanted my own server just because.

Since it’s complete overkill, I decided I needed to keep that train rolling and set it up as an Openstack all in one.

The Strategy

In the lead up to the server’s delivery, I looked into the different ways to set up the cloud. The first stop was microstack, which is a Canonical supported effort. Configuration was trivial, and I was able to quickly get it up and running in an Ubuntu server VM. After exploring a little bit, I decided that I wanted to experiment with Trove, Swift and Magnum, which aren’t offered in the package.

The next thing I looked at was devstack. Again, simple to get up and running. It worked well enough, but felt like I was locked in to a single way of running it.

The method I finally settled on was openstack-ansible, and I followed their AIO quickstart docs to get started. I practiced this a few times before the server arrived, and managed to come up with some runsheets and notes on what needed to happen to install each service.

Rough Edges

I found a couple of rough edges trying to install everything in one go, particularly Trove. Trove would often bring down my install process if I configured it out of the gate. As a result I got used to stepping through and installing services one at a time after the basic AIO scenario ran, which boiled down to:

1. Copy YML AIO service config into /etc as per instructions
2. Run the containers-deploy playbook
3. Run the relevant os-*-install playbook
4. I don’t know if this was necessary, but running the os-horizon-install playbook. This seemed to get the Horizon UI to show the newly installed service, but it may have just been a case of giving it time to update

The other part of the Trove setup that needed to be done was the manual configuration of the management network. Otherwise, despite being a little slow, this worked pretty well when I deployed on the metal, with a hiccup in the 27.0.1 tag forcing me to rollback to 27.0.0 and starting over completely. Hey, it happens.

Still having issues installing heat and therefore magnum, with the install throwing an error saying it can’t create the service user (despite the fact I can see the newly created service account in horizon). Will have to debug that, probably a problem with my configuration rather than an inherent problem in the Ansible playbooks.

Scripts and Configs I used

Base AIO install

This script pulls down the openstack-ansible repo, switches to the 27.0.0 tag, and proceeds to run each step

#!/bin/bash
### RUN AS ROOT ###

# Grab Ansible playbooks
git clone https://opendev.org/openstack/openstack-ansible \
    /opt/openstack-ansible
cd /opt/openstack-ansible

git checkout 27.0.0
./scripts/bootstrap-ansible.sh

# Copy configs
mkdir -p /etc/openstack_deploy/conf.d

# You could do this, but swift has a habit of failing. It's better to just go one by one
# cp etc/openstack_deploy/conf.d/{aodh,barbican,ceilometer,cinder,designate,glance,horizon,keystone,neutron,nova,octavia,placement,swift,trove,zun}.yml.aio /etc/openstack_deploy/conf.d/
# for f in $(ls -1 /etc/openstack_deploy/conf.d/*.aio); do mv -v ${f} ${f%.*}; done

# SCENARIO='aio_lxc_lxb_aodh_barbican_ceilometer_cinder_designate_glance_horizon_keystone_neutron_nova_octavia_placement_swift_trove_zun' ./scripts/bootstrap-aio.sh

./scripts/bootstrap-aio.sh

cd ./playbooks
openstack-ansible setup-hosts.yml
openstack-ansible setup-infrastructure.yml
openstack-ansible setup-openstack.yml

Trove network management

I needed to add an entry to provider_networks in /etc/openstack_user_config.yml in order to give Trove its own provider network. I pulled the following from the default Trove AIO config example in the repo.

cidr_networks:
    # ...
    dbaas: 172.29.252.0/22

- network:
   container_bridge: br-dbaas
   container_type: veth
   container_interface: eth13
   host_bind_override: eth13
   ip_from_q: dbaas
   type: flat
   net_name: dbaas-mgmt
   group_binds:
     - neutron_linuxbridge_agent
     - rabbitmq

used_ips:
    # ...
  - "172.29.252.1,172.29.252.10"
  - "172.29.252.50,172.29.255.255"

After updating this, I ran the os-neutron-install playbook to update the network, and was able to successfully run the os-trove-install playbook.

What I learned

I got a nice little Ansible refresher and have started setting up private repos in my github to configure my desktop whenever I reinstall Arch after the inevitable explosion. Combined with my NAS, it’s now absolutely no problem when my computers crash, so I’m pretty happy with that for now.

Next steps

I’m currently in the process of creating some terraform templates to add things like floating IP ranges and compute flavours and OS images. After that, I’ll be adding the various project infrastructure and probably adding ansible playbooks to configure the provisioned virtual servers. Going to be a fun little long term project.