Deploying Kolla-Ansible with External Ceph

When deploying Kolla-Ansible, it's naturally possible to deploy a Ceph cluster simultaneously, but in some cases, administrators may wish to manage Ceph separately from OpenStack, while still wanting to leverage Ceph as a storage backend. Fortunately, Kolla-Ansible can now use previously provided functionality to deploy OpenStack and utilize an external Ceph cluster. config override This article will explain how to use Kolla-Ansible and its config override feature to deploy OpenStack and use an external Ceph cluster.

Table of Contents

Preparation

Since external Ceph cluster usage is required, the first step is to deploy a Ceph cluster. There are multiple deployment methods available for Ceph, and you may refer to previously discussed approaches. Additionally, we need some foundational knowledge and related information about using Kolla-Ansible, which you can review in this article: Ceph-Ansible

In addition, we also need some basic knowledge related to using Kolla-Ansible, which can be referenced in this article:Through Kolla-Ansible and container-based deployment of OpenStack

Ceph Configuration

RBD Pool

OpenStack Nova, Cinder (Cinder Backup), and Glance can all use RBD as a storage backend, so it's necessary to create RBD pools for these three services.

sudo ceph osd pool create images 128
sudo ceph osd pool create vms 128
sudo ceph osd pool create volumes 128

Ceph Keyring Configuration

Ceph uses keyring for authentication, so similarly, we need to create corresponding keyrings for these three services and assign appropriate pool permissions.

sudo ceph auth get-or-create client.glance mon 'allow r' osd 'allow class-read object_prefix rdb_children, allow rwx pool=images' -o /etc/ceph/ceph.client.glance.keyring

sudo ceph auth get-or-create client.cinder mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rx pool=images' -o /etc/ceph/ceph.client.cinder.keyring

sudo ceph auth get-or-create client.cinder-backup mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=backups -o /etc/ceph/ceph.client.cinder-backup.keyring

sudo ceph auth get-or-create client.nova mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=vms, allow rx pool=images' -o /etc/ceph/ceph.client.nova.keyring

Kolla-Ansible Configuration

`globals.yml`

在 globals.yml We must set the option for deploying Ceph via Kolla-Ansible to 'no', while setting the option for using Ceph as a storage backend for Nova, Cinder, and Glance to 'yes'.

enable_ceph: "no"
glance_backend_ceph: "yes"
cinder_backend_ceph: "yes"
nova_backend_ceph: "yes"

Glance Configuration

Configuring Glance to use external Ceph involves three main steps:

在 glance-api.conf Configure using RBD backend
在 /etc/ceph/ceph.conf New Ceph Configuration
Add New /etc/ceph/ceph.client.images.keyring

The first step requires implementing via config override—first, create and add the following content /etc/kolla/config/glance/glance-api.conf to the configuration file.

[glance_store]
stores = rbd
default_store = rbd
rbd_store_pool = images
rbd_store_user = glance
rbd_store_ceph_conf = /etc/ceph/ceph.conf

Next, in step two, similarly to /etc/kolla/config/glance/ceph.conf adding Ceph configuration

[global]
fsid = 88a8ea91-df1d-4f67-b78b-52bb2f04df4d
mon_initial_members = ceph01, ceph02, ceph03
mon_host = 192.168.113.10,192.168.113.11,192.168.113.10
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx

Finally, place the already created keyring into /etc/kolla/config/glance/ceph.client.glance.keyring

sudo cp /etc/ceph/ceph.client.glance.keyring /etc/kolla/config/glance/ceph.client.glance.keyring

Kolla-Ansible will copy all ceph* files under the container's /etc/ceph directory.

Cinder Configuration

Cinder configuration steps are similar to Glance, first establish /etc/kolla/config/cinder/cinder-volume.conf to the configuration file.

[DEFAULT]
enabled_backends=rbd-1

[rbd-1]
rbd_ceph_conf=/etc/ceph/ceph.conf
rbd_user=cinder
backend_host=rbd:volumes
rbd_pool=volumes
volume_backend_name=rbd-1
volume_driver=cinder.volume.drivers.rbd.RBDDriver
rbd_secret_uuid = {{ cinder_rbd_secret_uuid }}

Next, configure Cinder-Backup, create /etc/kolla/config/cinder/cinder-backup.conf and include the following content

[DEFAULT]
backup_ceph_conf=/etc/ceph/ceph.conf
backup_ceph_user=cinder-backup
backup_ceph_chunk_size = 134217728
backup_ceph_pool=backups
backup_driver = cinder.backup.drivers.ceph
backup_ceph_stripe_unit = 0
backup_ceph_stripe_count = 0
restore_discard_excess_bytes = true

Next, similarly to ceph.conf<code> 放到 </code> /etc/kolla/config/cinder/ 下

Finally, you need to copy all keyrings.

sudo cp /etc/ceph/ceph.client.cinder.keyring /etc/kolla/config/cinder/cinder-backup/ceph.client.cinder.keyring
sudo cp /etc/ceph/ceph.client.cinder.keyring /etc/kolla/config/cinder/cinder-volume/ceph.client.cinder.keyring
sudo cp /etc/ceph/ceph.client.cinder-backup.keyring /etc/kolla/config/cinder/cinder-backup/ceph.client.cinder-backup.keyring

Nova Configuration

Nova configuration is also very similar, basically following the same steps

based on the following content, create /etc/kolla/config/nova/nova-compute.conf

[libvirt]
images_rbd_pool=vms
images_type=rbd
images_rbd_ceph_conf=/etc/ceph/ceph.conf
rbd_user=nova

將 ceph.conf<code> 放到 </code> /etc/kolla/config/nova/ 下

Finally, copy the keyring

sudo cp /etc/ceph/ceph.client.nova.keyring /etc/kolla/config/nova/ceph.client.nova.keyring

Real-time Deployment

Regarding the actual deployment commands, you can refer to the documentation for more details Through Kolla-Ansible and container-based deployment of OpenStack

cd kolla-ansible
tools/generate_passwords.py
tools/kolla-ansible -i ansible/inventory/multinode bootstrap-servers
tools/kolla-ansible -i ansible/inventory/multinode prechecks
tools/kolla-ansible -i ansible/inventory/multinode deploy

After the Playbook runs successfully, you can test using each service to ensure they are running properly.

Deploy Kolla-Ansible with External Ceph