Kubernetes offers multiple deployment options, and among various tools, kops stands out for its ease of use and high integration. This article will provide an in-depth look at the kops tool, guiding readers through practical steps to rapidly set up a Kubernetes cluster in an OpenStack environment.
Table of Contents
What is kops?
kops, short for Kubernetes Operations, as its name suggests, is a specialized tool designed for managing and operating Kubernetes clusters. Key features of kops include:
- Supports creation, maintenance, upgrade, and deletion of Kubernetes clusters
- Supports multiple cloud platforms, such as AWS, OpenStack/DigitalOcean (Beta), Azure/GCP (Alpha)
- Can generate Terraform templates from kops cluster definitions
- Manage Kubernetes cluster plugins
- Automated deployment of Kubernetes cluster
kops name
Storage
STATE_STORE
STATE STORE defines the location where kops stores its data; the data stored by kops includes cluster specs, instance groups, and SSH keys, among others.
API
Cluster Spec
Cluster spec defines the essential specifications required by the cluster itself, such as DNS, Load Balancer, and etcd cluster information.
Instance Group
Instance Group defines the associated configuration data for actual worker nodes, master nodes, and etcd nodes; for example, in OpenStack, this would include the flavor and image used by the virtual machine.
Cloud
The Cloud
The Cloud defines a unified Go interface for different cloud platforms, enabling kops to support multiple cloud environments by abstracting operations through this interface.
Tasks
A task refers to a single API call performed on a cloud environment.
Model
Model translates the previously defined cluster spec into actual tasks that correspond to real-world operations.
For example, if we define a load balancer in the cluster spec, the loadbalancer model will generate the necessary API calls to create the required load balancer in the actual cloud environment.
Install kops
Prerequisites
Before installing kops, you must install kubectl. Installation instructions can be found here.Official Documentation
Install via Homebrew
brew update && brew install kopsInstall via official release instructions
Linux
curl -Lo kops https://github.com/kubernetes/kops/releases/download/$(curl -s https://api.github.com/repos/kubernetes/kops/releases/latest | grep tag_name | cut -d '"' -f 4)/kops-linux-amd64
chmod +x kops
sudo mv kops /usr/local/bin/kopsMac
curl -Lo kops https://github.com/kubernetes/kops/releases/download/$(curl -s https://api.github.com/repos/kubernetes/kops/releases/latest | grep tag_name | cut -d '"' -f 4)/kops-darwin-amd64
chmod +x kops
sudo mv kops /usr/local/bin/kopsSelf-Service
Please ensure your environment has Go installed and that GOPATH and GOBIN are properly set up before building kops from source.
git clone git@github.com:kubernetes/kops.git
cd kops
makeLab: Set up Kubernetes Cluster
Download OpenStack Credentials
Note
Please contact the admin to enable load balancer usage with appropriate permissions.
Navigate to OpenStack Dashboard → Identity / Application Credentials, and select 'Create New Application Credential' in the upper-right corner.
Using CNTUG Infra Lab as an example, it is connected to https://openstack.cloudnative.tw/identity/application_credentials/
After the setup is complete, click Download openrc file, which will download a Shell Script file.
After the download is complete, you will need to input the RC file information into the shell environment variable
source openrc.sh
export OS_DOMAIN_NAME=DefaultSet kops STATE_STORE
kops will need to address the issue of storing data, which is the STATE_STORE mentioned above
Since Infra Labs provides Swift service, we can directly use it
export KOPS_STATE_STORE=swift://kopsCreate Kubernetes Cluster Template
Next, to use kops to create a cluster, it's very simple—just one command is needed:
Note
Please replace the command with
跟
kops create cluster \
--cloud openstack \
--name <cluster_name>.k8s.local \
--state ${KOPS_STATE_STORE} \
--zones nova \
--network-cidr 10.0.0.0/24 \
--image Ubuntu-22.04 \
--master-count=3 \
--node-count=1 \
--node-size m1.small \
--master-size m1.small \
--etcd-storage-type NVMe \
--api-loadbalancer-type public\
--topology private \
--bastion \
--ssh-public-key <ssh_key_path> \
--networking calico \
--os-ext-net public \
--os-dns-servers=8.8.8.8,8.8.4.4 \
--os-octavia=true \
--os-octavia-provider=ovnIn this lab, we will create a cluster with a total of 4 nodes, including 3 master/etcd nodes and 1 worker node.
The architecture diagram is as follows:
kops create cluster It will create cluster spec, instance group, and other data. Next, we need to use kops update cluster to actually create the Kubernetes cluster
kops update cluster --name <cluster_name> --yes --adminAt this point, kops will begin calling the OpenStack API to create the VMs, network, storage, load balancer, and other resources required for the Kubernetes cluster. After completion, please wait approximately 5–10 minutes for the entire cluster to boot up.
At this point, the new Kubernetes cluster's configuration will be embedded in ~/.kube/config 下
Note
If a 409 error occurs during update, and the load balancer in OpenStack Dashboard remains in PENDING_UPDATE status, please contact the administrator
Validate Kubernetes Cluster
Finally, we can verify that the Kubernetes cluster has been successfully deployed.
kops validate cluster --wait 5mWe will obtain the following output
Validating cluster igene.k8s.local
INSTANCE GROUPS
NAME ROLE MACHINETYPE MIN MAX SUBNETS
bastions Bastion m1.tiny 1 1 nova
master-nova-1 Master m1.small 1 1 nova
master-nova-2 Master m1.small 1 1 nova
master-nova-3 Master m1.small 1 1 nova
nodes-nova Node m1.small 1 1 nova
NODE STATUS
NAME ROLE READY
master-nova-1-nvhtbc master True
master-nova-2-jgnhbo master True
master-nova-3-vllcwf master True
nodes-nova-yxorvl node True
Your cluster igene.k8s.local is readyAt this point, we can begin to use kubectl to perform operations on the cluster.
The cluster provisioned using kops has already been set up cloud-provider-openstack and integrated with features such as Cinder CSI and Octavia Load Balancer.
kubectl get csidrivers.storage.k8s.io
NAME ATTACHREQUIRED PODINFOONMOUNT STORAGECAPACITY TOKENREQUESTS REQUIRESREPUBLISH MODES AGE
cinder.csi.openstack.org true true false <unset> false Persistent,Ephemeral 13hDelete Kubernetes Cluster
kops has tracked all resources created on OpenStack, so deleting the Kubernetes cluster and releasing resources is extremely simple—just one command:
kops delete cluster <cluster_name> --yeskops will delete all resources created on OpenStack.
Summary
In this lab, we have learned how to use kops to create a complete Kubernetes cluster on OpenStack. The cluster built by kops also includes cloud-provider-openstack related features, allowing users to directly use them within the cluster. persistant volume 和 load balancer related features.
Future labs related to Kubernetes will also guide you to use this method to set up a Kubernetes cluster.
Appendix
Add Instance with Different Flavor
kops can add instances with different flavors and quantities via instance groups.
Instance groupFor example:：
apiVersion: kops.k8s.io/v1alpha2
kind: InstanceGroup
metadata:
labels:
kops.k8s.io/cluster: <cluster_name>.k8s.local
name: nodes-nova-arm
spec:
image: Ubuntu-22.04-aarch64
machineType: a1.large
maxSize: 1
minSize: 1
role: Node
subnets:
- novaCreate an instance group via the following commands and update the cluster to establish new instances
cat ig.yaml | kops create -f -
kops update cluster --name <cluster_name>.k8s.local --yes --adminThe new instance will be created and then added to the cluster.
This is the kind of quality content I love to see.