Last update: 10 April 2022
Even though there are tools out there that automate the creation of k8s cluster, they aren't that flexible when it comes to customization. So if you are curious about k8s and want to know how to create your own cluster like a pro, you're in the right place ! This is part 1 of a series of posts where I'll be explaining how to deploy a highly available cluster.
At the end of this post, you'll be able to deploy a highly available k8s masters as shown below.
So let's get started !
Before starting you need to have these tools installed in your workstation.
$ pip install --user -U ansible kubernetes $ export PATH=$PAHT:$HOME/.local/bin $ ansible-galaxy collection install kubernetes.core
Once you install these tools, the next step is to clone the Ansible playbooks from GitHub
$ git clone https://github.com/pythops/k8s_the_hard_way $ cd k8s_the_hard_way
You may need to update the VirtulBox network config file to allow the
# File: /etc/vbox/networks.conf * 10.0.0.0/8
Let's create the VMs
$ vagrant up master1 master2 master3 lb
You should have the masters VMs up and running
$ vagrant status Current machine states: lb running (virtualbox) master1 running (virtualbox) master2 running (virtualbox) master3 running (virtualbox)
Etcd is a key-value store that is used by k8s to store all the cluster data. We're gonna deploy a highly available cluster with 3 nodes. With this setup we can afford to lose up to 1 node and still having a working cluster.
We're gonna create a set of certificates to secure all the communication.
$ ansible-playbook k8s-pki.yaml -t etcd
This will create and store all the certificates inside the directory
pki ├── etcd │ ├── etcd-peer-ca.crt │ ├── etcd-peer-ca.csr │ . . . │ └── k8s-etcd-3-server.key
Deploy Etcd cluster
$ ansible-playbook etcd.yaml
$ vagrant ssh master3 -c "etcdctl --cert /etc/etcd/k8s-etcd-3-server.crt --key /etc/etcd/k8s-etcd-3-server.key --cacert /etc/etcd/etcd-server-ca.crt --endpoints=10.0.0.101:2379,10.0.0.102:2379,10.0.0.103:2379 endpoint health" 10.0.0.101:2379 is healthy: successfully committed proposal: took = 52.012358ms 10.0.0.103:2379 is healthy: successfully committed proposal: took = 54.985654ms 10.0.0.102:2379 is healthy: successfully committed proposal: took = 49.513059ms
Now that we have an etcd cluster up and running, let's move to the next step and deploy the k8s masters.
A k8s master has 4 components:
For this setup we're gonna deploy only
We're gonna create a set of certificates to secure the communications within the master components and with the outside world as well.
For more infotmation about the k8s PKI, check the k8s official doc here 👉 https://kubernetes.io/docs/setup/best-practices/certificates/
$ ansible-playbook k8s-pki.yaml -t master
this will create all the certificates in
├── k8s │ ├── k8s-ca.crt │ ├── k8s-ca.key │ .... │ └── service-account.key
kubeconfig files are used to access the cluster. They are used by any component that needs to talk to the apiserver.
$ ansible-playbook kubeconfig.yaml -t master
This will create all the kubeconfig files in the
kubeconfig ├── kube-controller-manager.kubeconfig └── kube-scheduler.kubeconfig
To save time and bandwidth, we're gonna download the binaries to the host machine and then copy them to the VMs
$ ansible-playbook k8s-download.yaml
This will download the k8s master components to a
bin ├── kube-apiserver ├── kube-controller-manager └── kube-scheduler
Deploy master components
$ ansible-playbook k8s-master.yaml
ℹ️ One important option in apiserver options is
--encryption-provider-config which encrypts all the data before storing it in the etcd. the encryption key is defined in
$ ansible-playbook k8s-pki.yaml -t admin
This will create the admin certificate and private-key in
pki └── user-accounts ├── admin.crt ├── admin.csr └── admin.key
We need to generate the kubeconfig file as well
$ ansible-playbook kubeconfig.yaml -t admin
kubeconfig ├── admin.kubeconfig
We're gonna deploy a TCP loadbalancer using Nginx
$ ansible-playbook loadbalancer.yaml
Let's verify now that our setup is working
$ kubectl --kubeconfig kubeconfig/admin.kubeconfig cluster-info Kubernetes control plane is running at https://10.0.0.100:6443
So far we have deployed a highly available k8s masters. Let's move to part 2 to deploy k8s nodes.
Read more ...