Last update: 01 January 2022
GitOps is a set of practices where the source of truth for your applications and/or infrastructure is managed by git.
The two major benefits of implementing GitOps in my opinion are:
Any manual changes to your platform are detected and overriden to match the expected state defined in git.
It makes deploying complex platforms very easy. Think about a microservice platform with hundreds of microservices that you want to replicate elsewhere.
In this blog post, we're going to use FluxCD to implement GitOps for application management.
Let's start !
We're going to need:
- Flux CLI that you can download here https://fluxcd.io/docs/cmd/
- k8s cluster. For this demo, we're going to use minikube
We're going through a typical workflow of an app, from building a simple web app to deploying it inside Kubernetes.
I've created a simple web app using Flask that I hosted in this repo https://github.com/pythops/gitops-demo-app
├── app.py
├── Dockerfile
├── kustomize/
├── Makefile
├── Readme.md
└── tests/
To deploy the app in Kubernetes, we are going to use kustomize. The kustomize manifests can be found in the kustomize
folder in the root directory
├── kustomize
│ ├── deployment.yaml
│ ├── kustomization.yaml
│ └── service.yaml
One important thing to notice here is the image tag.
#File: kustomize/deployment.yaml
...
image: pythops/gitops:master-a8158c7c-1640370014 # {"$imagepolicy": "flux-system:image-repo-policy"}
...
Initially I added an arbitrary tag (check the initial commit here)
#File: kustomize/deployment.yaml
...
image: pythops/gitops:latest # {"$imagepolicy": "flux-system:image-repo-policy"}
...
But then let Flux patches the tag whenever a new image is built and pushed to the image registry. Later we will see how can we achieve this.
We are going to use Github actions that will run some tests, build the container image and push it to Dockerhub.
The important thing to Notice here is that the images tag are in this format <branch-name>-<hash>-<timestamp>
This format is important, as we're going to find it later in the Flux configuration.
#File: .github/workflows/ci.yaml
...
- name: Generate build ID
id: prep
run: |
branch=${GITHUB_REF##*/}
sha=${GITHUB_SHA::8}
ts=$(date +%s)
echo "::set-output name=BUILD_ID::${branch}-${sha}-${ts}"
...
- name: Build and publish container image
uses: docker/build-push-action@v2
with:
push: true
context: .
file: ./Dockerfile
tags: |
pythops/gitops:${{ steps.prep.outputs.BUILD_ID }}
Flux needs a place where to store its installation manifests. For that purpose, I'm going to use Github.
First, we're going to generate a token so Flux can connect to Github
$ export GITHUB_TOKEN=<Github token>
Run Flux pre checks
$ flux check --pre
► checking prerequisites
✔ Kubernetes 1.21.6 >=1.19.0-0
✔ prerequisites checks passed
Then deploy Flux
$ flux bootstrap github \
--owner=pythops \
--repository=gitops-demo-flux \
--path=demo \
--components=source-controller,kustomize-controller \
--components-extra=image-reflector-controller,image-automation-controller \
--private=false \
--personal=true
...
► confirming components are healthy
✔ image-automation-controller: deployment ready
✔ image-reflector-controller: deployment ready
✔ kustomize-controller: deployment ready
✔ source-controller: deployment ready
✔ all components are healthy
This command will:
To verify that all the Flux components are running correctly
$ kubectl -n flux-system get pods
NAME READY STATUS RESTARTS AGE
image-automation-controller-7b4fb699f4-xqp98 1/1 Running 0 37s
image-reflector-controller-65fb48496c-gnmmv 1/1 Running 0 37s
kustomize-controller-655b6bb4d5-v5hsb 1/1 Running 0 37s
source-controller-9df84c994-dpxnv 1/1 Running 0 37s
Finally, we need to generate a deploy key so Flux can commit to the gitops-demo-app
repo. The public key should be added to Gihub.
$ ssh-keygen -q -N "" -f ./identity
$ ssh-keyscan github.com > ./known_hosts
$ kubectl -n flux-system create secret generic ssh-credentials \
--from-file=./identity \
--from-file=./identity.pub \
--from-file=./known_hosts
We're going to clone the repo gitops-demo-flux
locally
$ git clone https://github.com/pythops/gitops-demo-flux
This initial structure and files are created when I bootstrapped Flux
└── demo
└── flux-system
├── gotk-components.yaml
├── gotk-sync.yaml
└── kustomization.yaml
To configure the management of the app gitops-web-app
by Flux, we're going to create the following yaml file and commit it to the repository inside the demo
directory.
#File: gitops-demo-app.yaml
apiVersion: source.toolkit.fluxcd.io/v1beta1
kind: GitRepository
metadata:
name: gitops-demo-app
namespace: flux-system
spec:
interval: 30s
url: ssh://git@github.com/pythops/gitops-demo-app
ref:
branch: master
secretRef:
name: ssh-credentials
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
name: gitops-demo-app
namespace: flux-system
spec:
interval: 30s
sourceRef:
kind: GitRepository
name: gitops-demo-app
path: ./kustomize
prune: true
targetNamespace: default
---
apiVersion: image.toolkit.fluxcd.io/v1beta1
kind: ImagePolicy
metadata:
name: image-repo-policy
namespace: flux-system
spec:
imageRepositoryRef:
name: gitops-demo-app
filterTags:
pattern: "^master-[a-f0-9]+-(?P<ts>[0-9]+)"
extract: "$ts"
policy:
numerical:
order: asc
---
apiVersion: image.toolkit.fluxcd.io/v1beta1
kind: ImageRepository
metadata:
name: gitops-demo-app
namespace: flux-system
spec:
image: pythops/gitops
interval: 1m
---
apiVersion: image.toolkit.fluxcd.io/v1beta1
kind: ImageUpdateAutomation
metadata:
name: gitops-demo-app
namespace: flux-system
spec:
interval: 1m
sourceRef:
kind: GitRepository
name: gitops-demo-app
git:
checkout:
ref:
branch: master
commit:
author:
email: fluxcdbot@users.noreply.github.com
name: fluxcdbot
messageTemplate: |
Automated image updated by Flux
[ci skip]
push:
branch: master
update:
path: ./kustomize
strategy: Setters
Let's see what each section is doing
GitRepository
This tells Flux from which git repository to fetch the app.
ref:
branch: master
url: ssh://git@github.com/pythops/gitops-demo-app
Flux will try the fetch the master branch from the repo defined in the url
Kustomization
This indicates that we're using kustomize to deploy the app. That's why we needed the option --components=kustomize-controller
path: ./kustomize
targetNamespace: default
Flux will use the manifests defined in the kustomize
folder and deploy them to the default
namespace.
ImageRepository
This scans the image repository and fetch the tag based on the policy that we define.
image: pythops/gitops
interval: 1m
Flux will scan the image registry pythops/gitops
each minute and fetch the image tags based on the defined ImagePolicy
ImagePolicy
filterTags:
pattern: "^master-[a-f0-9]+-(?P<ts>[0-9]+)"
extract: "$ts"
policy:
numerical:
order: asc
For each image tag, Flux will extract the timestamp field, and filter them by ascending order. This has the result to fetch the latest built image.
ImageUpdateAutomation
This patches the kustomize manifests with the right image tag that was fetched with the ImagePolicy
messageTemplate: |
Automated image updated by Flux
[ci skip]
...
update:
path: ./kustomize
Flux will go through the files in kustomize
folder, and patches the tags where it finds {"$imagepolicy": "flux-system:image-repo-policy"}
with the right tag.
Notice here that we're adding [ci skip]
in the commit message so we don't trigger another build that would trigger another patch commit that would trigger a build and so on ...
Once we push the file to the repository, Flux will start deploying the app.
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
gitops-demo-app-864b7548b6-f4xnn 1/1 Running 0 24s
Let's test the web app:
$ http http://gitops-demo-app
{
"version": "v1"
}
From then, we just need to focus on building a better app and let Flux do the rest.
As you can see, Flux makes implementing GitOps very simple.
In the next post, I'll go through a very intresting tool called Flagger to implement progressive delivery, so stay tuned ;)
Read more ...