Minikube setup

Last update: 08 January 2021

Introduction

In this post, I'm going through a setup that will let you access services inside minkube using services names, as you would do with any DNS domain, from the host without any complex configuration or any additional tools.

A simpler way to access services on minikube

For this setup, we are going to need:

podman to use as a driver.
dnsmasq for DNS resolution

Note: This setup has been tested only on Linux.

Podman setup

Podman requires passwordless running of sudo. You need to have this line in your sudoers file.

# File: /etc/sudoers
...
user ALL=(ALL) NOPASSWD: /usr/bin/podman
...

DNS setup

We need to change /etc/resolv.conf to add a search list that would expand the name to the format <name>.<namespace>.svc.cluster.local.

# File:/etc/resolv.conf

nameserver 127.0.0.1
search default.svc.cluster.local kafka.svc.cluster.local

To learn more about DNS search list, check the resolv.conf manual page.

Next, we're going to use dnsmasq as DNS resolver, as it allows sending DNS queries to different servers based on the domain.

The DNS server inside minikube has the IP 10.96.0.10, so we need to redirect all the DNS queries for k8s service resolution to that server.

# File: /etc/dnsmasq.conf

conf-file=/usr/share/dnsmasq/trust-anchors.conf

dnssec

resolv-file=/etc/resolv.conf.dnsmasq

# minikube
server=/cluster.local/10.96.0.10

listen-address=127.0.0.1

cache-size=1000

log-queries

Setup the IP address of upstream nameserver

# File: /etc/resolv.conf.dnsmasq
nameserver 1.1.1.1

Restart dnsmsq

$ sudo systemctl restart dnsmasq

Network route setup

By default, minikube creates a bridge network called minikube where the pods run. This network has a subnet 192.168.49.0/24 and minikube container has the IP 192.168.49.2

$ sudo podman network inspect minikube
...

            {
                "bridge": "cni-podman1",
                "hairpinMode": true,
                "ipMasq": true,
                "ipam": {
                    "ranges": [
                        [
                            {
                                "gateway": "192.168.49.1",
                                "subnet": "192.168.49.0/24"
                            }
                        ]
                    ],
                    "routes": [
                        {
                            "dst": "0.0.0.0/0"
                        }
                    ],
                    "type": "host-local"
                },
                "isGateway": true,
                "type": "bridge"
            },

$ sudo podman container inspect minikube | jq '.[0].NetworkSettings.Networks'                                                                                                                           ✘5
{
  "minikube": {
    "EndpointID": "",
    "Gateway": "192.168.49.1",
    "IPAddress": "192.168.49.2",
    "IPPrefixLen": 24,
    "IPv6Gateway": "",
    "GlobalIPv6Address": "",
    "GlobalIPv6PrefixLen": 0,
    "MacAddress": "ee:f9:d8:38:97:dd",
    "NetworkID": "minikube",
    "DriverOpts": null,
    "IPAMConfig": null,
    "Links": null
  }
}

Now we need to add a route, so we can reach services network inside minikube. The k8s services subnet is 10.96.0.0/12

$ sudo ip route add 10.96.0.0/12 via 192.168.49.2

To make this route persistent, we can create a systemd service to run after every boot

# File: /etc/systemd/system/minikube-network.service

[Unit]
Description=Network route to minikube network
After=network.target

[Service]
Type=oneshot
ExecStart=ip route add 10.96.0.0/12 via 192.168.49.2

[Install]
WantedBy=multi-user.target

Let's enable the minikube-network service

$ sudo systemd daemon-reload
$ sudo systemctl enable minikube-network.service

Run minikube

$ minikube start \
  --driver=podman \
  --container-runtime=cri-o \
  --kubernetes-version=v1.21.6 \
  --cpus=4 \
  --memory=12gb \
  --feature-gates="LocalStorageCapacityIsolation=false"

Test

Make sure that you're in minikube context

$ kubectl config current-context
minikube

Let's create a simple service to see if the setup is working properly

$ kubectl apply -f - <<EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello-world-app
  namespace: default
spec:
  selector:
    matchLabels:
      app: hello-world-app
  replicas: 1
  template:
    metadata:
      labels:
        app: hello-world-app
    spec:
      containers:
        - name: hello-world-app
          image: gcr.io/google-samples/hello-app:1.0
          securityContext:
            runAsUser: 1001
          ports:
            - containerPort: 8080

---
apiVersion: v1
kind: Service
metadata:
  name: hello-world-app
spec:
  selector:
    app: hello-world-app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080
EOF

From the host, we should be able to reach the service hello-world-app 🎉

$ curl http://hello-world-app
Hello, world!
Version: 1.0.0
Hostname: hello-world-app-78ffc6ff8b-rn245

PythOps