Photo Source: https://kind.sigs.k8s.io/
Use Case
The current cloud era is dominated heavily by Kubernetes (k8s). While the DevOps team manages App lifecycle (deploy, scale, upgrade, downgrade, etc.), the App Developer needs to build a similar environment locally. So before shipping the App, developer is required to ensure DevOps lifecyle is not impacted by any of the changes s/he makes.
So it is very much necessary for a Developer to have a local k8s cluster for testing. In short, this blog post will demonstrate following flow.
Diagram Source is here.
Assumption
You have go version go1.20.3 installed.
This complete setup is build on Ubuntu 22.04 release. If you are using another OS/platform, then you might need to modify the commands accordingly.
Let Start...
Prerequisite Setup
Clone the repo
[~]# git clone https://github.com/simplyatul/godevsetup.git
[~]# cd godevsetup
Create a local docker registry. We will use this registry to push the App's docker image.
[~/godevsetup]# ./local-docker-registry.sh
974282a4a1c561017a93cc45caefcc23f9a2ed1924ec8e949f76850e4818ad89
[~/godevsetup]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
974282a4a1c5 registry:latest "/entrypoint.sh /etc…" 4 seconds ago Up 3 seconds 0.0.0.0:5000->5000/tcp, :::5000->5000/tcp local-docker-registry
Now using KinD, create local k8s cluster with a master node and two worker nodes.
[~/godevsetup]# kind create cluster --config ./kind-local-k8s-cluster.yaml --name local-dev
Creating cluster "local-dev" ...
✓ Ensuring node image (kindest/node:v1.26.3) 🖼
✓ Preparing nodes 📦 📦 📦
✓ Writing configuration 📜
✓ Starting control-plane 🕹️
✓ Installing CNI 🔌
✓ Installing StorageClass 💾
✓ Joining worker nodes 🚜
Set kubectl context to "kind-local-dev"
You can now use your cluster with:
kubectl cluster-info --context kind-local-dev
Have a nice day! 👋
[~/godevsetup]# docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
82849b786a50 kindest/node:v1.26.3 "/usr/local/bin/entr…" 56 seconds ago Up 53 seconds local-dev-worker2
0d1153c315bf kindest/node:v1.26.3 "/usr/local/bin/entr…" 56 seconds ago Up 53 seconds local-dev-worker
d16cbeb2102c kindest/node:v1.26.3 "/usr/local/bin/entr…" 56 seconds ago Up 53 seconds 0.0.0.0:80->80/tcp, 127.0.0.1:43821->6443/tcp local-dev-control-plane
974282a4a1c5 registry:latest "/entrypoint.sh /etc…" 10 minutes ago Up 10 minutes 0.0.0.0:5000->5000/tcp, :::5000->5000/tcp local-docker-registry
See kind-local-k8s-cluster.yaml. We have made the KinD cluster ingress ready using node-labels: "ingress-ready=true".
Install the Nginix Controller. This is required so that our service can be reachable from outside the cluster.
[~/godevsetup]# kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/kind/deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
Connect the kind network with the local docker registry
[~/godevsetup]# docker network connect kind local-docker-registry
Step 1 - App Development
I have written a simple App in Go language. You can do this in any language of your choice. The App exposes a ReST API and returns the hostname. Check the GitHub repo. You can run the App locally using go run main.go and then hit the following command from another terminal.
$ curl localhost:8080
Hostname: atul-lap
Once you complete the coding, commit it. Optionally, you can push it to the remote repo as well.
Step 2 - Conatainerize
Build the docker image
[~/godevsetup]# docker build -t localhost:5000/godevsetup:latest .
Run it locally and check once
[~/godevsetup]# docker run --rm --name godevsetup -p 8080:8080 localhost:5000/godevsetup:latest
# from another terminal
[~/godevsetup]# curl localhost:8080
Hostname: 064a6217790f
Step 3 - Push Image to Local Registry
Push the image to the local registry
[~/godevsetup]# docker push localhost:5000/godevsetup:latest
[~/godevsetup]# curl -X GET localhost:5000/v2/godevsetup/tags/list {"name":"godevsetup","tags":["latest"]}
Step 4 - Deploy on Local K8s Cluster
Check the containers running on k8s worker nodes before we deploy the App.
[~/godevsetup]# docker exec local-dev-worker crictl ps
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
62e79b0d067d7 a329ae3c2c52f About an hour ago Running kindnet-cni 0 f4171cb3fcca7 kindnet-q9gcc
fb97952a78295 eb3079d47a23a About an hour ago Running kube-proxy 0 f586b0a23dd88 kube-proxy-64ng7
[~/godevsetup]# docker exec local-dev-worker2 crictl ps
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
a62234bb3add4 a329ae3c2c52f About an hour ago Running kindnet-cni 0 6bbf88872fe5b kindnet-jv67s
9f0f8a3a1eaa4 eb3079d47a23a About an hour ago Running kube-proxy 0 4aa4b67da1ff7 kube-proxy-fptm8
I have created the service.yaml which creates Deployment and Service k8s kinds for our App. Apply it using kubectl to deploy the App.
[~/godevsetup]# kubectl apply -f service.yaml
deployment.apps/godevsetup created
service/godevsetup created
ingress.networking.k8s.io/godevsetup-ingress created
[~/godevsetup]# kubectl get all
NAME READY STATUS RESTARTS AGE
pod/godevsetup-575fcd74d5-9k6d9 1/1 Running 0 17s
pod/godevsetup-575fcd74d5-f6xz6 1/1 Running 0 17s
pod/godevsetup-575fcd74d5-x7dtt 1/1 Running 0 17s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/godevsetup ClusterIP 10.96.73.129 <none> 8080/TCP 17s
service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 74m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/godevsetup 3/3 3 3 17s
NAME DESIRED CURRENT READY AGE
replicaset.apps/godevsetup-575fcd74d5 3 3 3 17s
You can also observe the App pods are created inside the worker nodes
[~/godevsetup]# docker exec local-dev-worker crictl ps
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
778beb3eee995 7df119c4f1283 28 minutes ago Running godevsetup 0 3e1d374e07eb5 godevsetup-575fcd74d5-9k6d9
2aaaaa94f5ec5 7df119c4f1283 28 minutes ago Running godevsetup 0 f43aab8a1bb08 godevsetup-575fcd74d5-x7dtt
62e79b0d067d7 a329ae3c2c52f 2 hours ago Running kindnet-cni 0 f4171cb3fcca7 kindnet-q9gcc
fb97952a78295 eb3079d47a23a 2 hours ago Running kube-proxy 0 f586b0a23dd88 kube-proxy-64ng7
[~/godevsetup]# docker exec local-dev-worker2 crictl ps
CONTAINER IMAGE CREATED STATE NAME ATTEMPT POD ID POD
5ff7d0313821e 7df119c4f1283 28 minutes ago Running godevsetup 0 50dcd5384b9ce godevsetup-575fcd74d5-f6xz6
a62234bb3add4 a329ae3c2c52f 2 hours ago Running kindnet-cni 0 6bbf88872fe5b kindnet-jv67s
9f0f8a3a1eaa4 eb3079d47a23a 2 hours ago Running kube-proxy 0 4aa4b67da1ff7 kube-proxy-fptm8
Step 5 - Verify
Finally, verify the App's ReST endpoint.
[~/godevsetup]# curl localhost
Hostname: godevsetup-575fcd74d5-f6xz6
[~/godevsetup]# curl localhost
Hostname: godevsetup-575fcd74d5-9k6d9
[~/godevsetup]# curl localhost
Hostname: godevsetup-575fcd74d5-x7dtt
Step 6 - Enhance/Bug Fix the App
First, remove the App from the k8s cluster
[~/godevsetup]# kubectl delete -f service.yaml
deployment.apps "godevsetup" deleted
service "godevsetup" deleted
ingress.networking.k8s.io "godevsetup-ingress" deleted
And now repeat Steps 1-5.
Sign Off...
We are done. This blog covers how to set up a local k8s cluster using KinD. And how can we leverage it for our App deployment and testing.
Thank you for reading the post. Please comment on how you find this. Connect me on Twitter for getting updates on future blog posts.