Lorenzo Soligo GitOps 101 - da dove veniamo, dove stiamo andando
Kubernetes is a (Game) Loop!
while True:
current_state = get_current_state()
desidered_state = get_desidered_state()
reconcile(current_state, desidered_state)
OK, this is a super high level description to the thing but can help to set the context about what Kubernetes is
Somebody calls the simple loop above a reconciliation loop, a sort of pattern.
This enables you to be "declarative", just state what you need from the system, not how reach a "desidered" condition.
And enable Kubernetes to be "Intent based": match the resources with users "desire".
New ideas is Software Engineering and Information Technology.
New ways of working.
Internet, hardware disaggregation, virtualization.
Decouple and isolate business logic from "infrastructural" stuff.
Use of "Dependency Inversion" for enabling pluggability.
A thing can have multiple contexts (bounded contexts), or multiple point of view.
For example a "Customer" in a CRM can have some attributes relevant for the "Sales" department, but not so much for "Support" department.
A complex application can have many business logic.
Break the monolith to enable horizontal scaling.
"Sales" and "Support" can be separated and talk via an interface.
"Sales" and "Support" can be developed by different teams.
The application deployment is part of the application and should be easy. The Twelve-Factor App.
Migration from in memory interfaces to Web / REST Application Programming Interface.
Concept of "Resource".
Concept of "Single Source of Truth".
Too much to say. I'll stop here. Just a mention for Team Topologies (2019). Build teams around projects and not in functional silos.
Infrastructure as code.
Automation.
Cloud Computing.
Distributed Systems is hard. Many things can go wrong, see Designing Data-Intensive Applications chapter 8.
Distributed System are built upon cheap but unreliable components.
Write a platform where lessons learned while implementing distributed system become code that can help to operate and monitor a system.
Possibility to plug-in or extend the resources under control.
What is the techie stuff that made Kubernetes even possible?
Operative systems capabilities to isolate Resources.
Namespace | Isolates |
---|---|
Cgroup | Cgroup root directory |
IPC | System V IPC, POSIX message queues |
Network | Network devices, stacks, prots, etc. |
Mount | Mount points |
PID | Process IDs |
Time | Boot and monotonic clocks |
User | User and Group IDs |
UTS | Hostname and NIS domain name |
Docker automated all this and added a specific way to build images and handle process communication, but we can build our "containers" just with Linux.
Tools like Podman or Docker make running containers easy for everyone by abstracting the different Linux technologies used under the hood from the user.
Anyway these tools are doing more than what we showed in the previous demo.
Linux Networking capabilities:
This time I will draw something with https://excalidraw.com/
For reference look at these videos:
That our v-net-0 Linux Bridge it's docker0?
Let us see another sketch on https://excalidraw.com/
For Container to Container communication, Docker can expose a service to the host, or to other containers by mapping a port. But we have to be explicit and tell to do so:
docker run --name nginx -p 8080:80 ngnix
The inside port 80 is exposed as 8080 to the host, and to other containers
Docker has other ways to handle the host network.
In custom bridge mode, more container can share the same netns, this means they can communicate with localhost:port, so you can have a web API app container and a database container on a custom bridge, and you can expose just the API port (security is enhanced).
docker network create mynetwork
docker run -it --rm --name=container-a --network=mynetwork busybox /bin/sh
docker run -it --rm --name=container-b --network=mynetwork busybox /bin/sh
This is a special case of custom networking where another container joins the network of another container, same here, the containers are on the same netns. This is similar to how a Pod works on Kubernetes.
docker run -it --rm --name=container-a busybox /bin/sh
docker run -it --rm --name=container-b --network=container:container-a busybox /bin/sh
docker run --net=none --name busybox busybox ip a
Cross-host networking usually uses an overlay network, which builds a mesh between hosts and employs a large block of IP addresses within that mesh. The overlay network functionality built into Docker is called Swarm. When you connect a host to a swarm, the Docker engine on each host handles communication and routing between the hosts.
OK, I said Docker donated runc to the Open Container Initiative, and donated the runtime and image specification, then what happened?
The truth is that Kubernetes is made of multiple projects! Under the Linux Foundation started a specification work that produced
Did I spoke about pluggability at the beginning of this thing? Here is the result, we have two CRI reference implementations:
Both of them call runc (or some sort of fork of it) to launch containers.
But why the CNI? Because people realized that container networking implemented in: the Linux Kernel Network Namespaces, Docker, rkt, Mesos, Kubernetes and bla bla, everybody do the same things:
All the stuff needed for building the Kubernetes Network Model.
So they: Linux Foundation and Kubernetes, created this application interface, but as I told ya this enabled pluggabilty, and so we have tons of CNI compliant network plugins, using different network protocols in some cases.
A logical network built on top of a physical one. In practice is implemented by encapsulate an "inner" IP into an "outer" IP. A trivial overlay is IP-over-IP overlay. Real life examples: VxLAN, IPSec, VPN.
Bret Fisher is better than me, we seat on Giants shoulders!
Since it's an aggregate of open source projects it's not simple to install it. Sometimes one of these projects break each other.
This is true for Vanilla's Kubernetes. It's very much the same as Linux, but, as Linux you can choose a "Kubernetes Distribution" from a Linux provider that developed "installers" for you to simplify the installation, or look at managed versions From a Cloud provider like EKS, GCP, AKS.
At the time of this writing.
https://github.com/keobox/kubernetes/tree/fix-containerd-issue
cd vagrant-provisioning
vagrant up
Note: I started browsing this Kubernetes Books just after the first meeting and seems nice 'cause is following the same "white box" approach I'm following. The nice thing that the Cluster built with this book have, althought is minimal, is an HA cluster:
Ours Cluster is not a HA Cluster.
cd vagrant-provisioning
vagrant up
vagrant ssh kmaster
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
sudo systemctl status containerd
sudo systemctl status kubelet
kubectl get nodes
To use kubectl on the host machine, hence outside the cluster, you can take it from the cluster.
mkdir -p ~/.kube
scp -P 2222 -i .vagrant/machines/kmaster/virtualbox/private_key vagrant@127.0.0.1:.kube/config ~/.kube/playground-config
export KUBECONFIG=~/.kube/playground-config
kubectl get nodes
kubecrl get nodes -o wide
Is possible to add an INSECURE Kubernetes Web console in just one line:
kubectl apply -f https://k8smastery.com/insecure-dashboard.yaml
This installs a NodePort service (more on this below), and is possible to connect like this:
kubectl get svc dashboard
Note down the node port and connect with a browser to http://172.16.16.100:3XXXXX and press "skip".
kubectl run web --image=nginx
kubectl get pods -o wide
kubectl logs nginx
kubectl delete pod web
The Kubernetes engineers recognized that a Cluster may have many use case, for this reason they created some abstractios to address this use cases, so there are some kinds of workloads.
The ReplicaSet maintains the desired number of copies of a Pod running within the cluster. If a Pod or the host on which it’s running fails, Kubernetes launches a replacement.
A Deployment manages a ReplicaSet.
kubectl create deployment web --image=nginx --replicas=3
kubectl get all -n default
What happened? A Walkthrought! The "Loop" in action!
kubectl get pods -o wide
kubectl delete pod web-xxx-yyy && kubectl get pods -w
kubectl scale deployment web --replicas=4 && kubectl get pods -w
kubectl scale deployment web --replicas=2 && kubectl get pods -w
kubectl get deployment web -o yaml > web.yaml
The ReplicaSet managed by the Deployment has 2 important parameters in the spec for a RollingUpdate: maxSurge which is the percentage of the pods launched on a new version and maxUnavailable which is the percentage of pods deleted in the old version.
Suppose to do an update to nginx version 1.23 but that does not exists eh eh.
Check the image name in a pod
kubectl describe pod web-xxx-yyy
try the update
kubectl scale deployment web --replicas=10
kubectl set image deployments/web nginx==nginx:1.23 && kubectl get replicasets -w
kubectl rollout status deploy web
kubectl rollout undo deploy web
A DaemonSet runs one copy of the Pod on each node in the Kubernetes cluster. This workload model provides the flexibility to run daemon processes such as log management, monitoring, storage providers, or network providers that handle Pod networking for the cluster.
Calico, our CNI implementation is implemented as a DaemonSet. This means we can see 1 daemon per node, hence 3 Pods.
kubectl get daemonset -n kube-system calico-node
kubectl get pods -n kube-system -o wide | grep calico-node
A StatefulSet controller ensures that the Pods it manages have durable storage and persistent identity. StatefulSets are appropriate for situations where Pods have a similar definition but need a unique identity, ordered deployment and scaling, and storage that persists across Pods rescheduling.
Since StatefulSet deal with Storage they depend of the presence of an implementation of the CSI, Container Storage Interface, installed in the cluster. The main resources to deal with storage are PersistentVolumeClaim and PersistentVolume.
Difference respect to normal (stateless) deployments
Required packages installation on nodes plus longhorn installation.
Patch servers:
sudo apt install -y nfs-common
sudo systemctl enable --now iscsid
Install long horn in the cluster
curl -LO https://raw.githubusercontent.com/longhorn/longhorn/v1.2.4/deploy/longhorn.yaml
kubectl apply -f longhorn.yaml
vagrant ssh kmaster
cd examples/chapter-07/files
cat sleep-set.yaml
kubectl apply -f sleep-set.yaml
kubectl get statefulset
kubectl get service
kubectl exec sleep-0 -- /bin/sh -c 'hostname > /storagedir/myhost'
kubectl exec sleep-0 -- /bin/cat /storagedir/myhost
kubectl exec sleep-1 -- /bin/sh -c 'hostname > /storagedir/myhost'
kubectl exec sleep-1 -- /bin/cat /storagedir/myhost
Now if I delete a Pod the replacement will take the same storage
kubectl delete pod sleep-0
kubectl get pods -w
kubectl exec sleep-0 -- /bin/cat /storagedir/myhost
Clean up
vagrant ssh kmaster
kubectl delete -f examples/chapter-07/files/sleep-set.yaml
This type of Service is the default and exists on an IP that is only visible within the cluster. It enables cluster resources to reach one another via a known address while maintaining the security boundaries of the cluster itself.
Pingolo!
A Show me the code moment, then...
cd ~/src/pingolo
kubectl apply -f kubernetes/deploy.yaml
kubectl get all -n pingolo
# try to connect to http://172.16.16.100
kubectl logs -n pingolo -l usecase=ping-from-pod
# try to ping something outside to cluster
kubectl apply -f shpod.yaml
kubectl get pods -n shpod
# try to ping the shpod
A way for exposing applications outside the Cluster.
The Ingress Controller is an application that runs in a cluster and configures an HTTP load balancer according to Ingress resources.
From the kubernetes book. NOT WORKING on my Cluster at least for ports 80 and 443, cause I have to do tweaks on Calico I'm not able to, so that the load balancer can bind on port 80 and 443.
curl -Lo ingress-controller.yaml https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.1/deploy/static/provider/cloud/deploy.yaml
cp examples/setup/roles/k8s/templates/ingress-patch.yaml.j2 ingress-patch.yaml
kubectl apply -f ingress-controller.yaml
kubectl patch -n ingress-nginx service/ingress-nginx-controller --patch-file ingress-patch.yaml
There are many Kubernetes distributions out there. All of them has "opinions". Their basic functions is to provide installer or applications to make Kubernetes installation less painful then the vanilla installation.