Introduction to Google Kubernetes Engine
We have used Kubernetes distribution k3s using docker containers via k3d. In a production environment the task of maintaining a Kubernetes cluster is often left to third parties. A managed Kubernetes as a service is often the best choice as the additional work required in maintenance exceeds the benefits of a personal cluster. In some, somewhat rare, cases setting up and maintaining your own cluster is a reasonable option. A case for it would be that your company/organization already has the hardware and/or wants to stay independent from providers, one such example could be a University. Another primary reason for operating your own Kubernetes cluster is that regulations rule out all other choices.
Maintaining your own Kubernetes cluster is one way to increase costs. The other is running servers that are not in use. To avoid costs Serverless solutions could be more cost-efficient. A Kubernetes cluster of improper size can get really expensive really fast. An excellent option to start out with would be a high-tier environment for serverless workloads, such as Google Cloud Run or AWS Fargate. In them you can run any container, compared to older solutions like Cloud Functions or AWS Lambda, which have a wildly varying support for different languages, environments and tooling.
Let's focus on the Google Kubernetes Engine (GKE) costs for now. Note that the GKE costs a little bit more than its competitors.
The calculator here https://cloud.google.com/products/calculator offers us a picture of the pricing. I decided to try a cheap option: 6 nodes in 1 zonal cluster using 1 vCPU each. The datacenter location is in Finland and I don't need a persistent disk. If we wanted less than 6 nodes why would we even use Kubernetes? The total cost for this example was ~145€ / ~$160 per month. Adding additional services such as a Load balancer increase the cost. If you find the billing for Google Cloud Platform confusing, you're not alone: Coursera has ~5 hour course for "Understanding Your Google Cloud Platform Costs".
During part 3 we will be using GKE either by using the student credits available to students with helsinki.fi email or the free credits offered by Google. You are responsible for making sure that the credits last for the whole part and if all of them are consumed I can not help you.
After redeeming the credits we can create a project with the billing account. The Google Cloud UI can be confusing. On the resources page we can create a new project and let's name it "dwk-gke" for the purposes of this course. After creating this project make sure that the project is linked to the correct billing account from the top-left dropdown and billing and then "Account Management". It should look like this (In this case the account is "DevOps with Kubernetes" and project "dwk-gke"):
Install the Google Cloud SDK. Instructions here. After that login and set the previously created project to be used.
$ gcloud -v Google Cloud SDK 363.0.0 bq 2.0.71 core 2021.10.29 gsutil 5.4 $ gcloud auth login ... You are now logged in $ gcloud config set project dwk-gke Updated property [core/project].
We can now create a cluster. You can choose any zone we want from the list here. I chose Finland. Notice that one region (e.g. europe-north1) may have multiple regions (e.g. -a). Let's add another flag:
--cluster-version=1.22. This will ask GKE to use a version that will be default in June 2022. GKE release schedule here.
$ gcloud container clusters create dwk-cluster --zone=europe-north1-b --cluster-version=1.22 ERROR: (gcloud.container.clusters.create) ResponseError: code=400, message=Failed precondition when calling the ServiceConsumerManager: tenantmanager:: Consumer should enable service:container.googleapis.com before generating a service account.
Let's enable the service in question,
container.googleapis.com, before retrying.
$ gcloud services enable container.googleapis.com Operation "operations/acf.p2-385245615727-2f855eed-e785-49ac-91da-896925a691ab" finished successfully. $ gcloud container clusters create dwk-cluster --zone=europe-north1-b --cluster-version=1.22 ... Creating cluster dwk-cluster in europe-north1-b... ... kubeconfig entry generated for dwk-cluster. NAME LOCATION MASTER_VERSION MASTER_IP MACHINE_TYPE NODE_VERSION NUM_NODES STATUS dwk-cluster europe-north1-b 1.22.8-gke.2200 22.214.171.124 e2-medium 1.22.8-gke.2200 3 RUNNING
It set the kubeconfig to point in the right direction already. But if you need to do it again we can set the kubeconfig like this:
$ gcloud container clusters get-credentials dwk-cluster --zone=europe-north1-b Fetching cluster endpoint and auth data. kubeconfig entry generated for dwk-cluster.
You can check the cluster info with
kubectl cluster-info to verify it's pointing in the right direction.
The cluster we have now is almost like the one we had locally. Let's apply this application that creates a random string and then serves an image based on that random string. This will create 6 replicas of the process "seedimage".
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes-hy/material-example/e11a700350aede132b62d3b5fd63c05d6b976394/app6/manifests/deployment.yaml
Exposing the service is where the differences start. Instead of an ingress we'll use LoadBalancer service. Now as a warning the next step is going to add into the cost of the cluster as well.
Apply the following:
apiVersion: v1 kind: Service metadata: name: seedimage-svc spec: type: LoadBalancer # This should be the only unfamiliar part selector: app: seedimage ports: - port: 80 protocol: TCP targetPort: 3000
A load balancer service asks for Google services to provision us a load balancer. We can wait until the service gets an external IP:
$ kubectl get svc --watch NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.31.240.1 <none> 443/TCP 144m seedimage-svc LoadBalancer 10.31.241.224 126.96.36.199 80:30215/TCP 94s
If we now access http://188.8.131.52 with our browser we'll see the application up and running. By refreshing the page we can also see that the load balancer sometimes offers us a different image. Note that this is available in port 80, so https will not work.
$ gcloud container clusters delete dwk-cluster --zone=europe-north1-b
And when resuming progress create the cluster back.
$ gcloud container clusters create dwk-cluster --zone=europe-north1-b --cluster-version=1.22
Google Kubernetes Engine will automatically provision a persistent disk for your PersistentVolumeClaim - just don't set the storage class. If you want you can read more about it here.
Services are quite simple. Let's try using ingress since it offers us additional tools in exchange for complexity.
NodePort type service is required with an Ingress in GKE. Even though it is NodePort, GKE does not expose it outside the cluster. Let's test this by continuing with the previous example.
apiVersion: v1 kind: Service metadata: name: seedimage-svc spec: type: NodePort selector: app: seedimage ports: - port: 80 protocol: TCP targetPort: 3000
apiVersion: networking.k8s.io/v1 kind: Ingress metadata: name: seedimage-ing spec: rules: - http: paths: - path: / pathType: Prefix backend: service: name: seedimage-svc port: number: 80
When you have applied those you can view the port from the list of ingresses:
$ kubectl get ing NAME CLASS HOSTS ADDRESS PORTS AGE seedimage-ing <none> * 184.108.40.206 80 2m13s
Now the address here will be the way to access the application. This will take a moment to deploy, responses may be 404 and 502 as it becomes available. The Ingress performs health checks by GET requesting / and expects an HTTP 200 response.