What is Reddit's opinion of
Kubernetes?
From 3.5 billion Reddit comments
➔ Kubernetes website
By popularity on Reddit, this Service is:
100 reviews of this app found across Reddit:
> Kubernetes, also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications.
Imagine you have a computer and you want to run a piece of software on it. So you just do.
Now imagine you have a thousand computers and you want to run the piece of software on a hundred of them --- and your coworkers have a different piece of software and they want to run it on some computers too and so on.
Instead of having a whiteboard and reserving computers, you run kubernetes, tell it what to run and how many (and some other constraints) and it figures it out for you.
From then on one does not worry about specific computers and what goes where, you just run your software on your cluster.
I keep [this]( https://kubernetes.io/docs/reference/kubectl/cheatsheet/ ) (official k8s docs) bookmarked, mostly for all the filtering/formatting options.
> I mean things casual developers could use in any meaningful way?
Not really, unless you're casually running a large scale distributed system.
You can look at the major users of existing container managers like Kubernetes and Apache Mesos to get a sense of who would actually use this.
If you try Tectonic Sandbox, you get a turnkey local version (thanks to Vagrant).
CoreOS has a good guide for getting started: https://coreos.com/tectonic/docs/latest/tutorials/sandbox/first-app.html
Other distributions should have something similar. And if all else fails, Kubernetes.io also has good tutorials: https://kubernetes.io/docs/tutorials/
If you're confused about "Why" you'd use Kubernetes, you may not need it yet. For me, the biggest reason to use it is for when you start running a lot of containers. With Kubernetes, it's often as easy to manage 3 containers as it is 300.
That's the real gist of it. Once you start thinking about updating 300 containers running on say, systemd instead, it's a headache. It's a long, manual, and error prone process. Kubernetes just mostly figures things out for you after you tell it what you want.
The tutorials won't show you that side of stuff, if you haven't already experienced the pain of large-scale distributed management. The tutorials will introduce you to Kubernetes primitives and the basics for getting stuff done.
Full disclosure: I work for CoreOS.
im willing to bet you have not set proper resource limits on your pods:
https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/
and your applications are eating up all the available cpus.
This has been kindof discussed before.
Kubernetes isn't necessarily the right answer here imo. You might be better off learning linux and ansible; and just setting this all up on some cloud servers on Digital Ocean, Linode, or Vultr.
If you really really just want to get some Kubernetes learning in though, start by learning the types of workloads it has available to see if they can fit the model you need.
Operators are meant to manage the lifecycle of a specific application (e.g drain a database, move replicas around, take backups, coordinate version upgrades etc). I think a lot of people are mixing up what operators are vs a custom controller.
This is not that use case. Just because you can do something doesn't make it a valid pattern.
https://kubernetes.io/docs/concepts/extend-kubernetes/operator/
> The Operator pattern aims to capture the key aim of a human operator who is managing a service or set of services. Human operators who look after specific applications and services have deep knowledge of how the system ought to behave, how to deploy it, and how to react if there are problems.
The issue isn't so much that people do it, it's that it's the recommended procedure for many "new and shiny" softwares. For example (2min of googling since I don't keep a list) this proposes it https://pi-hole.net/ and it's not necessarily piped in bash that's problematic. Kubernetes uses it to add GPG keys for example : https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl
People take the path of least resistence so it's important to explain that there are risks involved.
You have to add a admission controller to get the same thing in vanilla k8s, https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#podnodeselector
Close enough to the openshift version that it doesn't feel too alien.
No it's not. "A CPU" is one vCPU/core/hyperthread/whatever-your-infra-calls-it. How fast that is will vary based on your hardware; there is no such 1GHz approximation. https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-cpu
IMO, it would be better if there were such an approximation, because as it stands it's impossible to write good CPU limits/requests without knowing where it will be deployed.
There are some good tips in there but not all great for CKA. No one is going to remember a bajillion aliases and set up their bashrc for 20 minutes for a shortcut you use 3 times. I don't think you really need more than:
k=kubectl kn='kubectl config set-context --current --namespace' export dry='--dry-run=client -o yaml'
Along with the completion stuff and the complete -F __start_kubectl k
I also edit /etc/vim/vimrc & at bottom put: set ts=2 sts=2 sw=2 ai
Honestly the CKA is its own beast and shouldn't be conflated with useful stuff to actually admin a cluster.
Anyway IMO the actual cheat sheet you can access during the exam is more useful for CKA; https://kubernetes.io/docs/reference/kubectl/cheatsheet/
Getting the kubeadm CM from kube-system is a good tip, nice to get both pod and svc networks from one place quickly.
There is also some deprecated stuff in there.
k get cs Warning: v1 ComponentStatus is deprecated in v1.19+
Ubuntu 18.04 and kubeadm are the easiest option I've found. https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/
If you choose Flannel, the yaml in the last release doesn't work with k8s 1.12. Use this version of the yaml https://raw.githubusercontent.com/coreos/flannel/13a990bb716c82a118b8e825b78189dcfbfb2f1e/Documentation/kube-flannel.yml
This bash script I wrote for use in aws may or may not help https://github.com/cablespaghetti/kubeadm-aws/blob/develop/master.sh
>It makes it easy to run anywhere
No it does not because Bitcoin Core binaries already run anywhere without being a Docker image. So using Docker to be platform independent cannot be the reason Garzik uses it. A Docker image however makes it super easy to deploy node instances on cloud platforms.
Their mention of Kubernetes is an even more clear giveaway because it is software designed to automate large scale software deployment.
A pod is not going to get evicted from a node for just any reason. Outside of complete node failure, it would take the pod exceeding resource limits or dying on its own for the control plane to intervene in its lifecycle. This page in the documentation explains it more in depth.
For whatever it's worth, I made something that does almost exactly what you are describing, but instead of just SSH connections, it gives you a full graphical desktop inside pods. You can check it out here - https://github.com/kvdi/kvdi
Honestly, your idea sounds like it could be a cool feature to add. It's been a couple months since I last touched it, but at the time I was in the middle of giving it full VM support on top of the existing containerized "desktops". But the foundations are there to add ideas like the one you have pretty easily.
Helm adds a fair bit of complexity on top of Kubernetes but I think it can also help show you around. Take a look at the helm charts repository for some ideas that will be useful for you. If you know something well like rabbitmq, redis, etc... and you know what running them looks like (ports, storage, etc...) then deploy them using helm and then look at their charts source to get an understanding of helm and kubernetes.
If you see something in the chart that you don't know, look it up, the Kubernetes docs are pretty good at explaining what the different resource types are and how they relate to each other.
Did you check the documentation? As you are using k8s then you can inyect the secrets to the container via files or environment variables. Do NOT put the secrets in the image.
But Traefik supports standard ingress resources as well. They are simply just festure rich enough. The community is working on a standard for things such as IngressRoutes in the Gateway API. https://kubernetes.io/blog/2021/04/22/evolving-kubernetes-networking-with-the-gateway-api/
- Don't run containers root
- Have good RBAC roles set up. Don't wildly grant people cluster-reader or cluster-admin because it's easy -- take the time to build up the policy that you actually need.
- Have good pod security policies set up (https://kubernetes.io/docs/concepts/policy/pod-security-policy/). For some examples, take a look at the OpenShift default cluster policy, which is significantly more restrictive than the default Kubernetes policy.
- Make use of quota to prevent accidental overconsumption of resources.
- If you're fairly concerned, know where your images are coming from. There's a few different ways that you could restrict where images come from (blocking registries in
/etc/containers/registries.confin crio, validating registries in a custom admission plugin, etc).
For setting up a pi4 as a k8s: I was able add kernel modules to Ubuntu server Arm64 version to boot from USB. I don't remember which ones, because I made an image I use to flash on my USB drives. I had to update the firmware on each new pi too. This looks close to what I did https://www.instructables.com/Raspberry-Pi-4-USB-Boot-No-SD-Card/
I recommend naming all the pi hostnames similar,
I used this guide to install k8s from kubeadm and select the recommended containerd https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/
Docker swarm is a ton of fun too.
A lot of my knowledge came from doing, failing and trying again. I am sorry if my answer seemed dismissive or beneath your knowledge set. There is so much to learn and talk about and focused questions work best for me. I am very happy to answer as much as I can for you or anyone else that asks.
I think it might be best for me to make a video series that goes through my setup and explains elements of the cluster in a seflhosted solution.
Assuming you're using plexinc/pms-docker, it takes a PLEX_CLAIM environment variable.
You can pass secrets to pods as environment variables as described here: https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets-as-environment-variables
The recommended way is to use Pod Topology Spread Constraints
Specifically using the key kubernetes.io/hostname
millicores are one of the standard ways of specifying CPU resource limits in kube, so that is indeed where they got it
Is there a reason you dont use ConfigMaps?
https://kubernetes.io/docs/concepts/configuration/configmap/#configmaps-and-pods
Nice guide. I bookmarked it and going to send it some who are just starting with kubernetes.
​
I have one suggestion:
When talking about the K8s master components you listed etcd instead of controller-manager. In my opinion its far more important to know about controller-manager than etcd in that context.
https://kubernetes.io/docs/concepts/overview/components/#kube-controller-manager
It sounds sounds very much like the "debug container" feature, don't you think? 😉 https://kubernetes.io/docs/tasks/debug-application-cluster/debug-running-pod/#ephemeral-container-example
Disclaimer: I took the CKA before this question/feature existed so I'm just guessing.
99.9% chance this is just someone in IBM HR being dumb.
But... it might actually be possible to find someone with this much experience if you count experience with Borg, the direct predecessor of Kubernetes.
From the Kubernetes blog:
That says "more than a decade" but doesn't make it 100% clear that Borg was the system used for containerization all that time.
However, the Borg paper ("Large-scale cluster management at Google with Borg") gives another detail:
> scheme that Google MapReduce jobs have used to obtain resources from Borg since about 2008
So that means Borg existed as far back as 2008, which is 12 years ago. And probably longer.
For a quick start, I am a fan of minikube. By default it will run in a single VM, but like a "real" kubernetes cluster you can join additional worker nodes to it.
>My understanding was that Kubernetes would let me define a group of containers, possibly with some rules about scaling, and then deploy them all together, consistently, anywhere that I had a working Kubernetes setup.
That's mostly true. If you need persistent storage or a particular ingress setup then you may need to customize your resource manifests for your particular provider. But for the type of stuff you can do with docker compose, you absolutely should be able to run the same setup on any vanilla Kubernetes cluster.
You can set the scaling parameters of your services via the horizontal pod autoscaler. If you're going to use that you'll also want to use cluster auto-scaling if your provider supports it.
>I expected to be writing a declarative config describing the architecture I wanted and then using that to replicate my setup in another environment
Resource manifests are Kubernetes' equivalent of compose files, and it's relatively easy to port compose files to k8s manifests. There's even a tool to convert them for you.
>but the tutorials I've been looking at all lean pretty heavily on using kubectl to set up services.
kubectl is like docker-compose: it's just a command-line tool that executes API requests. In the case of resource manifests, it's taking declarative config files and translating them into calls to the Kubernetes API to do your bidding.
>I'd like to see if I can get it set up for dockerized deploys/CI
The most basic way to do this is to define your applications as deployments and then use something like Jenkins to orchestrate rolling updates using kubectl.
That is all fine, especially when you say our docs do not explain it well enough, we should fix that. Later if you have time please specify where you landed and which docs were to generic/complex or whatever your thoughts are.
​
The whole story of Kyma I tried to explain here as clear as possible https://kubernetes.io/blog/2019/05/23/kyma-extend-and-build-on-kubernetes-with-ease/ and also recorded a 7min video about it's key features.
​
In short:
In general, main thing about Kyma is that it is a way to go with monoliths that you want to turn into cloud native apps. It was donated to open source by SAP so a company that you might suspect knows somethign about monoliths.
Also, to speed up Kubernetes adoption for large teams that are used to working with monoliths, we designed Kyma to come with all batteries included. So you do not have to, on your own, decide on what tools to use, which service mesh, what monitoring, what logging, because we bring them all integrated. So Kyma is ready not only to write solutions on it but also to operate them easily.
Let's see if there's a pattern:
Docker in Docker: dind
Kubernetes in Docker: kind
Kubernetes in Kubernetes: ... ... KUBEINCEPTION! (Kubeception from Giant Swarm works as well.)
I would encourage raising awareness of how to adjust the stabilization window on scale down or scale up to change the speed of scaling. The default window is 300 seconds, which can be a long time. Here's the link to that documentation.
apiVersion: autoscaling/v2beta2 kind: HorizontalPodAutoscaler metadata: name: target-hpa spec: behavior: scaleDown: stabilizationWindowSeconds: 60 maxReplicas: 10 minReplicas: 1 scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: target-deployment metrics: - type: Resource resource: name: cpu target: type: Utilization averageUtilization: 75
> I’ve looked it up on the web and found nothing related
Really? It is not exactly a common term, all search engines should find it with ease. Are you behind a heavily filtered network or something?
First two links on google:
As well as every other link on the first page, and likely every other page.
Docker is really easy (and actually convenient) for local development. Kubernetes is a bit more complex, but for that you can use Minikube.
As for project ideas; a chat application is fine. You can have a 'back end' service, a single page application 'front end' service, and a database (to store user profiles / chat for example). So that's 3 different kubernetes pods already. In addition you can also add metrics collection, logging, etc. Additional services you could deploy inside kubernetes.
So NFS has some issues with any sqlite DB (so much sqlite recommends not putting a DB on NFS due to how it handles file locking). If you're not aware, most apps like sonarr/radarr/tautulli/etc rely on sqlite. I had some issues where I was running into DB lock errors. The other issue is it seems there isnt a StorageClass that supports dynamic provisioning on NFS (or iSCSI). That means you need to create a PV manually for every mount point (not the worst thing) and possibly the LUN manually in the case of iscsi. Check out https://kubernetes.io/docs/concepts/storage/storage-classes/ to see the list of supported storage techs for the storage class provisioner. A lot of helm templates are built on the false assumption you have a storage class provisioner.
Both iSCSI and NFS seem to have their use within k8s (I personally mount my config dir's on iSCSI, media on NFS but that will change with gluster) if you dont mind a bit of work.
Gluster and Ceph are both pretty well supported within k8s. It has an auto provisioner to automatically create volumes so pods can spin up pretty easily.
Random shout out, if you're setting up some bare metal k8s check out metallb (https://github.com/google/metallb) it's a load balancer you can deploy & works great.
You can whitelist IP addresses by specifying .spec.loadBalancerSourceRanges in your service yaml.
See https://kubernetes.io/docs/concepts/services-networking/service for an example as well as several other options for configuring ELBs.
You’re describing the core concepts of what most container orchestration systems provide you. A way to manage, interconnect, and schedule one or more containers in a sensible way. E.g. The concept of a Pod in Kubernetes: https://kubernetes.io/docs/concepts/workloads/pods/pod/
This is very specifically out of scope for Kubernetes, see "What is Kubernetes?".
Kubernetes makes it easy for users to implement a modular PaaS that suites their needs, or even FaaS. The main issue with PaaS is that it's hard to get it right for different use-cases, it's either too simple and opinionated or too complicated and has very opinionated defaults. It's much easier to think of Kubernetes as the Lego for building PaaS (or FaaS, as I said, and OpenFaaS is a good example of that).
Can you clarify "when the job competes kill the create"?
It sounds like you want the pod to do some work, then terminate when that work is complete. If this is what you are after you should look at creating a pod or a job instead of a deployment. https://kubernetes.io/docs/concepts/workloads/controllers/job/
Isn't the nginx ingress controller also able to route based on the hostname via name based routing? Guessing I'm missing something here! https://kubernetes.io/docs/concepts/services-networking/ingress/#name-based-virtual-hosting
I run a selfhosted NUC-based cluster, where all relevant nodes are schedulable. I use rook as storage.
Before I set proper limits on the OSD's, the memory would frequently run out on nodes, that would sometimes lead to the following fun sideeffects:
- All pods on that node killed
- Ceph cluster marking the OSD as out, and start to rebuild
- this caused even more load on the remaining OSD's
- which then trigger higher RAM usage on the remaining nodes, which caused a nice loop all over.
Now, this was a consequence of me not knowing what I was doing, using too small nodes, and copying in a fairly substansial amount of data.
When the node that had stuff killed was a master-node also, the cluster also started doing funny stuff, as one can assume it does when a 3-master-cluster suddenly only consists of two, or even one Master-node being ready.
All in all, for me this was a great learning experience.
It made me hugely impressed of the resilience of both rook, ceph and k8s in general - stuff might break, but the parts come together again fairly easy.
But to the topic;
Yes, you can use masters as worker-nodes.
That might also mean that your jobs will affect the cluster's stability in general, only you can say for certain if that cost is worth it.
I have also read that there are some security implications, but I'm not there yet - so I can't provide any information about that.
edit: Also, for reference:
kubectl taint nodes --all node-role.kubernetes.io/master-
This is what allows master-nodes to run schedules.
https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/ This is the documentation behind the consept of tains.
Kubernetes by default sets ndots to five, so it doesn't assume it's an fqdn unless there are at least five periods in the host name. You can adjust this with a custom DNS config:
https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-dns-config
Normally the resolution should fail fast enough even looking up all the possible search domains that the fqdn does get looked up, but you can manually lower ndots to help with this.
You may also be running into a conntrack race condition. This can be fixed by using the single request reopen option on libc based pods. I don't know if there's a fix for Alpine base pods yet. Here's an article that explains the problem in more detail:
https://medium.com/techmindtickle/intermittent-delays-in-kubernetes-e9de8239e2fa
The first thing I always do is run
Kubectl logs [pod name]
That command will display the application logs.
If the issue is that the pods are not spinning up you can also use
Kubectl describe pod [pod name]
This command gives a lot of information, the section that is helpful is the events section.
The link below is for the interacting with pods section of the kubernetes documents, and there is helpful info there. Kubernetes documents
You should define rds instance as Service of type: ExternalName and use it as regular service.
https://kubernetes.io/docs/concepts/services-networking/service/#externalname
If the file doesn't contain secret data (i.e. passwords, private certificates) then you should use a ConfigMap. If it does contain secret data then a secret is the right choice. See the documentation neilwatson posted for secrets, or the ConfigMap documentation
Airflow is great to hook those services. We are transitioning to integrate it with Kubernetes to have more flexibility in our architecture.
https://kubernetes.io/blog/2018/06/28/airflow-on-kubernetes-part-1-a-different-kind-of-operator/
You can set your API keys as environment variables on your server. The way you'd go about doing this varies based on your production environment. For example, if you're working off of a kubernetes cluster then you could define a secrets file, then expose your secrets in your environment: https://kubernetes.io/docs/concepts/configuration/secret/
> Do you foresee kubeadm becoming the standard way to create / upgrade clusters (outside of hosted installations)?
Kubeadm is widely recognized as a standard way to deploy Kubernetes clusters with a wide variety of options. While the Kubernetes community supports multiple cluster deployment solutions simultaneously (mainly because there is no single best solution that will satisfy all the needs of everyone), Kubeadm (https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/) - is the right solution that we may suggest for deploying the production-ready Kubernetes clusters.
But again, there are a lot of multiple solutions to deploy and manage Kubernetes clusters, all of them have their own pros&cons. Kubeadm is just one of them.
Kubernetes is way overkill for almost everybody. Unless you are a huge corp with a need for an in-house cloud platform, it will be overkill. Kubernetes is closer to OpenStack than to AWS ECS, which is what OP was considering.
Kubernetes is also notoriously difficult to install and maintain. There is currently 43 different ways to install Kubernets and it is highly unclear which ones will still be around in a year. Once it is up an running, it is a fine product, but be ready to allocate large resources to admin it.
If you really want Kubernetes, buy it as a service from Google Cloud.
you can use ephemeral containers (which moves into beta in 1.23)
https://kubernetes.io/docs/concepts/workloads/pods/ephemeral-containers/
not who you're asking, but: the goal is to keep credentials out of a file that might be copy pasted or moved. It's also harder to deal with .env files in a dockerized app: now you have a file full of credentials that's part of your image AND should you need to change or rotate a credential now that's a whole new build/deploy versus pointing the env var to a new value.
It's not bad practice to use environment variables, but a .env file is a less good choice than a secrets management service.
Instead you the facilities available with however you're deploying the thing. Examples:
- https://kubernetes.io/docs/concepts/configuration/secret/
- https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data-secrets.html
- This is a bit dated now that most container runtimes have secrets management baked in, but this is a thing you can do too: https://chrisguitarguy.com/2017/12/23/secrets-in-dockerized-applications/ fetch secrets into env vars during a docker entrypoint script or during application bootstrapping. Probably don't want to do this on every request, so if running outside of docker you might have a script that fetches these then runs FPM with
clear_env = Nofor workers.
There's also stuff around logging when secrets were accessed and by who/what that you'll get for "free" out of secrets management services.
come on - this is barely "advanced patterns". You don't even go into things like aggregated roles.
I wouldn't bother with this article - just go to the official docs if you want actual "advanced" patterns: https://kubernetes.io/docs/reference/access-authn-authz/rbac/
I agree with the others, it is not necessary since the mumshad course is very good and include lab's (there are also free labs in katakoda) but anyway I used an R720 for practice and for fun (ubuntu 18.04 + kubeadm install instructions from kubernes.io and i did all the task's) Then I left the automatic installation with ansible (I passed CKA 1 week ago).
I hope it helps you, sorry for my English :3
The biggest things which helped me speed up were:
- As I went through the trial tests I bookmarked the page which gave me information if I needed it (such as how to backup etcd). This way I had helpful bookmarks instead of just links to subjects.
- Name the bookmarks something useful, assuming you would be looking quickly for an answer.
- Setting up the alias k=kubectl and autocomplete was a real timesaver. (https://kubernetes.io/docs/tasks/tools/install-kubectl-linux/#enable-shell-autocompletion)
It sounds like you're on the right track here. Integrate git into your CD tool of choice, managed with a particular SA, monitor any create events against the k8's API that aren't executed from that SA, and I feel like you're 90% of the way there.
Edit: here's the documentation on audit logging. https://kubernetes.io/docs/tasks/debug-application-cluster/audit/
You need to set up ingress with host based routing inside your AKS cluster (https://kubernetes.io/docs/concepts/services-networking/ingress/). You would set up a service for your pods, and ingress should be configured so that each host (the subdomain) routes traffic to the appropriate pod. Ideally you would want to deploy your pod as a deployment. Your DNS records would point to your AKS cluster. I think that should be able to do it.
Essentially, DNS record -> AKS IP -> ingress looks at the host and routes the traffic -> service -> pod
This. You can communicate with services in other namespaces, on the same cluster, using the full service discovery DNS name, unless it has been restricted with a network policy. We use these for intra-cluster rest API calls, and they are of the form:
<service_name>.<service_namespace>.svc.cluster.local:<service_port>
It'll run 4 processes. CPU limits in K8s are not CPU cores, either.
https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#meaning-of-cpu
I don't think the CPU limits work the way you're thinking they do. It's more of a weighting thing, rather than a limit.
"A Container with spec.containers.resources.requests.cpu of 0.5 is guaranteed half as much CPU as one that asks for 1 CPU. "
"A Container might or might not be allowed to exceed its CPU limit for extended periods of time. However, it will not be killed for excessive CPU usage."
"The spec.containers[].resources.limits.cpu is converted to its millicore value and multiplied by 100. The resulting value is the total amount of CPU time that a container can use every 100ms. A container cannot use more than its share of CPU time during this interval."
This last part is important. If you spec a CPU limit and it gets limited, it's a "stop the world" limiting. The process is just frozen until the next 100ms interval. This can lead to situations where you hit the limit in, say, 5ms, then for the next 95ms the process is just frozen and unresponsive to anything.
You can Translate a Docker Compose File to Kubernetes Resources quite easily. So that nice if you have plans to run in k8s.
Awesome, love the diagrams.
Any way to note that services and endpoints can refer to other entities, not just pods? Eg I've used that to refer to an elastic cluster running outside kubernetes.
Look at this: https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#mounted-configmaps-are-updated-automatically
You need to mount the config map as a volume. Environment variables from a config map don't get updates.
Also you should probably use a StatefulSet instead of Deployment. Just look it up.
And instead of scaling down and then up, you can run:
kubectl rollout restart statefulset mysql
k8s (and CoreDNS) inherits DNS configs from your nodes. Make sure your /etc/resolv.conf is set up right on your rPis that make up your cluster, specifically the ndots option, nameservers, and search list.
Edit: this might also be helpful
The test itself isn't very difficult. You get a free retake so don't worry about failure. What you should focus on at this point is preparing your test environment, make sure you have all your bookmarks for kubernetes.io/docs prepared and that you are well rested. You DO have the kubernetes cheat sheet bookmarked right?
https://kubernetes.io/docs/reference/kubectl/cheatsheet/
Time saving tips I'd recommend are typing in a command + --help to have the entire command/options in front of you while typing. Make sure to type commands into the notepad provided with the test, it is much faster to edit a command in a text editor than to scroll thru your command history and navigate to that one character you need to correct. Commit to a specific amount of time with each question and move on if you are dragging. I had a whole hour left over after completing all questions to go back and check my work.
Best of luck on your K8s Journey, this is only one step!
Put a taint on the nodes with effect NoSchedule, then use nodeSelector on your pod template to favor those nodes by label and include a toleration to overrule the taint for workloads that are specially permitted to run there. This is a textbook example of the purpose for those features.
https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/
You are taking the only instance offline to "force" an upgrade instead of performing a "rolling upgrade". Use a Deployment instead. https://kubernetes.io/docs/concepts/workloads/controllers/deployment/
Using the local persistence volumes in kubernetes was the only way we found to get reasonable performance with staying in a container model. https://kubernetes.io/blog/2019/04/04/kubernetes-1.14-local-persistent-volumes-ga/. We are using ephemeral for configuration and limited data storage. Prioritizing the pods and allowing kubernetes to manage the local space ( format, attach and reclaim) Couple google searches will show some helm/charts people have written for Postgres.
The best way to learn kubernetes is from the official kubernetes documentation . The explanation is clear and newbie friendly
https://kubernetes.io/docs/home/
You can learn kubernetes with minikube and try to deploy jenkins inside it
You need to specify the name of the service account like this: system:serviceaccount:(NAMESPACE):(SERVICEACCOUNT) as described here.
The full command then becomes this:
$ kubectl auth can-i get pods --namespace rbac-test --as system:serviceaccount:rbac-test:rbac-test-sa
yes
We’ve been trying to get a stable multi-master k8s setup up and running (following Kelsey Hightower’s Kubernetes the Hard Way guide) on Azure (without AKS, for good reasons) for 3 weeks now, and it’s been a constant uphill battle. We initially tried setting up the cluster using kubespray and then another cluster using kubeadm and both approaches were painful and didn’t work due to complex and esoteric networking issues. We had to hack the kubespray Ansible Playbooks to get them to work in the first place, because it doesn’t seem as though they’re properly tested for use in Azure, but then our networking would arbitrarily break and we couldn’t figure out why.
The biggest learning I’ve taken from the experience is that there’s no way around getting into the details (and with k8s, there’s so much detail it’d make your head spin). We’ve literally been learning Golang so that we could read the code for certain k8s components, since in certain small but critical places the k8s documentation is currently either out of date, or makes assumptions about your background knowledge. We also ended up learning about crucial parts of how k8s works from Github issues as opposed to the documentation, where I would’ve expected more from the maintainers.
I personally find this incredibly frustrating, because any sufficiently well-written complex system follows the rule: make the simple things easy and the difficult things possible. K8s doesn’t follow this rule. It’s not mature enough yet.
Once kubeadm works better, I think things will get better (I hope).
Until then, we will continue to fight this uphill battle.
In the container ideal world, every component is a separate continer. Part of the idea is that once you have the need for scaling things, you just deploy a dozen of container A, a couple extra container B, and maybe one more C.
Also, containers shouldn't contain an OS. The ideal golang container is just the go binary, and maybe a config file. My project builds go containers with a busybox base image, just so you get a shell in the container if you really need it.
For the database, you still want to run MySQL in the container, but use a volume mapping to persist the data somewhere on your server.
If you're running more than one server in the real deployment, I would start thinking about Kubernetes. It takes all of the networking bits and abstracts them into "services". This deals with the port/network routing issues you're talking about.
You could start with Minikube on a single server to try it out.
Rather than implementing this directly in the application, I would model this using Kubernetes: https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/
And have your app do the thing once via a CLI command that would be executed in a container.
You need a scheduler. K8s, mesos, rancher, dockerswarm.
You can do horizontal scaling, not vertical. Meaning in kubernetes you would assign horizontal-pod-autoscalers (hpas) to watch the cpu usage of a deployment and it would scale up new pods as necessary. The new pods would be based on the original deployment so the resource limits would be exactly the same, hence why it's a horizontal and not vertical scale.
https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/
Don't even attempt to do this with your databases until you fully understand how they work. You will likely destroy all of your data, or you'll wind up in a split brain scenario, etc.
Stateful containers (ones that need to maintain persistent data through restarts) are still somewhat of a challenge to manage. Effectively running your own Dockerized MySQL means leveraging storage options correctly, making sure the right EBS volumes are mounted to the right container instance, etc.
Kubernetes is tackling this with StatefulSets (https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/), but AWS is pushing the use of their Elastic File System for this pattern (https://aws.amazon.com/blogs/compute/using-amazon-efs-to-persist-data-from-amazon-ecs-containers/).
The easy way is to just use RDS instead. Then, you're not actually Dockerizing MySQL because you're not really managing a MySQL cluster manually. Instead, your other, stateless containers simply connect to RDS using variables defined in the environment like RDS_HOST, RDS_USERNAME, RDS_PASSWORD.
SACREMENT QUE KUBERNETES C'EST DE LA MARDE À INSTALLER QUAND TU VEUX JUSTE UNE OPTION BARE METAL.
BESOIN DE L'INSTALLER ABSOLUMENT SUR CENTOS? OHH OHH OH, NOUS AVONS CE PETIT GUIDE POUR VOUS AIDER À FAIRE UNE POC TOUTE SIMPLE.
ASTEUR BONNE CHANCE POUR INSTALLER KUBE-DNS ET KUBE-DASHBOARD, C'EST PAS INSTALLABLE. TU VOULAIS PAS PAYER POUR UNE OPTION CLOUD, HEIN? BEN DÉBROUILLE TOI MON CAVE À GRAND COUP DE LIENS SUR GITHUB QUI N'APPORTENT AUCUNE SOLUTION CONCRÈTE.
C'EST CERTAIN QUE JE VAIS RÉUSSIR À FAIRE MARCHE LA PATENTE, JE SAIS JUSTE PAS QUAND. J'HAIS ÇA GOSSER DE MÊME.
https://kubernetes.io/docs/concepts/storage/storage-classes/
You can create your own provisioner, then a storageClassName that uses the provisioner
As far as persistence, that is controlled by the policy you set on the reclaimPolicy
So its basically an order of:
- provisioner (netapp, glusterfs, whatever)
- storageClass (how you references that provisioner & other settings you need to make it work the way you want)
- persistent volume (creating the actual volume, which the provisioner takes care of under the hood, also where you define sizes & usage, readwrite, reclaimPolicy)
- persistent volume claim (where you actually use the PV in your pods)
It won't be scheduled to a node without the spare ram if you have appropriate memory requests set on the pod: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
A limit of 3 per node isn't dynamic, what if one node has twice the ram?
You can specify a limit to the resources that a container may access.
You might enjoy this part of the docs. There are examples...
https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/
Hey yes I would take a look at the official docs https://kubernetes.io/docs/reference/ports-and-protocols/ It can sometimes be hard to find things in the docs but I would recommend getting familiar with them if you are intending to use kube especially in places where you are worried about specifics like ports in this case. Good luck!
It's native Windows containers without Linux stuff. We're not talking about WSL2. Some kubernetes products already support Windows containers in production. https://kubernetes.io/docs/setup/production-environment/windows/intro-windows-in-kubernetes/
If creating a service imperitavely, I would go with `kubectl expose`: https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#expose
I guess in your case you are just creating the service but not the endpoints for it.
You should not use environment variables for secrets. They can be inspected and leaked in a number of ways.
Here's the former head of security at Docker on this subject: https://diogomonica.com/2017/03/27/why-you-shouldnt-use-env-variables-for-secret-data/
Swarm and k8s have their own Secrets mechanisms that are more secure:
https://docs.docker.com/engine/swarm/secrets/ https://kubernetes.io/docs/concepts/configuration/secret/
Just wanted to add here that your namespace may have some LimitRange[1] assigned to it, which means that if you don't set resources in your pod you may get some defaults applied to it anyway. This is good practice specially in multi-tenant clusters.
I tend to set strict LimitRange to my clusters so it "encourage" developers to specify theirs apps resources properly or get throttled/OOMKilled often. :)
Becareful with >10 deprecated apis removed https://kubernetes.io/blog/2021/08/04/kubernetes-1-22-release-announcement/#removal-of-several-deprecated-beta-apis (>10 seems a bit high to still use the word several....)
What I usually do is label namespaces with name: <namespace-name> and then I can use the following construct everywhere that needs a <code>LabelSelector</code> (e.g. NetworkPolicy):
namespaceSelector: matchLabels: name: <source-ns>
When I took the exam, I just went ahead and labelled the source namespace with kubectl label ns <source-ns> name=<source-ns> :D
OP ignore all the others that say you’re being stupid. Follow this:
https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/
Kubeadm is the official installer. Once you remove the noschedule taint you can deploy work to it. The biggest things to know here:
1) Make sure to take backups and ship them somewhere if the etcd cluster created by Kubeadm, it holds all your state.
2) Since your work is running on the same as the master, your performance under heavy load will suffer. Just be aware.
3) Since your work is running on the same as master, your security surface area is much larger and well.
4) When you do scale out, skip to 3 nodes. Etcd needs it for quorum and you’ll have a bad time with just 2. That is assuming you scale out your masters as well.
I’ve been deploying k8s to bare metal for a few years, happy to answer any questions.
Persistent volumes on AWS have node affinities set that will ensure the pod is scheduled in the correct zone, assuming everything is configured correctly.
K8S is explicitly designed with no maintenance model. It's designed with rip/replace in mind; if you leave a k8s cluster up for 6+ months you're literally doing it wrong.
Or right from the official site: https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/#what-kubernetes-is-not
"Does not provide nor adopt any comprehensive machine configuration, maintenance, management, or self-healing systems."
Even hosted to solve the rebuild, your still need CICD to get the containers on the new cluster ... Because I'm certainly not taking on that technical debt.
No... You don't play with K8S unless you plan full CICD.
Oh fu.. I mixed it up. This is on their Docs
​
"Minor releases occur approximately every 3 months, so each minor release branch is maintained for approximately 9 months"
​
Check this : https://kubernetes.io/docs/setup/version-skew-policy/
​
You will only see most of the components (Kubectl and kubeapiserver) supports only upto 1.12
​
I guess your on the edge of the cliff mate. They only support three minor version from the current version. 1.14, 1.13 and 1.12,
​
Better to upgrade right now.
-- Going through Documentation and practicing all the tasks twice
-- Kubernetes in Action by Marko Luksa
-- kubectl --dry-run --save-config is your friend
-- Go thru kubectl cheat sheet and practice creating all k8s objects (Deployments,services,bare pods,etc )on the commandline using kubectl than prepping up yaml files from scratch.
​
During exam ,
-- Time management is important . You dont know a question , skip it and come back at last.
-- You can use documentation so use search functionality in kubernetes.io wisely and widely.
All the best for the exam.
at least with k8s the true way seems to be ConfigMaps: https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap
we use these (with openshift) for injecting any sort of configuration on top of a base image
It seems that your comment contains 1 or more links that are hard to tap for mobile users. I will extend those so they're easier for our sausage fingers to click!
Here is link number 1 - Previous text "pod"
^Please ^PM ^/u/eganwall ^with ^issues ^or ^feedback! ^| ^Delete
Francesc just did a great episode of Just For Func that teaches Cobra and how to build CLI utilities using it!
https://www.youtube.com/watch?v=WvWPGVKLvR4
For OP's question of the design patterns - I really like the syntax of kubectl, the kubernetes command line utility.
So, there's tunable behavior as to how CronJobs actually deal with delay and failure, and the defaults aren't always the best option, depending on your workload. There are a couple knobs that you can tune, IIRC, but the first one to look at would probably be <code>concurrencyPolicy</code>, which controls what happens if a CronJob would have two running at the same time.
The default solution, developed by Kubernetes project itself is Kubernetes Dashboard (https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/).
There are also a few third-party UI solutions, where my personal preference is the Scope by Weave (https://github.com/weaveworks/scope).
I'm using Rancher for pretty much 90% of my homelab. It takes care of everything and provides a useful web interface and is still able to use your docker-compose files. I also use GitLab as container registry for services that don't have official containers yet.
With Rancher 2.0 they will switch over to running a full Kubernetes cluster in the background, which is de facto the industry standard for container orchestration.
I still use Proxmox to host multiple nodes for Rancher and openmediavault to provide storage for the containers via NFS.
They also offer RancherOS which is a lightweight OS made entirely from Docker containers. I'm running it on a few nodes with no issues so far.
For 1)
https://kubernetes.io/docs/concepts/configuration/assign-pod-node/ There's an easier way though. Just set your request cpu and memory to be as high as one full node, and no pods will be scheduled on the same node.
2)
Running in docker or not doesn't solve the horizontal scaling problem, but it does solve the resiliency. One node dies, another one takes over with no human interaction. Mix that with proxysql and mysql orchestrator (which you would do anyway if you're not running in kubernetes), and you have something quite nice.
3)
it's called Kubernetes Federation. There's no magic formula for this, making a mysql cross region work with or without docker/kubernetes is as hard. My feeling is that, if you need to go cross region, it would probably make sense to start sharing your database. Until then, multi zone is fine (and easy)
Thats a great shout, thanks. To be honest, I don't have any readiness and liveness probes deployed, only healthecks for the Services.
Looking at this doc it seems like readiness probe is what I need. But I'll deploy liveness probes as well, they are super useful.
Check out https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/ .
Here is a little snipplet of a migration job I use: ####### # Migration job ######## --- apiVersion: batch/v1 kind: Job metadata: generateName: migration- labels: job: migration spec: template: metadata: labels: job: migration spec: containers: - name: server image: XXXXXXXX.dkr.ecr.us-west-2.amazonaws.com/someCoolContainer:1.0 command: ["./bin/migrate.js"] ... ...