Kubernetes 1.6 – Taints and Tolerances for monitoring your cluster
At the office we use Datadog for monitoring our production systems, so when I spun up a test Kubernetes cluster (running Tectonic) I wanted to monitor using the same tool. Unfortunately the initial deploy of the Datadog agent didn’t work quite as well I was hoping. While I used their stock Kubernetes YAML as prescribed by the knowledgebase, it did not cover monitoring the Master nodes. Datadog support suggested I go another route (or host on docker based monitoring) which would have worked — but wasn’t as slick as I hoped. Fortunately just a few days ago I discovered the solution: Taints and Tolerances.
node-role.kubernetes.io/master:NoSchedule”. In the Datadog example I was working with, it’s just a 3 line change to their YAML, to get the agent deployed on ALL nodes of the Kubernetes cluster (not just the workers). Below is a full copy of the up-to-date YAML file which only requires the modification of the ‘
API_KEY’ to properly used.
Keep in mind, this trick will work with any type of daemonset. If you have other infrastructure support pieces that you’d like to run on all your cluster nodes, such as monitoring, data storage or log collection; the same few lines will get you going.