As a devops/SRE maintaining and managing the yaml seems a cumbersome task. There is great a challenging in maintaining highly readable yaml while keeping it simple. Helm and Kustomize are the two amazing tool to maintain these yaml. Both of these comes with amazing features choosing one over another is very difficult task in itself as the tradeoff is pretty high. So how about have both.
Another important challenge to maintain in multiple environment system to to maintain hierarchal yaml structure, such that the repeatitive values are maintained as base and values for each environment overlayed on top of it.
As in above the single app (cert-manager in our case with jetstack helm chart) is broken in to two folders as
overlays, helm_base folder comprises of the helm chart definition in Chart.yaml file as
apiVersion: v2 description: A Helm chart for cert-manager name: cert-manager type: application version: v1.3.1 dependencies: — name: cert-manager version: v1.3.1 repository: https://charts.jetstack.io
Note that the repository is defined as the dependencies this approach is helpful for defining multiple charts and segregating the values for it. In addition to the Chart definition it also comprises
kustomization.yml this defines the resource definitions to be applied.
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization commonLabels: app: cert-manager resources: - all.yml
You may notice the resources section which mentions the
all.yml which does not exist we will get back to it later. commonLabels section adds this label to all the resources defined in
Next folder is the overlay folder which contains the kustomize definition for each environment. The example above contains only for dev environment. For our
cert-manger chart we have a folder for each environment. This folder contains kustomization definitions, helm overrride values and additional resources. Lets get to the
apiVersion: kustomize.config.k8s.io/v1beta1 kind: Kustomization bases: - ../../../helm_base/cert-manager namespace: cert-manager resources: - namespace.yml - cluster-issuer.yaml
When a this configuration is applied it goes back to the helm_base directory to particular app folder and applies the kustomization on that folder in addition to that there are other resources which needs to be applied in resources sections. Like I mentioned below the kustomization folder in base contains a kustomize config which contains the resource to apply named all.yml which doesnot exist currently. This
all.yml file contains the resource definition which needs to be generated from the helm chart before we apply the overlay.
cd overlays/dev/cert-manager helm dependency build ../../../helm_base/cert-manager helm template ../../../helm_base/cert-manager --namespace cert-manager --name-template cert-manager -f values-override.yaml > ../../../helm_base/cert-manager/all.yml kubectl apply -k .
The last command applies all the resources in the all.yml and the additional resources defined in the resources section. Hence, we use the beautiful helm template as well as the powerful kustomize feature. Apart from this any resources generated from helm chart can be patch using kustomize which gives great power for changing the resources definition.