The usage of Helm Hooks

This post describes how to deal with the situation when the chart templates have dependencies. We can easily solve the problem by Helm hooks. Using ServiceAccount and Secret as the example.

Story

I used to deploy the service accounts and service account secrets by Helm, and I encountered Failed to deploy a secret with type kubernetes.io/service-account-token, after diving into deep, I figured out what was happened in my Kubernetes cluster.

We have a bunch of configuration files that can configure the Kubernetes cluster. For example, we are going to build a Kubernetes cluster with Multus support, we’ll need to have a Multus service account and configure it. After the configuration, we can make Multus has the ability to orchestrate Kubernetes container networking.

Therefore, we have some essential objects need to be created:

  1. Service account: multus-sa
  2. Service account secret: multus-sa-secret
  3. Cluster role for RBAC: multus-pod-networks-lister
  4. Cluster role binding: multus-rb

Problems

The Helm client renders templates automatically when deployment, but the sequence of rendered files to apply is not consistent. And this will cause a problem when we have a dependency between 2 objects.

For example, we created ServiceAccount before the Secret. But when we actually deployed to Kubernetes cluster, Helm will deploy the Secret before ServiceAccount. And it leads to a fault, Secret can’t bind to the correct ServiceAccount, the data in secret will be missing.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: multus-sa
namespace: kube-system
---
apiVersion: v1
kind: Secret
metadata:
name: multus-sa-secret
namespace: kube-system
annotations:
kubernetes.io/service-account.name: multus-sa
type: kubernetes.io/service-account-token

Helm Hooks

Therefore, we need to use Helm Hooks annotations pre-install to declare which object should be created before every objects else. But once the object is applied to these hooks, it’s not managed by Helm, so you can’t use helm delete <release-name> --purge to delete it.

However, you can also declare by assigning before-hook-creation to hook-delete-policy. And when Helm is trying to deploy a chart, it’ll check if the object exists or not, deletes the object if it’s in Kubernetes cluster.

1
2
3
4
5
6
7
8
9
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: multus-sa
namespace: kube-system
annotations:
"helm.sh/hook": "pre-install"
"helm.sh/hook-delete-policy": "before-hook-creation"

There are other available hooks can satisfy more needs, you can find all hooks in Helm Docs - The available hooks. And there is one hook looks very useful for me, so I’ll describe and write the example code here.

When we created the CustomResourceDefinition, sometimes we need the CRDs created before anything else, because we have some objects depend on the CRD resources. In this kind of situation, use the crd-install to declare this CRD resource is depended on by other objects.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
# name must match the spec fields below, and be in the form: <plural>.<group>
name: network-attachment-definitions.k8s.cni.cncf.io
annotations:
"helm.sh/hook": "crd-install"
"helm.sh/hook-delete-policy": "before-hook-creation"
spec:
# group name to use for REST API: /apis/<group>/<version>
group: k8s.cni.cncf.io
# version name to use for REST API: /apis/<group>/<version>
version: v1
# either Namespaced or Cluster
scope: Namespaced
names:
# plural name to be used in the URL: /apis/<group>/<version>/<plural>
plural: network-attachment-definitions
# singular name to be used as an alias on the CLI and for display
singular: network-attachment-definition
# kind is normally the CamelCased singular type. Your resource manifests use this.
kind: NetworkAttachmentDefinition
# shortNames allow shorter string to match your resource on the CLI
shortNames:
- net-attach-def

Reference

GitHub - Failed to deploy a secret with type kubernetes.io/service-account-token
Helm Docs - The available hooks
GitHub - CloudNativeSetup