202503141441 Infisical Operator For Homelab

I’ve been looking at a vew GitOps tooling such as FluxCD and ArgoCD to manage my application deployments once I am done with the migration state. With a tool like Flux or Argo, anytime I do require a version update or some changes in configurations – it will be automated entirely.

This will also support me in segregating my clusters into various environments to which currently, I treat my cluster as prod.

While investigating into both CD tooling, I also wanted to integrate it with Tanka Grafana which is my preferred way to defining Kubernetes YAML’s via Jsonnet and also with the k8s-libsonnet abstraction. Both CD tooling aren’t exactly tightly integrate to Tanka as tanka is merely a manifest generation tooling. However, I’ve discovered that with Tanka, you could also export your defined jsonnets.

This is a lifesaver because both CD tooling work mostly with “pure” YAML manifests and in case of FluxCD (at least based on the research I have done so far) – it uses Kustomize to generate and diff the YAMLs.

However, one problem I faced while doing a dry-run is that my secrets are managed locally and generated as YAML’s. This is not ideal – I do not want my secrets to be outputted.

Here’s an example

Running tk export yaml/ environments/home --merge-strategy replace-envs outputs the YAMLs into the yaml/ I’ve made in.

It outputs these files which is lovely.

However, it also outputs secrets as a Kubernetes YAML which is not ideal. These YAML’s thus cannot be pushed to Git for version control. Therefore, in its present state – storing secrets as static values in a local file is not a great idea.

Infisical Operator To The Rescue!

At work, we use a similar secrets management tool called Doppler. However, I wanted to try something entirely different and a SaaS that can be hosted in my own premises. I’ve opted to go with Infisical as I’ve had some prior experience last year testing it. My initial response to it was good and I was eager to try it again once I had my own Kubernetes cluster.

I’ve deployed Infisical into my “private” VPS after deploying Coolify. You might want to check out the writeup I did here <TODO: ADD THE LINK TO COOLIFY DOCS>.

With a Kubernetes operator, managing of the secrets becomes a lot easier. Firstly, I can store all my secrets outside of the cluster – ensuring that if the cluster does go down, the secrets do not disappear either. Furthermore, this decouples the secrets from the application workloads altogether which is favourable considering Infisical will be the main tool to manage all my secrets down the line. The cluster would therefore just remain as a way to provision and manage application workloads.

I first installed it as a Helm chart followed by considering the InfisicalSecret CRD. It was quite a breeze!

Afterwards the next step was to ensure the InfisicalSecret CRD can be a reusable manifest. If I were to adopt a static manifest approach, each InfisicalSecret YAML would be stored together with the application configuration manifests.

Naturally that led me to believe I could introduce an abstraction to programmatically produce InfisicalSecret objects to sync and create kubernetes secrets. By using Jsonnet with Tanka, I can elevate this to the next level. I find this pattern to be nice as i’ve colocated the secrets instantiation as close as I can to the application deployment configuration.

This approach is not something I came up rather, it is an approach we take at work as well.

Infiscal Secret CRD Builder

Define an object that can output a InfiscalSecret object.

infSecBuilder:: {
  sec(config):: {
    local namespace = config.namespace,
    local secretName = config.secretName,
    local projectSlug = config.projectSlug,

    infisicalSecretCRD: {
      apiVersion: 'secrets.infisical.com/v1alpha1',
      kind: 'InfisicalSecret',
      metadata: {
        name: 'infisical-' + secretName,
      },
      spec: {
        hostAPI: $._config.configMap.configMap.hostAPI,
        resyncInterval: 10,
        authentication: {
          universalAuth: {
            credentialsRef: {
              secretName: 'infisical-universal-auth-credentials',
              secretNamespace: namespace,
            },
            secretsScope: {
              envSlug: 'prod',
              secretsPath: '/',
              projectSlug: projectSlug,
            },
          },
        },
        managedKubeSecretReferences: [
          {
            secretName: secretName,
            secretNamespace: namespace,
            creationPolicy: 'Orphan',
          },
        ],
      },
    },
  },
},

Note that this abstraction is not the best. For instance, some fields are just static values that should be dynamic such as the envSlug.

However, my point is this – we could use Infisical Secrets operator as a way to sync secrets to the cluster. This greatly unlocks my Kubernetes cluster to move closer towards GitOps and achieving the goal of 202501111134 A Portable Homelab