GitLab CI template for Helmfile¶

This project implements a GitLab CI/CD template to deploy your application to a Kubernetes platform using helmfile.

Usage¶

This template can be used both as a CI/CD component or using the legacy include:project syntax.

Use as a CI/CD component¶

Add the following to your .gitlab-ci.yml:

include:
  # 1: include the component
  - component: $CI_SERVER_FQDN/to-be-continuous/helmfile/gitlab-ci-helmfile@3.2.4
    # 2: set/override component inputs
    inputs:
      # ⚠ this is only an example
      base-app-name: wonderapp
      review-enabled: true
      staging-enabled: true
      prod-enabled: true

Use as a CI/CD template (legacy)¶

Add the following to your .gitlab-ci.yml:

include:
  # 1: include the template
  - project: 'to-be-continuous/helmfile'
    ref: '3.2.4'
    file: '/templates/gitlab-ci-helmfile.yml'

variables:
  # 2: set/override template variables
  # ⚠ this is only an example
  HELMFILE_BASE_APP_NAME: wonderapp
  HELMFILE_REVIEW_ENABLED: "true"
  HELMFILE_STAGING_ENABLED: "true"
  HELMFILE_PROD_ENABLED: "true"

Understand¶

This chapter introduces key notions and principle to understand how this template works.

Managed deployment environments¶

This template implements continuous delivery/continuous deployment using helmfile for projects hosted on Kubernetes platforms.

It allows you to manage automatic deployment & cleanup of standard predefined environments. Each environment can be enabled/disabled by configuration. If you're not satisfied with predefined environments and/or their associated Git workflow, you may implement you own environments and workflow, by reusing/extending the base (hidden) jobs. This is advanced usage and will not be covered by this documentation.

The following chapters present the managed predefined environments and their associated Git workflow.

Review environments¶

The template supports review environments: those are dynamic and ephemeral environments to deploy your ongoing developments (a.k.a. feature or topic branches).

When enabled, it deploys the result from upstream build stages to a dedicated and temporary environment. It is only active for non-production, non-integration branches.

It is a strict equivalent of GitLab's Review Apps feature.

It also comes with a cleanup job (accessible either from the environments page, or from the pipeline view).

Integration environment¶

If you're using a Git Workflow with an integration branch (such as Gitflow), the template supports an integration environment.

When enabled, it deploys the result from upstream build stages to a dedicated environment. It is only active for your integration branch (develop by default).

Production environments¶

Lastly, the template supports 2 environments associated to your production branch (main or master by default):

a staging environment (an iso-prod environment meant for testing and validation purpose),
the production environment.

You're free to enable whichever or both, and you can also choose your deployment-to-production policy:

continuous deployment: automatic deployment to production (when the upstream pipeline is successful),
continuous delivery: deployment to production can be triggered manually (when the upstream pipeline is successful).

Deployment context variables¶

In order to manage the various deployment environments, this template provides a couple of dynamic variables that you might use in your hook scripts and helmfile templates.

The environment variables are available in the helmfile template with the {{ env "<ENV_VAR_NAME>" }} directive.

environment variable	template directive	Description
`$environment_name`	`{{ env "environment_name" }}`	a generated application name to use for the current deployment environment (ex: `myproject-review-fix-bug-12` or `myproject-staging`). This should be used as release name in the `helmfile.yaml` - details below
`$environment_type`	`{{ env "environment_type" }}`	the current deployment environment type (`review`, `integration`, `staging` or `production`)
`$environment_url`	`{{ env "environment_url" }}`	the environment URL (see also environment url management)
`$environment_hostname`	`{{ env "environment_hostname" }}`	the environment hostname, extracted from the environment URL
`$kube_namespace`	`{{ .Namespace }}`	the Kubernetes namespace currently used for deployment/cleanup

The $environment_type is also passed as --environment parameter to helmfile which allows you to inject a set of values specific to the environment type (see helmfile environment values).

The docker image output variables passed by the Docker template (docker_image, docker_tag, ...) will also be available with the env directive in the helmfile template (e.g. {{ env "docker_tag" }}).

Example use in helmfile manifest¶

Below a helmfile.yaml example illustrating the use of environment variables:

environments:
  default:
  integration:
  review:
  production:
---
repositories:
  - name: mycharts
    url: https://repo.example.io/mycharts/stable
releases:
- name: {{ env "environment_name" | Default "myapp" }}
  chart: mycharts/webapp
  version: "1.2.0" 
  namespace: {{ .Namespace }}  
  values:
  - image:
      registry: {{ regexReplaceAll "^([^/]+)\\/([^:^@]+)(@|:|)(.*)$" (env "docker_image") "${1}" }}
      repository: {{ regexReplaceAll "^([^/]+)\\/([^:^@]+)(@|:|)(.*)$" (env "docker_image") "${2}" }}
      {{- if eq .Environment.Name "production" }}
      tag: {{ env "docker_tag" }}
      {{- else }}
      digest: {{ env "docker_digest" }}
      {{- end }}
    ingress:
      enabled: true
      hostname: {{ env "environment_hostname" }}

Generated environment name¶

The ${environment_name} variable is generated to designate each deployment environment with a unique and meaningful release and application name. By construction, it is suitable for inclusion in DNS, URLs, Kubernetes labels...

It is built from:

the application base name (defaults to $CI_PROJECT_NAME but can be overridden globally and/or per deployment environment - see configuration variables)
GitLab predefined $CI_ENVIRONMENT_SLUG variable (sluggified name, truncated to 24 characters)

The ${environment_name} variable is then evaluated as:

<app base name> for the production environment
<app base name>-$CI_ENVIRONMENT_SLUG for all other deployment environments
${environment_name} can also be overriden per environment with the appropriate configuration variable

Examples (with an application's base name myapp):

`$environment_type`	Branch	`$CI_ENVIRONMENT_SLUG`	`$environment_name`
`review`	`feat/blabla`	`review-feat-bla-xmuzs6`	`myapp-review-feat-bla-xmuzs6`
`integration`	`develop`	`integration`	`myapp-integration`
`staging`	`main`	`staging`	`myapp-staging`
`production`	`main`	`production`	`myapp`

Deployment and cleanup scripts¶

The Helmfile template requires you to provide a helmfile.yaml file to deploy and delete the application.

The environment deployment is processed as follows:

optionally executes the helmfile-pre-deploy.sh script in your project to perform specific environment pre-initialization (for e.g. create required services),
helmfile apply with $environment_type as value for the --environment parameter,
optionally executes the helmfile-post-deploy.sh script in your project to perform specific environment post-initialization stuff,

The environment deletion is processed as follows:

optionally executes the helmfile-pre-delete.sh script in your project to perform specific environment pre-cleanup stuff,
helmfile destroy with $environment_type as value for the --environment parameter,
optionally executes the helmfile-post-delete.sh script in your project to perform specific environment post-cleanup (for e.g. delete bound services).

Using variables¶

You have to be aware that your helmfile state files and deployment (and cleanup) scripts have to be able to cope with various environments (review, integration, staging and production), each with different application names, exposed routes, settings, ... Most of the complexity can be handled by helmfile by using helmfile environments, environment variables, and templating. All environment variables are at your disposal within your helmfile templates:

deployment context variables provided by the CI/CD template,
any GitLab CI variable
any custom variable (ex: ${SECRET_TOKEN} that you have set in your project CI/CD variables)

The deployment context variable $environment_type is passed as helmfile environment and can be used to select environment values and customize releases per deployment environment type.

Environments URL management¶

The Helmfile template supports two ways of providing your environments url:

a static way: when the environments url can be determined in advance, probably because you're exposing your routes through a DNS you manage,
a dynamic way: when the url cannot be known before the deployment job is executed.

The static way can be implemented simply by setting the appropriate configuration variable(s) depending on the environment (see environments configuration chapters):

$HELMFILE_ENVIRONMENT_URL to define a default url pattern for all your envs,
$HELMFILE_REVIEW_ENVIRONMENT_URL, $HELMFILE_INTEG_ENVIRONMENT_URL, $HELMFILE_STAGING_ENVIRONMENT_URL and $HELMFILE_PROD_ENVIRONMENT_URL to override the default.

Each of those variables support a late variable expansion mechanism with the %{somevar} syntax, allowing you to use any dynamically evaluated variables such as ${environment_name}.

Example:

variables:
  HELMFILE_BASE_APP_NAME: "wonderapp"
  # global url for all environments
  HELMFILE_ENVIRONMENT_URL: "https://%{environment_name}.nonprod.acme.domain"
  # override for prod (late expansion of $HELMFILE_BASE_APP_NAME not needed here)
  HELMFILE_PROD_ENVIRONMENT_URL: "https://$HELMFILE_BASE_APP_NAME.acme.domain"
  # override for review (using separate resource paths)
  HELMFILE_REVIEW_ENVIRONMENT_URL: "https://wonderapp-review.nonprod.acme.domain/%{environment_name}"

To implement the dynamic way, your deployment script shall simply generate a environment_url.txt file in the working directory, containing only the dynamically generated url. When detected by the template, it will use it as the newly deployed environment url.

Deployment output variables¶

Each deployment job produces output variables that are propagated to downstream jobs (using dotenv artifacts):

$environment_type: set to the type of environment (review, integration, staging or production),
$environment_name: the application name (see below),
$environment_url: set to the environment URL (whether determined statically or dynamically).

Those variables may be freely used in downstream jobs (for instance to run acceptance tests against the latest deployed environment).

You may also add and propagate your own custom variables, by pushing them to the helmfile.env file in your deployment script.

Configuration reference¶

Secrets management¶

Here are some advices about your secrets (variables marked with a ):

Manage them as project or group CI/CD variables:
- masked to prevent them from being inadvertently displayed in your job logs,
- protected if you want to secure some secrets you don't want everyone in the project to have access to (for instance production secrets).
In case a secret contains characters that prevent it from being masked, simply define its value as the Base64 encoded value prefixed with @b64@: it will then be possible to mask it and the template will automatically decode it prior to using it.
Don't forget to escape special characters (ex: $ -> $$).

Global configuration¶

The Helm template uses some global configuration used throughout all jobs.

Input / Variable	Description	Default value
`cli-image` / `HELMFILE_CLI_IMAGE`	The Docker image used to run helmfile set the version required by your Kubernetes server	`ghcr.io/helmfile/helmfile:latest`
`path` / `HELMFILE_PATH`	The path to your `helmfile.yaml`	`./helmfile.yaml`
`scripts-dir` / `HELMFILE_SCRIPTS_DIR`	The folder where hook scripts are located	`.` (root project dir)
`kube-namespace` / `KUBE_NAMESPACE`	The default Kubernetes namespace to use	`"${CI_PROJECT_NAME}-${CI_PROJECT_ID}-${CI_ENVIRONMENT_SLUG}"` (see GitLab doc)
`KUBE_CONTEXT`	Defines the context to be used in `KUBECONFIG`. When using GitLab agents with the CI/CD workflow, the value should be like `path/to/agent/project:agent-name`. To use different agents per environment, define an environment-scoped CI/CD variable for each agent.	none
`HELMFILE_DEFAULT_KUBE_CONFIG`	The default kubeconfig to use (either content or file variable)	`$KUBECONFIG` (thus supports the GitLab Kubernetes integration when enabled)
`deploy-args` / `HELMFILE_DEPLOY_ARGS`	The helmfile command with options to deploy the application (without dynamic global parameters such as `helmfile.yaml` path and environment name)	`apply --wait`
`delete-args` / `HELMFILE_DELETE_ARGS`	The helmfile command with options to cleanup the application (without dynamic global parameters such as `helmfile.yaml` path and environment name)	`destroy`
`base-app-name` / `HELMFILE_BASE_APP_NAME`	Base application name	`$CI_PROJECT_NAME` (see GitLab doc)
`environment-url` / `HELMFILE_ENVIRONMENT_URL`	Default environments url (only define for static environment URLs declaration) supports late variable expansion (ex: `https://%{environment_name}.helm.acme.com`)	none
`HELMFILE_PGP_PRIVATE_KEY_FILE`	PGP Private key for decrypting helmfile secrets (optional)	none
`HELMFILE_PGP_PASSPHRASE`	Passphrase for PGP private key (optional)	none
`image-pull-secret-name` / `HELMFILE_IMAGE_PULL_SECRET_NAME`	Name of the `docker-registry` k8s secret that will be created if the special GitLab deploy token is available.	`gitlab-registry`

Review environments configuration¶

Review environments are dynamic and ephemeral environments to deploy your ongoing developments (a.k.a. feature or topic branches).

They are disabled by default and can be enabled by setting the HELMFILE_REVIEW_ENABLED variable (see below).

Here are variables supported to configure review environments:

Input / Variable	Description	Default value
`review-enabled` / `HELMFILE_REVIEW_ENABLED`	Set to `true` to enable `review` env	none (disabled)
`review-app-name` / `HELMFILE_REVIEW_APP_NAME`	Application name for `review` env	`"${HELMFILE_BASE_APP_NAME}-${CI_ENVIRONMENT_SLUG}"` (ex: `myproject-review-fix-bug-12`)
`review-environment-url` / `HELMFILE_REVIEW_ENVIRONMENT_URL`	The review environments url (only define for static environment URLs declaration and if different from default)	`$HELMFILE_ENVIRONMENT_URL`
`review-namespace` / `HELMFILE_REVIEW_NAMESPACE`	The Kubernetes namespace to use for `review` env (only define to override default)	`$KUBE_NAMESPACE`
`HELMFILE_REVIEW_KUBE_CONFIG`	Specific kubeconfig for `review` env (only define to override default)	`$HELMFILE_DEFAULT_KUBE_CONFIG`
`review-autostop-duration` / `HELMFILE_REVIEW_AUTOSTOP_DURATION`	The amount of time before GitLab will automatically stop `review` environments	`4 hours`

Integration environment configuration¶

The integration environment is the environment associated to your integration branch (develop by default).

It is disabled by default and can be enabled by setting the HELMFILE_INTEG_ENABLED variable (see below).

Here are variables supported to configure the integration environment:

Input / Variable	Description	Default value
`integ-enabled` / `HELMFILE_INTEG_ENABLED`	Set to `true` to enable `integration` env	none (disabled)
`integ-app-name` / `HELMFILE_INTEG_APP_NAME`	Application name for `integration` env	`$HELMFILE_BASE_APP_NAME-integration`
`integ-environment-url` / `HELMFILE_INTEG_ENVIRONMENT_URL`	The integration environment url (only define for static environment URLs declaration and if different from default)	`$HELMFILE_ENVIRONMENT_URL`
`integ-namespace` / `HELMFILE_INTEG_NAMESPACE`	The Kubernetes namespace to use for `integration` env (only define to override default)	`$KUBE_NAMESPACE`
`HELMFILE_INTEG_KUBE_CONFIG`	Specific kubeconfig for `integration` env (only define to override default)	`$HELMFILE_DEFAULT_KUBE_CONFIG`
`integ-autostop-duration` / `HELMFILE_INTEG_AUTOSTOP_DURATION`	The amount of time before GitLab will automatically stop the `integration` env	`never`

Staging environment configuration¶

The staging environment is an iso-prod environment meant for testing and validation purpose associated to your production branch (main or master by default).

It is disabled by default and can be enabled by setting the HELMFILE_STAGING_ENABLED variable (see below).

Here are variables supported to configure the staging environment:

Input / Variable	Description	Default value
`staging-enabled` / `HELMFILE_STAGING_ENABLED`	Set to `true` to enable `staging` env	none (disabled)
`staging-app-name` / `HELMFILE_STAGING_APP_NAME`	Application name for `staging` env	`$HELMFILE_BASE_APP_NAME-staging`
`staging-environment-url` / `HELMFILE_STAGING_ENVIRONMENT_URL`	The staging environment url (only define for static environment URLs declaration and if different from default)	`$HELMFILE_ENVIRONMENT_URL`
`staging-namespace` / `HELMFILE_STAGING_NAMESPACE`	The Kubernetes namespace to use for `staging` env (only define to override default)	`$KUBE_NAMESPACE`
`HELMFILE_STAGING_KUBE_CONFIG`	Specific kubeconfig for `staging` env (only define to override default)	`$HELMFILE_DEFAULT_KUBE_CONFIG`
`staging-autostop-duration` / `HELMFILE_STAGING_AUTOSTOP_DURATION`	The amount of time before GitLab will automatically stop the `staging` env	`never`

Production environment configuration¶

The production environment is the final deployment environment associated with your production branch (main or master by default).

It is disabled by default and can be disabled by setting the HELMFILE_PROD_ENABLED variable (see below).

Here are variables supported to configure the production environment:

Input / Variable	Description	Default value
`HELMFILE_PROD_ENABLED`	Set to `true` to enable `production` env	none (disabled)
`prod-app-name` / `HELMFILE_PROD_APP_NAME`	Application name for `production` env	`$HELMFILE_BASE_APP_NAME`
`prod-environment-url` / `HELMFILE_PROD_ENVIRONMENT_URL`	The production environment url (only define for static environment URLs declaration and if different from default)	`$HELMFILE_ENVIRONMENT_URL`
`prod-namespace` / `HELMFILE_PROD_NAMESPACE`	The Kubernetes namespace to use for `production` env (only define to override default)	`$KUBE_NAMESPACE`
`HELMFILE_PROD_KUBE_CONFIG`	Specific kubeconfig for `production` env (only define to override default)	`$HELMFILE_DEFAULT_KUBE_CONFIG`
`prod-deploy-strategy` / `HELMFILE_PROD_DEPLOY_STRATEGY`	Defines the deployment to production strategy. One of `manual` (i.e. one-click) or `auto`.	`manual`

`helmfile-lint` job¶

This job runs a helm lint across all of the charts/releases in the helmfile manifest and uses the following variables:

Input / Variable	Description	Default value
`lint-enabled` / `HELMFILE_LINT_ENABLED`	Set to `true` to enable Helmfile lint	none (disabled)
`lint-args` / `HELMFILE_LINT_ARGS`	The helmfile command with options to trigger the analysis	`lint`

`helmfile-test` job¶

This job runs Helm tests against the specified releases in the helmfile manifest.

It is disabled by default and can be enabled by setting the HELMFILE_TEST_ENABLED variable (see below).

It uses the following variables:

Input / Variable	Description	Default value
`test-enabled` / `HELMFILE_TEST_ENABLED`	Set to `true` to enable Helm test	none (disabled)
`test-args` / `HELMFILE_TEST_ARGS`	The helmfile command with options
to perform acceptance test (without dynamic global arguments such as the helmfile.yaml path, namespace and environment name)	`test`

Variants¶

Vault variant¶

This variant allows delegating your secrets management to a Vault server.

Configuration¶

In order to be able to communicate with the Vault server, the variant requires the additional configuration parameters:

Input / Variable	Description	Default value
`TBC_VAULT_IMAGE`	The Vault Secrets Provider image to use (can be overridden)	`registry.gitlab.com/to-be-continuous/tools/vault-secrets-provider:latest`
`vault-base-url` / `VAULT_BASE_URL`	The Vault server base API url	none
`vault-oidc-aud` / `VAULT_OIDC_AUD`	The `aud` claim for the JWT	`$CI_SERVER_URL`
`VAULT_ROLE_ID`	The AppRole RoleID	must be defined
`VAULT_SECRET_ID`	The AppRole SecretID	must be defined

Usage¶

Then you may retrieve any of your secret(s) from Vault using the following syntax:

@url@http://vault-secrets-provider/api/secrets/{secret_path}?field={field}

With:

Parameter	Description
`secret_path` (path parameter)	this is your secret location in the Vault server
`field` (query parameter)	parameter to access a single basic field from the secret JSON payload

Example¶

include:
  # main template
  - component: $CI_SERVER_FQDN/to-be-continuous/helmfile/gitlab-ci-helmfile@3.2.4
  # Vault variant
  - component: $CI_SERVER_FQDN/to-be-continuous/helmfile/gitlab-ci-helmfile-vault@3.2.4
    inputs:
      # audience claim for JWT
      vault-oidc-aud: "https://vault.acme.host"
      vault-base-url: "https://vault.acme.host/v1"
      # $VAULT_ROLE_ID and $VAULT_SECRET_ID defined as a secret CI/CD variable

variables:
  # Secrets managed by Vault
  HELMFILE_DEFAULT_KUBE_CONFIG: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/my-app/helmfile/noprod?field=kube_config"
  HELMFILE_PROD_KUBE_CONFIG: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/my-app/helmfile/prod?field=kube_config"

GitLab CI template for Helmfile¶

Usage¶

Use as a CI/CD component¶

Use as a CI/CD template (legacy)¶

Understand¶

Managed deployment environments¶

Review environments¶

Integration environment¶

Production environments¶

Deployment context variables¶

Example use in helmfile manifest¶

Generated environment name¶

Deployment and cleanup scripts¶

Using variables¶

Environments URL management¶

Deployment output variables¶

Configuration reference¶

Secrets management¶

Global configuration¶

Review environments configuration¶

Integration environment configuration¶

Staging environment configuration¶

Production environment configuration¶

helmfile-lint job¶

helmfile-test job¶

Variants¶

Vault variant¶

Configuration¶

Usage¶

Example¶

`helmfile-lint` job¶

`helmfile-test` job¶