GitLab CI template for Kubernetes (k8s)¶

This project implements a GitLab CI/CD template to deploy your application to a Kubernetes platform using declarative configuration or Kustomize.

Usage¶

This template can be used both as a CI/CD component or using the legacy include:project syntax.

Use as a CI/CD component¶

Add the following to your .gitlab-ci.yml:

include:
  # 1: include the component
  - component: $CI_SERVER_FQDN/to-be-continuous/kubernetes/gitlab-ci-k8s@6.3.0
    # 2: set/override component inputs
    inputs:
      # ⚠ this is only an example
      kubectl-image: registry.hub.docker.com/bitnami/kubectl:1.26
      base-app-name: wonderapp
      review-space: myapp-nonprod
      staging-space: myapp-nonprod
      prod-space: myapp-prod

Use as a CI/CD template (legacy)¶

Add the following to your .gitlab-ci.yml:

include:
  # 1: include the template
  - project: 'to-be-continuous/kubernetes'
    ref: '6.3.0'
    file: '/templates/gitlab-ci-k8s.yml'

variables:
  # 2: set/override template variables
  # ⚠ this is only an example
  K8S_KUBECTL_IMAGE: registry.hub.docker.com/bitnami/kubectl:1.26
  K8S_BASE_APP_NAME: wonderapp
  K8S_REVIEW_SPACE: myapp-nonprod
  K8S_STAGING_SPACE: myapp-nonprod
  K8S_PROD_SPACE: myapp-prod

Understand¶

This chapter introduces key notions and principle to understand how this template works.

Managed deployment environments¶

This template implements continuous delivery/continuous deployment for projects hosted on Kubernetes platforms.

It allows you to manage automatic deployment & cleanup of standard predefined environments. Each environment can be enabled/disabled by configuration. If you're not satisfied with predefined environments and/or their associated Git workflow, you may implement you own environments and workflow, by reusing/extending the base (hidden) jobs. This is advanced usage and will not be covered by this documentation.

The following chapters present the managed predefined environments and their associated Git workflow.

Review environments¶

The template supports review environments: those are dynamic and ephemeral environments to deploy your ongoing developments (a.k.a. feature or topic branches).

When enabled, it deploys the result from upstream build stages to a dedicated and temporary environment. It is only active for non-production, non-integration branches.

It is a strict equivalent of GitLab's Review Apps feature.

It also comes with a cleanup job (accessible either from the environments page, or from the pipeline view).

Integration environment¶

If you're using a Git Workflow with an integration branch (such as Gitflow), the template supports an integration environment.

When enabled, it deploys the result from upstream build stages to a dedicated environment. It is only active for your integration branch (develop by default).

Production environments¶

Lastly, the template supports 2 environments associated to your production branch (master or main by default):

a staging environment (an iso-prod environment meant for testing and validation purpose),
the production environment.

You're free to enable whichever or both, and you can also choose your deployment-to-production policy:

continuous deployment: automatic deployment to production (when the upstream pipeline is successful),
continuous delivery: deployment to production can be triggered manually (when the upstream pipeline is successful).

Supported authentication methods¶

The Kubernetes template supports 3 ways of login/accessing your Kubernetes cluster(s):

Using GitLab agents with the CI/CD workflow: when enabled, the template automatically retrieves and uses your Kubernetes cluster configuration (KUBECONFIG env), don't forget to set the KUBE_CONTEXT variable (to path/to/agent/project:agent-name) as stated in the documentation.
By defining an explicit kubeconfig from env (either file or yaml content),
By defining explicit kubeconfig exploded parameters from env: server url, server certificate authority and user token.

Deployment context variables¶

In order to manage the various deployment environments, this template provides a couple of dynamic variables that you might use in your hook scripts, deployment manifests and other deployment resources:

${environment_type}: the current deployment environment type (review, integration, staging or production)
${environment_name}: a generated application name to use for the current deployment environment (ex: myproject-review-fix-bug-12 or myproject-staging) - details below
${environment_name_ssc}: the above application name in SCREAMING_SNAKE_CASE format (ex: MYPROJECT_REVIEW_FIX_BUG_12 or MYPROJECT_STAGING)
${k8s_namespace}: the Kubernetes namespace currently used for deployment/cleanup

Generated environment name¶

The ${environment_name} variable is generated to designate each deployment environment with a unique and meaningful application name. By construction, it is suitable for inclusion in DNS, URLs, Kubernetes labels... It is built from:

the application base name (defaults to $CI_PROJECT_NAME but can be overridden globally and/or per deployment environment - see configuration variables)
GitLab predefined $CI_ENVIRONMENT_SLUG variable (sluggified name, truncated to 24 characters)

The ${environment_name} variable is then evaluated as:

<app base name> for the production environment
<app base name>-$CI_ENVIRONMENT_SLUG for all other deployment environments
${environment_name} can also be overriden per environment with the appropriate configuration variable

Examples (with an application's base name myapp):

`$environment_type`	Branch	`$CI_ENVIRONMENT_SLUG`	`$environment_name`
`review`	`feat/blabla`	`review-feat-bla-xmuzs6`	`myapp-review-feat-bla-xmuzs6`
`integration`	`develop`	`integration`	`myapp-integration`
`staging`	`main`	`staging`	`myapp-staging`
`production`	`main`	`production`	`myapp`

Supported deployment methods¶

The Kubernetes template supports three techniques to deploy your code:

script-based deployment,
template-based deployment using raw Kubernetes manifests (with variables substitution),
template-based deployment using Kustomization files.

1: script-based deployment¶

In this mode, you only have to provide a shell script that fully implements the deployment using the kubectl CLI.

The deployment script is searched as follows:

look for a specific k8s-deploy-$environment_type.sh in the $K8S_SCRIPTS_DIR directory in your project (e.g. k8s-deploy-staging.sh for staging environment),
if not found: look for a default k8s-deploy.sh in the $K8S_SCRIPTS_DIR directory in your project,
if not found: the GitLab CI template assumes you're using the template-based deployment policy.

k8s-deploy-$environment_type.sh or k8s-deploy.sh needs to be executable, you can add flag execution with: git update-index --chmod=+x k8s-deploy.sh

2: template-based deployment¶

In this mode, you have to provide a Kubernetes deployment file in your project structure, and let the GitLab CI template kubectl apply it.

The template processes the following steps:

optionally executes the k8s-pre-apply.sh script in your project to perform specific environment pre-initialization (for e.g. create required services),
looks for your Kubernetes deployment file, performs variables substitution and kubectl apply it,
1. look for a specific deployment-$environment_type.yml in your project (e.g. deployment-staging.yml for staging environment),
2. fallbacks to default deployment.yml.
optionally executes the k8s-post-apply.sh script in your project to perform specific environment post-initialization stuff,

k8s-pre-apply.sh or k8s-post-apply.sh needs to be executable, you can add flag execution with: git update-index --chmod=+x k8s-pre-apply.sh

3: Kustomize-based deployment¶

In this mode, you have to provide a Kustomization file in your project structure, and let the template kubectl apply it.

The template processes the following steps:

optionally executes the k8s-pre-apply.sh script in your project to perform specific environment pre-initialization (for e.g. create required services),
looks for your Kustomization file, performs variables substitution and kubectl apply it,
1. looks for an environment-specific overlay file ./$environment_type/kustomization.yaml (e.g. ./staging/kustomization.yaml for staging environment),
2. fallbacks to default kustomization.yaml.
optionally executes the k8s-post-apply.sh script in your project to perform specific environment post-initialization stuff,

k8s-pre-apply.sh or k8s-post-apply.sh needs to be executable, you can add flag execution with: git update-index --chmod=+x k8s-pre-apply.sh

Variables substitution is performed by the deprecated feature from Kustomize based on configMapGenerator, using a non-valuated variable from a config map.

Readiness script¶

After deployment (either script-based or template-based), the GitLab CI template optionally executes the k8s-readiness-check.sh hook script to wait & check for the application to be ready (if not found, the template assumes the application was successfully started).

k8s-readiness-check.sh needs to be executable, you can add flag execution with: git update-index --chmod=+x k8s-pre-apply.sh

Supported cleanup methods¶

The Kubernetes template supports three techniques to destroy an environment (actually only review environments):

script-based deployment,
template-based deployment using raw Kubernetes manifests (with variables substitution),
template-based deployment using Kustomization files.

1: script-based cleanup¶

In this mode, you only have to provide a shell script that fully implements the environment cleanup using the kubectl CLI.

The a deployment script is searched as follows:

look for a specific k8s-cleanup-$environment_type.sh in the $K8S_SCRIPTS_DIR directory in your project (e.g. k8s-cleanup-staging.sh for staging environment),
if not found: look for a default k8s-cleanup.sh in the $K8S_SCRIPTS_DIR directory in your project,
if not found: the GitLab CI template assumes you're using the template-based cleanup policy.

k8s-cleanup-$environment_type.sh or k8s-cleanup.sh needs to be executable, you can add flag execution with: git update-index --chmod=+x k8s-cleanup.sh

TIP: a nice way to implement environment cleanup is to declare the label app=${environment_name} on every Kubernetes object associated to your environment. Then environment cleanup can be implemented very easily with command kubectl delete all,pvc,secret,ingress -l "app=${environment_name}"

2: template-based cleanup¶

In this mode, you mainly let Kubernetes delete all objects from your Kubernetes deployment file.

The template processes the following steps:

optionally executes the k8s-pre-cleanup.sh script in your project to perform specific environment pre-cleanup stuff,
looks for your Kubernetes deployment file, performs variables substitution and kubectl delete it,
1. look for a specific deployment-$environment_type.yml in your project (e.g. deployment-staging.yml for staging environment),
2. fallbacks to default deployment.yml.
optionally executes the k8s-post-cleanup.sh script in your project to perform specific environment post-cleanup (for e.g. delete bound services).

k8s-pre-cleanup.sh or k8s-post-cleanup.sh needs to be executable, you can add flag execution with: git update-index --chmod=+x k8s-pre-cleanup.sh

3: Kustomize-based cleanup¶

In this mode, you mainly let Kubernetes delete all objects from your Kustomization file(s).

The template processes the following steps:

optionally executes the k8s-pre-cleanup.sh script in your project to perform specific environment pre-cleanup stuff,
looks for your Kustomization file, performs variables substitution and kubectl delete it,
1. looks for an environment-specific overlay file ./$environment_type/kustomization.yaml (e.g. ./staging/kustomization.yaml for staging environment),
2. fallbacks to default kustomization.yaml.
optionally executes the k8s-post-cleanup.sh script in your project to perform specific environment post-cleanup (for e.g. delete bound services).

k8s-pre-cleanup.sh or k8s-post-cleanup.sh needs to be executable, you can add flag execution with: git update-index --chmod=+x k8s-pre-cleanup.sh

Cleanup job limitations¶

When using this template, you have to be aware of one limitation (bug) with the cleanup job.

By default, the cleanup job triggered automatically on branch deletion will fail due to not being able to fetch the Git branch prior to executing the job (obviously because the branch was just deleted). This is pretty annoying, but as you may see above, deleting an env may require scripts from the project...

In a future version, the template will rely on Kustomize and will be able to delete an entire environment using the app label. In the meantime we're just sorry about this bug, but for now there is not much we can do:

remind to delete your review env manually before deleting the branch
otherwise you'll have to do it afterwards from your computer (using kubectl CLI) or from the some k8s web console.

Using variables¶

You have to be aware that your deployment (and cleanup) scripts have to be able to cope with various environments, each with different application names, exposed routes, settings, ... Part of this complexity can be handled by the lookup policies described above (ex: one script/manifest per env) and also by using available environment variables:

deployment context variables provided by the template:
- ${environment_type}: the current environment type (review, integration, staging or production)
- ${environment_name}: the application name to use for the current environment (ex: myproject-review-fix-bug-12 or myproject-staging)
- ${environment_name_ssc}: the application name in SCREAMING_SNAKE_CASE format (ex: MYPROJECT_REVIEW_FIX_BUG_12 or MYPROJECT_STAGING)
- ${k8s_namespace}: the Kubernetes namespace currently used for deployment/cleanup
- ${hostname}: the environment hostname, extracted from the current environment url (after late variable expansion - see below)
any GitLab CI variable
any custom variable (ex: ${SECRET_TOKEN} that you have set in your project CI/CD variables)

While your scripts may simply use any of those variables, your Kubernetes and Kustomize resources can use variable substitution with the syntax ${VARIABLE_NAME}. Each of those patterns will be dynamically replaced in your resources by the template right before using it.

You can prevent any line from being processed by appending # nosubst at the end of the line. For instance in the following example, ${REMOTE_SERVICE_NAME} won't be replaced by its environment value during GitLab job execution:

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app: ${APPLICATION_NAME}
  name: ${APPLICATION_NAME}
data:
  application.yml: |
    remote:
      some-service:
          name: '${REMOTE_SERVICE_NAME}' # nosubst

In order to be properly replaced, curly braces are mandatory (ex: ${MYVAR} and not $MYVAR). Moreover, multiline variables must be surrounded by double quotes (").

Example:
[...]
containers:
- name: restaurant-app
  env:
  # multiline variable
  - name: MENU
    value: "${APP_MENU}"

Environments URL management¶

The K8S template supports two ways of providing your environments url:

a static way: when the environments url can be determined in advance, probably because you're exposing your routes through a DNS you manage,
a dynamic way: when the url cannot be known before the deployment job is executed.

The static way can be implemented simply by setting the appropriate configuration variable(s) depending on the environment (see environments configuration chapters):

$K8S_ENVIRONMENT_URL to define a default url pattern for all your envs,
$K8S_REVIEW_ENVIRONMENT_URL, $K8S_INTEG_ENVIRONMENT_URL, $K8S_STAGING_ENVIRONMENT_URL and $K8S_PROD_ENVIRONMENT_URL to override the default.

Each of those variables support a late variable expansion mechanism with the %{somevar} syntax, allowing you to use any dynamically evaluated variables such as ${environment_name}.

Example:

variables:
  K8S_BASE_APP_NAME: "wonderapp"
  # global url for all environments
  K8S_ENVIRONMENT_URL: "https://%{environment_name}.nonprod.acme.domain"
  # override for prod (late expansion of $K8S_BASE_APP_NAME not needed here)
  K8S_PROD_ENVIRONMENT_URL: "https://$K8S_BASE_APP_NAME.acme.domain"
  # override for review (using separate resource paths)
  K8S_REVIEW_ENVIRONMENT_URL: "https://wonderapp-review.nonprod.acme.domain/%{environment_name}"

To implement the dynamic way, your deployment script shall simply generate a environment_url.txt file in the working directory, containing only the dynamically generated url. When detected by the template, it will use it as the newly deployed environment url.

Deployment output variables¶

Each deployment job produces output variables that are propagated to downstream jobs (using dotenv artifacts):

$environment_type: set to the type of environment (review, integration, staging or production),
$environment_name: the application name (see below),
$environment_url: set to the environment URL (whether determined statically or dynamically).

Those variables may be freely used in downstream jobs (for instance to run acceptance tests against the latest deployed environment).

You may also add and propagate your own custom variables, by pushing them to the kubernetes.env file in your deployment script or hook.

Configuration reference¶

Secrets management¶

Here are some advices about your secrets (variables marked with a ):

Manage them as project or group CI/CD variables:
- masked to prevent them from being inadvertently displayed in your job logs,
- protected if you want to secure some secrets you don't want everyone in the project to have access to (for instance production secrets).
In case a secret contains characters that prevent it from being masked, simply define its value as the Base64 encoded value prefixed with @b64@: it will then be possible to mask it and the template will automatically decode it prior to using it.
Don't forget to escape special characters (ex: $ -> $$).

Global configuration¶

The Kubernetes template uses some global configuration used throughout all jobs.

Input / Variable	Description	Default value
`kubectl-image` / `K8S_KUBECTL_IMAGE`	the Docker image used to run Kubernetes `kubectl` commands set the version required by your Kubernetes server	`registry.hub.docker.com/bitnami/kubectl:latest`
`base-app-name` / `K8S_BASE_APP_NAME`	Default application name	`$CI_PROJECT_NAME` (see GitLab doc)
`environment-url` / `K8S_ENVIRONMENT_URL`	Default environments url (only define for static environment URLs declaration) supports late variable expansion (ex: `https://%{environment_name}.k8s.acme.com`)	none
`KUBE_CONTEXT`	Defines the context to be used in `KUBECONFIG`. When using GitLab agents with the CI/CD workflow, the value should be like `path/to/agent/project:agent-name`. To use different agents per environment, define an environment-scoped CI/CD variable for each agent.	none
`K8S_DEFAULT_KUBE_CONFIG`	The default kubeconfig to use (either content or file variable)	required if not using exploded kubeconfig parameters
`url` / `K8S_URL`	the Kubernetes API url	required if using exploded kubeconfig parameters
`K8S_CA_CERT`	the default Kubernetes server certificate authority	optional if using exploded kubeconfig parameters
`K8S_TOKEN`	Default service account token	required if using exploded kubeconfig parameters
`scripts-dir` / `K8S_SCRIPTS_DIR`	directory where k8s scripts (hook scripts) are located	`.` (root project dir)
`kustomize-enabled` / `K8S_KUSTOMIZE_ENABLED`	Set to `true` to force using Kustomize	none (disabled)
`kustomize-args` / `K8S_KUSTOMIZE_ARGS`	Additional `kubectl kustomize` options for example: `--enable-helm`	none
`DOCKER_CONTAINER_STABLE_IMAGE`	Docker image name to use for staging/prod	has to be defined when not chaining execution from Docker template
`DOCKER_CONTAINER_UNSTABLE_IMAGE`	Docker image name to use for review	has to be defined when not chaining execution from Docker template

Review environments configuration¶

Review environments are dynamic and ephemeral environments to deploy your ongoing developments (a.k.a. feature or topic branches).

They are disabled by default and can be enabled by setting the K8S_REVIEW_SPACE variable (see below).

Here are variables supported to configure review environments:

Input / Variable	Description	Default value
`review-space` / `K8S_REVIEW_SPACE`	k8s namespace for `review` env	none (disabled)
`review-app-name` / `K8S_REVIEW_APP_NAME`	application name for `review` env	`"${K8S_BASE_APP_NAME}-${CI_ENVIRONMENT_SLUG}"`
`review-environment-url` / `K8S_REVIEW_ENVIRONMENT_URL`	The review environments url (only define for static environment URLs declaration and if different from default)	`$K8S_ENVIRONMENT_URL`
`K8S_REVIEW_KUBE_CONFIG`	Specific kubeconfig for `review` env (only define if not using exploded kubeconfig parameters and if different from default)	`$K8S_DEFAULT_KUBE_CONFIG`
`review-url` / `K8S_REVIEW_URL`	Kubernetes API url for `review` env (only define if using exploded kubeconfig parameters and if different from default)	`$K8S_URL`
`K8S_REVIEW_CA_CERT`	the Kubernetes server certificate authority for `review` env (only define if using exploded kubeconfig parameters and if different from default)	`$K8S_CA_CERT`
`K8S_REVIEW_TOKEN`	service account token for `review` env (only define if using exploded kubeconfig parameters and if different from default)	`$K8S_TOKEN`
`review-autostop-duration` / `K8S_REVIEW_AUTOSTOP_DURATION`	The amount of time before GitLab will automatically stop `review` environments	`4 hours`

Integration environment configuration¶

The integration environment is the environment associated to your integration branch (develop by default).

It is disabled by default and can be enabled by setting the K8S_INTEG_SPACE variable (see below).

Here are variables supported to configure the integration environment:

Input / Variable	Description	Default value
`integ-space` / `K8S_INTEG_SPACE`	k8s namespace for `integration` env	none (disabled)
`integ-app-name` / `K8S_INTEG_APP_NAME`	application name for `integration` env	`$K8S_BASE_APP_NAME-integration`
`integ-environment-url` / `K8S_INTEG_ENVIRONMENT_URL`	The integration environment url (only define for static environment URLs declaration and if different from default)	`$K8S_ENVIRONMENT_URL`
`K8S_INTEG_KUBE_CONFIG`	Specific kubeconfig for `integration` env (only define if not using exploded kubeconfig parameters and if different from default)	`$K8S_DEFAULT_KUBE_CONFIG`
`integ-url` / `K8S_INTEG_URL`	Kubernetes API url for `integration` env (only define if using exploded kubeconfig parameters and if different from default)	`$K8S_URL`
`K8S_INTEG_CA_CERT`	the Kubernetes server certificate authority for `integration` env (only define if using exploded kubeconfig parameters and if different from default)	`$K8S_CA_CERT`
`K8S_INTEG_TOKEN`	service account token for `integration` env (only define if using exploded kubeconfig parameters and if different from default)	`$K8S_TOKEN`

Staging environment configuration¶

The staging environment is an iso-prod environment meant for testing and validation purpose associated to your production branch (main or master by default).

It is disabled by default and can be enabled by setting the K8S_STAGING_SPACE variable (see below).

Here are variables supported to configure the staging environment:

Input / Variable	Description	Default value
`staging-space` / `K8S_STAGING_SPACE`	k8s namespace for `staging` env	none (disabled)
`staging-app-name` / `K8S_STAGING_APP_NAME`	application name for `staging` env	`$K8S_BASE_APP_NAME-staging`
`staging-environment-url` / `K8S_STAGING_ENVIRONMENT_URL`	The staging environment url (only define for static environment URLs declaration and if different from default)	`$K8S_ENVIRONMENT_URL`
`K8S_STAGING_KUBE_CONFIG`	Specific kubeconfig for `staging` env (only define if not using exploded kubeconfig parameters and if different from default)	`$K8S_DEFAULT_KUBE_CONFIG`
`staging-url` / `K8S_STAGING_URL`	Kubernetes API url for `staging` env (only define if using exploded kubeconfig parameters and if different from default)	`$K8S_URL`
`K8S_STAGING_CA_CERT`	the Kubernetes server certificate authority for `staging` env (only define if using exploded kubeconfig parameters and if different from default)	`$K8S_CA_CERT`
`K8S_STAGING_TOKEN`	service account token for `staging` env (only define if using exploded kubeconfig parameters and if different from default)	`$K8S_TOKEN`

Production environment configuration¶

The production environment is the final deployment environment associated with your production branch (main or master by default).

It is disabled by default and can be enabled by setting the K8S_PROD_SPACE variable (see below).

Here are variables supported to configure the production environment:

Input / Variable	Description	Default value
`prod-space` / `K8S_PROD_SPACE`	k8s namespace for `production` env	none (disabled)
`prod-app-name` / `K8S_PROD_APP_NAME`	application name for `production` env	`$K8S_BASE_APP_NAME`
`prod-environment-url` / `K8S_PROD_ENVIRONMENT_URL`	The production environment url (only define for static environment URLs declaration and if different from default)	`$K8S_ENVIRONMENT_URL`
`K8S_PROD_KUBE_CONFIG`	Specific kubeconfig for `production` env (only define if not using exploded kubeconfig parameters and if different from default)	`$K8S_DEFAULT_KUBE_CONFIG`
`prod-url` / `K8S_PROD_URL`	Kubernetes API url for `production` env (only define if using exploded kubeconfig parameters and if different from default)	`$K8S_URL`
`K8S_PROD_CA_CERT`	the Kubernetes server certificate authority for `production` env (only define if using exploded kubeconfig parameters and if different from default)	`$K8S_CA_CERT`
`K8S_PROD_TOKEN`	service account token for `production` env (only define if using exploded kubeconfig parameters and if different from default)	`$K8S_TOKEN`
`prod-deploy-strategy` / `K8S_PROD_DEPLOY_STRATEGY`	Defines the deployment to production strategy. One of `manual` (i.e. one-click) or `auto`.	`manual`

kube-score job¶

The Kubernetes template enables kube-score analysis of your Kubernetes object definitions. This job is mapped to the test stage and is active by default.

Here are its parameters:

Input / Variable	Description	Default value
`kube-score-image` / `K8S_KUBE_SCORE_IMAGE`	Docker image to run kube-score	`registry.hub.docker.com/zegl/kube-score:latest` it is recommended to set a tool version compatible with your Kubernetes cluster
`score-disabled` / `K8S_SCORE_DISABLED`	Set to `true` to disable the `kube-score` analysis	none (enabled)
`score-extra-opts` / `K8S_SCORE_EXTRA_OPTS`	Additional options to `kube-score` command line	none

Variants¶

Vault variant¶

This variant allows delegating your secrets management to a Vault server.

Change default `K8S_KUBECTL_IMAGE`¶

The Vault variant requires curl or wget to retrieve secrets from the Vault server, which makes it incompatible with the default K8S_KUBECTL_IMAGE from Bitnami, as curl and wget are no longer part of it.

As a result, when using the Vault variant, you'll have to select a K8S_KUBECTL_IMAGE that - in addition to kubectl - contains curl or wget. For instance container-oc images (see example below).

Configuration¶

In order to be able to communicate with the Vault server, the variant requires the additional configuration parameters:

Input / Variable	Description	Default value
`TBC_VAULT_IMAGE`	The Vault Secrets Provider image to use (can be overridden)	`registry.gitlab.com/to-be-continuous/tools/vault-secrets-provider:latest`
`vault-base-url` / `VAULT_BASE_URL`	The Vault server base API url	none
`vault-oidc-aud` / `VAULT_OIDC_AUD`	The `aud` claim for the JWT	`$CI_SERVER_URL`
`VAULT_ROLE_ID`	The AppRole RoleID	must be defined
`VAULT_SECRET_ID`	The AppRole SecretID	must be defined

Usage¶

Then you may retrieve any of your secret(s) from Vault using the following syntax:

@url@http://vault-secrets-provider/api/secrets/{secret_path}?field={field}

With:

Parameter	Description
`secret_path` (path parameter)	this is your secret location in the Vault server
`field` (query parameter)	parameter to access a single basic field from the secret JSON payload

Example¶

include:
  # main template
  - component: $CI_SERVER_FQDN/to-be-continuous/kubernetes/gitlab-ci-k8s@6.3.0
    inputs:
      # ⚠ oc-container image (includes required curl)
      kubectl-image: registry.hub.docker.com/docker.io/appuio/oc:v4.14
  # Vault variant
  - component: $CI_SERVER_FQDN/to-be-continuous/kubernetes/gitlab-ci-k8s-vault@6.3.0
    inputs:
      # audience claim for JWT
      vault-oidc-aud: "https://vault.acme.host"
      vault-base-url: "https://vault.acme.host/v1"
      # $VAULT_ROLE_ID and $VAULT_SECRET_ID defined as a secret CI/CD variable

variables:
    # Secrets managed by Vault
    K8S_DEFAULT_KUBE_CONFIG: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/my-app/kubernetes/noprod?field=kube_config"
    K8S_PROD_KUBE_CONFIG: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/my-app/kubernetes/prod?field=kube_config"

Google Cloud variant¶

This variant uses Application Default Credentials through the GOOGLE_APPLICATION_CREDENTIALS variable using Workload Identity federation.

List of requirements before using this variant:

You must have a Workload Identity Federation Pool and Provider configured,
You must have a Service Account with the roles/iam.workloadIdentityUser IAM role granted to the Workload Identity principal matching your Gitlab project or group,
Optionally, you can set the GOOGLE_CLOUD_PROJECT template variable to define the default Google Cloud project.
You must have create a kubeconfig.yaml configuration which enable application default credentials for kubectl

The Gitlab documentation has some details about Workload Identity Federation integration.

This blog post about OIDC impersonation through Workload Identify Federation might also be of help.

Configuration¶

The variant requires the additional configuration parameters:

Input / Variable	Description	Default value
`gcp-oidc-aud` / `GCP_OIDC_AUD`	The `aud` claim for the JWT token	`$CI_SERVER_URL`
`gcp-oidc-provider` / `GCP_OIDC_PROVIDER`	Default Workload Identity Provider associated with GitLab to authenticate with OpenID Connect	none
`gcp-oidc-account` / `GCP_OIDC_ACCOUNT`	Default Service Account to which impersonate with OpenID Connect authentication	none
`gcp-review-oidc-provider` / `GCP_REVIEW_OIDC_PROVIDER`	Workload Identity Provider associated with GitLab to authenticate with OpenID Connect on `review` environment (only define to override default)	none
`gcp-review-oidc-account` / `GCP_REVIEW_OIDC_ACCOUNT`	Service Account to which impersonate with OpenID Connect authentication on `review` environment (only define to override default)	none
`gcp-integ-oidc-provider` / `GCP_INTEG_OIDC_PROVIDER`	Workload Identity Provider associated with GitLab to authenticate with OpenID Connect on `integration` environment (only define to override default)	none
`gcp-integ-oidc-account` / `GCP_INTEG_OIDC_ACCOUNT`	Service Account to which impersonate with OpenID Connect authentication on `integration` environment (only define to override default)	none
`gcp-staging-oidc-provider` / `GCP_STAGING_OIDC_PROVIDER`	Workload Identity Provider associated with GitLab to authenticate with OpenID Connect on `staging` environment (only define to override default)	none
`gcp-staging-oidc-account` / `GCP_STAGING_OIDC_ACCOUNT`	Service Account to which impersonate with OpenID Connect authentication on `staging` environment (only define to override default)	none
`gcp-prod-oidc-provider` / `GCP_PROD_OIDC_PROVIDER`	Workload Identity Provider associated with GitLab to authenticate with OpenID Connect on `production` environment (only define to override default)	none
`gcp-prod-oidc-account` / `GCP_PROD_OIDC_ACCOUNT`	Service Account to which impersonate with OpenID Connect authentication on `production` environment (only define to override default)	none
`kubectl-image` / `K8S_KUBECTL_IMAGE`	The Docker image used to run Kubernetes `kubectl` commands on GKE	`gcr.io/google.com/cloudsdktool/cloud-sdk:latest`

Example¶

With a common default GCP_OIDC_PROVIDER and GCP_OIDC_ACCOUNT configuration for non-prod environments, and a specific one for production:

include:
  # main template
  - component: $CI_SERVER_FQDN/to-be-continuous/kubernetes/gitlab-ci-k8s@6.3.0
  # Google Cloud variant
  - component: $CI_SERVER_FQDN/to-be-continuous/kubernetes/gitlab-ci-k8ss-gcp@6.3.0
    inputs:
      # common OIDC config for non-prod envs
      gcp-oidc-provider: "projects/<gcp_nonprod_proj_id>/locations/global/workloadIdentityPools/<pool_id>/providers/<provider_id>"
      gcp-oidc-account: "<name>@$<gcp_nonprod_proj_id>.iam.gserviceaccount.com"
      # specific OIDC config for prod
      gcp-prod-oidc-provider: "projects/<gcp_prod_proj_id>/locations/global/workloadIdentityPools/<pool_id>/providers/<provider_id>"
      gcp-prod-oidc-account: "<name>@$<gcp_prod_proj_id>.iam.gserviceaccount.com"

GitLab CI template for Kubernetes (k8s)¶

Usage¶

Use as a CI/CD component¶

Use as a CI/CD template (legacy)¶

Understand¶

Managed deployment environments¶

Review environments¶

Integration environment¶

Production environments¶

Supported authentication methods¶

Deployment context variables¶

Generated environment name¶

Supported deployment methods¶

1: script-based deployment¶

2: template-based deployment¶

3: Kustomize-based deployment¶

Readiness script¶

Supported cleanup methods¶

1: script-based cleanup¶

2: template-based cleanup¶

3: Kustomize-based cleanup¶

Cleanup job limitations¶

Using variables¶

Environments URL management¶

Deployment output variables¶

Configuration reference¶

Secrets management¶

Global configuration¶

Review environments configuration¶

Integration environment configuration¶

Staging environment configuration¶

Production environment configuration¶

kube-score job¶

Variants¶

Vault variant¶

Change default K8S_KUBECTL_IMAGE¶

Configuration¶

Usage¶

Example¶

Google Cloud variant¶

Configuration¶

Example¶

Change default `K8S_KUBECTL_IMAGE`¶