Using To Be Continuous¶

This page presents the general principles of use supported throughout all to be continuous templates.

Include a template¶

As previously said, each template may be included in your .gitlab-ci.yml file using one of the 3 techniques:

include:component to use it as a CI/CD component,
include:project to use it as a regular template,
or include:remote (this technique might be interesting if you want to test to be continuous from your Self-Managed GitLab without installing it locally).

For example (CI/CD component):

include:
  # Maven template with exact version '3.9.0'
  - component: $CI_SERVER_FQDN/to-be-continuous/maven/gitlab-ci-maven@3.9.0
  # AWS template with minor version alias '5.2' (uses the latest available patch version)
  - component: $CI_SERVER_FQDN/to-be-continuous/aws/gitlab-ci-aws@5.2

Our templates are versioned (compliant with Semantic Versioning):

each version is exposed through a Git tag such as 1.1.0, 2.1.4, ...
for convenience purpose, our templates also maintain a minor version alias tag (ex: 2.1), always referencing the latest patched version within that minor version, and also a major version alias tag (ex: 2), always referencing the latest minor version within that major version.
our recommendation is to use a fixed version of each template (either exact, minor or major), and upgrade when a new valuable feature is rolled out.
you may also chose to use the latest released version (discouraged as a new version with breaking changes would break your pipeline). For this, simply include the template from the default branch.

Configure a template¶

Each template comes with a predefined configuration (whenever possible), but is always overridable:

with inputs if using the include:component technique,
or with variables if using include:project](https://docs.gitlab.com/ee/ci/yaml/#includeproject) or include:remote.

Some template features are also enabled by defining the right variable(s).

Use as a CI/CD component¶

Here is an example of a Maven project that:

overrides the Maven version used (with image input),
overrides the build arguments (with build-args),
enables SonarQube analysis (by defining the sonar-url input and the SONAR_AUTH_TOKEN secret variable),

include:
  # 1: include the component
  - component: $CI_SERVER_FQDN/to-be-continuous/maven/gitlab-ci-maven@3.9.0
    # 2: set/override component inputs
    inputs:
      # use Maven 3.6 with JDK 8
      image: "maven:3.6-jdk-8"
      # use 'cicd' Maven profile
      build-args: 'verify -Pcicd'
      # enable SonarQube analysis
      sonar-url: "https://mysonar.domain.my"
      # SONAR_AUTH_TOKEN defined as a secret CI/CD variable

Use as a regular template¶

Here is an example of a Maven project that:

overrides the Maven version used (with MAVEN_IMAGE variable),
overrides the build arguments (with MAVEN_BUILD_ARGS),
enables SonarQube analysis (by defining SONAR_URL and SONAR_AUTH_TOKEN),

# 1: include the template(s)
include:
  - project: 'to-be-continuous/maven'
    ref: '3.9.0'
    file: '/templates/gitlab-ci-maven.yml'

# 2: set/override template variables
variables:
  # use Maven 3.6 with JDK 8
  MAVEN_IMAGE: "maven:3.6-jdk-8"
  # use 'cicd' Maven profile
  MAVEN_BUILD_ARGS: 'verify -Pcicd'
  # enable SonarQube analysis
  SONAR_URL: "https://mysonar.domain.my"
  # SONAR_AUTH_TOKEN defined as a secret CI/CD variable

This is the basic pattern for configuring the templates!

You'll find configuration details in each template reference documentation.

Debugging to be continuous jobs¶

Each template enable debug logs when $TRACE is set to true.

So you may simply manually run your pipeline, and set TRACE=true interactively.

This is different (and complementary) to GitLab's CI_DEBUG_TRACE variable.

Security notice

When using the CI_DEBUG_TRACE variable in GitLab, it's important to be aware of the potential security risks associated with it. Setting CI_DEBUG_TRACE to true enables detailed tracing of all commands executed during a CI/CD job, including the output of environment variables, command arguments, and any sensitive information that might be exposed during the pipeline's execution. This can include credentials, tokens, API keys, and other confidential data. If these logs are not properly secured, they can be accessed by unauthorized users, leading to potential security breaches. Therefore, it is recommended to use CI_DEBUG_TRACE only when necessary and to ensure that sensitive information is appropriately masked or removed from the logs. Additionally, access to these logs should be restricted to authorized personnel only to minimize the risk of exposing critical information.

Docker Images Versions¶

to be continuous templates use - whenever possible - required tools as container images. And when available, the latest image version is used.

In some cases, using the latest version is a good thing, and in some other cases, the latest version is bad.

latest is good for:
- DevSecOps tools (Code Quality, Security Analysis, Dependency Check, Linters ...) as using the latest version of the tool is the best way to ensure you're likely to detect vulnerabilities as soon as possible (well, as soon as new vulnerabilities are known and covered by DevSecOps tools).
- Public cloud CLI clients as there is only one version of the public cloud, and the official container image is likely to evolve at the same time as the APIs.
latest is not good for:
- Build tools as your project is developped using one specific version of the language / the build tool, and you would like to control when you change version.
- Infrastructure-as-Code tools for the same reason as above.
- Acceptance tests tools as the same reason as build tools.
- Private cloud CLI clients as you may not have installed the latest version of - say - OpenShift or Kubernetes, and you'll need to use the client CLI version that matches your servers version.

To summarize

Make sure you explicitely override the container image versions of your build, Infrastructure-as-Code, private cloud CLI clients and acceptance tests tools matching your project requirements.
Be aware that sometimes your pipeline may fail (without any change from you) due to a new version of DevSecOps tool that either highlights a new vulnerability (), or due to a bug or breaking change in the tool ( happens).

Secrets managements¶

Most of our templates manage secrets (access tokens, user/passwords, ...).

Our general recommendation for those secrets is to manage them as project or group CI/CD variables:

masked to prevent them from being inadvertently displayed in your job logs,
protected if you want to secure some secrets you don't want everyone in the project to have access to (for instance production secrets).

What if a secret can't be masked?¶

It may happen that a secret contains characters that prevent it from being masked.

In that case there is a simple solution: simply encode it in Base64 and declare the variable value as the Base64 string prefixed with @b64@. This value can be masked, and it will be automatically decoded by our templates (make sure you're using a version of the template that supports this syntax).

The Base64 string is not allowed to contain linebreaks. If you are for example using base64 to encode, be sure to use the -w 0 option to disable line wrapping.

Example

CAVE_PASSPHRASE={"open":"$€5@me"} can't be masked, but the Base64 encoded secret can.

Then just declare instead:

CAVE_PASSPHRASE=@b64@eyJvcGVuIjoiJOKCrDVAbWUifQ==

Scoped variables¶

All our templates support a generic and powerful way of limiting/overriding some of your environment variables, depending on the execution context.

This feature is comparable to GitLab Scoping environments with specs feature, but covers a broader usage:

can be used with non-secret variables (defined in your .gitlab-ci.yml file),
variables can be scoped by any other criteria than deployment environment.

The feature is based on a specific variable naming syntax:

# syntax 1: using a unary test operator
scoped__<target var>__<condition>__<cond var>__<unary op>=<target val>

# syntax 2: using a comparison operator
scoped__<target var>__<condition>__<cond var>__<cmp op>__<cmp val>=<target val>

mind the double underscore that separates each part.

Where:

Name	Description	Possible values / examples
`<target var>`	Scoped variable name	any example: `MY_SECRET`, `MAVEN_BUILD_ARGS`, ...
`<condition>`	The test condition	one of: `if` or `ifnot`
`<cond var>`	The variable on which relies the condition	any example: `CI_ENVIRONMENT_NAME`, `CI_COMMIT_REF_NAME`, ...
`<unary op>`	Unary test operator to use	only: `defined`
`<cmp op>`	Comparison operator to use	one of: `equals`, `startswith`, `endswith`, `contains`, `in` or their ignore case version: `equals_ic`, `startswith_ic`, `endswith_ic`,`contains_ic` or `in_ic`
`<cmp val>`	Sluggified value to compare `<cond var>` against	any With `in` or `in_ic` operators, matching values shall be separated with double underscores
`<target val>`	The value `<target var>` takes when condition matches	any (can even use other variables that will be expanded)

Which variables support this?

The scoped variables feature has a strong limitation: it may only be used for variables used in the script and/or before_script parts; not elsewhere in the .gitlab-ci.yml file.

They don't support scoped variables:

variables used to parameterize the jobs container image(s) (ex: MAVEN_IMAGE or K8S_KUBECTL_IMAGE),
variables that enable/disable some jobs behavior (ex: MAVEN_DEPLOY_ENABLED, NODE_AUDIT_DISABLED or AUTODEPLOY_TO_PROD),
variables used in artifacts, cache or rules sections (ex: PYTHON_PROJECT_DIR, NG_WORKSPACE_DIR or TF_PROJECT_DIR).

They do support scoped variables:

credentials (logins, passwords, tokens, ...),
configuration URLs,
tool CLI options and arguments (ex: MAVEN_BUILD_ARGS or PHP_CODESNIFFER_ARGS)

If you have any doubt: have a look at the template implementation.

How variable values are sluggified?

Each character that is not a letter, a digit or underscore is replaced by an underscore (_).

Examples:

Wh@t*tH€!h3¢k becomes: Wh_t_tH__h3_k
feat/add-welcome-page becomes: feat_add_welcome_page

Example 1: scope by environment

variables:
  # default configuration
  K8S_URL: "https://my-nonprod-k8s.domain"
  MY_DATABASE_PASSWORD: "admin"

  # overridden for prodution environment
  scoped__K8S_URL__if__CI_ENVIRONMENT_NAME__equals__production: "https://my-prod-k8s.domain"
  # MY_DATABASE_PASSWORD is overridden for prod in my project CI/CD variables using
  # scoped__MY_DATABASE_PASSWORD__if__CI_ENVIRONMENT_NAME__equals__production

Example 2: scope by branch

variables:
  # default Angular build arguments (default configuration)
  NG_BUILD_ARGS: "build"

  # use 'staging' configuration on develop branch
  scoped__NG_BUILD_ARGS__if__CI_COMMIT_REF_NAME__equals__develop: "build --configuration=staging"

  # use 'production' configuration and optimization on master branch
  scoped__NG_BUILD_ARGS__if__CI_COMMIT_REF_NAME__equals__master: "build --configuration=production --optimization=true"

Example 3: scope on tag

variables:
  # default Docker build configuration
  DOCKER_BUILD_ARGS: "--build-arg IMAGE_TYPE=snapshot"

  # overridden when building image on tag (release)
  scoped__DOCKER_BUILD_ARGS__if__CI_COMMIT_TAG__defined: "--build-arg IMAGE_TYPE=release"

Proxy configuration¶

Our templates don't have any proxy configuration set by default, but they all support standard Linux variables:

http_proxy
https_proxy
ftp_proxy
no_proxy

As a result, you may perfectly define those variables in your project:

either globally as group or project variables or in the top variables block definition of your .gitlab-ci.yml file,
either locally in specific jobs,
or for all jobs from one single template (see below).

Certificate Authority configuration¶

Our templates all come configured with the Default Trusted Certificate Authorities, but they all support the CUSTOM_CA_CERTS variable to configure additional certificate authorities.

When set, this variable shall contain one or several certificates in PEM format, then the template will assume those are trusted certificates, and add them accordingly to the right trust store.

Again, you may perfectly set CUSTOM_CA_CERTS in your project:

either globally as group or project variables or in the top variables block definition of your .gitlab-ci.yml file,
either locally in specific jobs,
or for all jobs from one single template (see below).

Configurable Git references¶

Production and integration branches¶

As explained earlier, to be continuous supports various Git branching models with at least one production branch (main or master by default), and possibly one integration branch (develop by default).

Those eternal branches can be easily configured by overriding the following global variables (regular expression patterns):

variables:
  # default production ref name (regex pattern)
  PROD_REF: '/^(master|main)$/'
  # default integration ref name (regex pattern)
  INTEG_REF: '/^develop$/'

Those variables are used internally thoughout all to be continuous templates.

Release tag pattern¶

Some to be continuous templates also support publish & release. Those templates trigger the publication of released packages only on Git tags matching a predefined pattern. By default the pattern enforces semantic versioning but can be overridden.

variables:
  # default release tag name (pattern)
  RELEASE_REF: '/^v?[0-9]+\.[0-9]+\.[0-9]+(-[a-zA-Z0-9-\.]+)?(\+[a-zA-Z0-9-\.]+)?$/'

Extended `[skip ci]` feature¶

GitLab skips triggering the CI/CD pipeline when [skip ci] or [ci skip] is present in the Git commit message.

This feature can be handy, but in some situations you would like to skip the CI/CD pipeline only under certain circumstances. Example: when creating a release with a Git commit containing only bumpversion changes (setting the new released version in configuration and/or documentation files). This commit will also be pushed as a tag. You might want to prevent the commit from being processed twice (one from the origin branch and one from the tag pipeline).

For this, all the recent versions of to be continuous templates implement an extended [skip ci] feature. It is now possible to skip selectively the CI/CD pipeline if your Git commit message contains a part of the following format:

[ci skip on <comma separated words>]
or:
[skip ci on <comma separated words>]

Supported words are:

Words	Description
`tag`	skipped on tag pipelines
`mr`	skipped on Merge Request pipelines
`branch`	skipped on branch pipelines
`default`	skipped on the default project branch
`prod`	skipped on the production branch
`integ`	skipped on the integration branch
`dev`	skipped on any development branch (other than production or integration)

Merge Request workflow¶

One thing that has to be chosen with GitLab CI/CD is the Merge Request workflow strategy.

By default, to be continuous implements the merge request pipelines strategy, with the following workflow declaration:

# use Merge Request pipelines rather than branch pipelines
workflow:
  rules:
    # prevent MR pipeline originating from production or integration branch(es)
    - if: '$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ $PROD_REF || $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME =~ $INTEG_REF'
      when: never
    # on non-prod, non-integration branches: prefer MR pipeline over branch pipeline
    - if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS && $CI_COMMIT_REF_NAME !~ $PROD_REF && $CI_COMMIT_REF_NAME !~ $INTEG_REF'
      when: never
    # the 7 next rules implement the extended [skip ci] feature
    - if: '$CI_COMMIT_MESSAGE =~ "/\[(ci skip|skip ci) on ([^],]*,)*tag(,[^],]*)*\]/" && $CI_COMMIT_TAG'
      when: never
    - if: '$CI_COMMIT_MESSAGE =~ "/\[(ci skip|skip ci) on ([^],]*,)*branch(,[^],]*)*\]/" && $CI_COMMIT_BRANCH'
      when: never
    - if: '$CI_COMMIT_MESSAGE =~ "/\[(ci skip|skip ci) on ([^],]*,)*mr(,[^],]*)*\]/" && $CI_MERGE_REQUEST_ID'
      when: never
    - if: '$CI_COMMIT_MESSAGE =~ "/\[(ci skip|skip ci) on ([^],]*,)*default(,[^],]*)*\]/" && $CI_COMMIT_REF_NAME =~ $CI_DEFAULT_BRANCH'
      when: never
    - if: '$CI_COMMIT_MESSAGE =~ "/\[(ci skip|skip ci) on ([^],]*,)*prod(,[^],]*)*\]/" && $CI_COMMIT_REF_NAME =~ $PROD_REF'
      when: never
    - if: '$CI_COMMIT_MESSAGE =~ "/\[(ci skip|skip ci) on ([^],]*,)*integ(,[^],]*)*\]/" && $CI_COMMIT_REF_NAME =~ $INTEG_REF'
      when: never
    - if: '$CI_COMMIT_MESSAGE =~ "/\[(ci skip|skip ci) on ([^],]*,)*dev(,[^],]*)*\]/" && $CI_COMMIT_REF_NAME !~ $PROD_REF && $CI_COMMIT_REF_NAME !~ $INTEG_REF'
      when: never
    # default: execute
    - when: always

If you want to switch to the branch pipelines strategy, simply add the following to your .gitlab-ci.yml file:

workflow:
  rules:
    # prevent Merge Request pipeline
    - if: $CI_MERGE_REQUEST_ID
      when: never
    # /!\ you MAY keep the 7 rules implementing the extended [skip ci] feature
    # >> here <<
    - when: always

Warning

Merge Request pipelines has not always been the default workflow strategy. Use the latest version of each to be continuous templates to be guaranteed to use this one, or else explicitly redefine the strategy you want in your .gitlab-ci.yaml file.

Test & Analysis jobs rules¶

As explained in this chapter, by default to be continuous implements an adaptive pipeline strategy with test & analysis jobs:

Adaptive Pipeline

This behavior is implemented with a common block of rules, shared among all test & analysis jobs:

# test job prototype: implement adaptive pipeline rules
.test-policy:
  rules:
    # on tag: auto & failing
    - if: $CI_COMMIT_TAG
    # on ADAPTIVE_PIPELINE_DISABLED: auto & failing
    - if: '$ADAPTIVE_PIPELINE_DISABLED == "true"'
    # on production or integration branch(es): auto & failing
    - if: '$CI_COMMIT_REF_NAME =~ $PROD_REF || $CI_COMMIT_REF_NAME =~ $INTEG_REF'
    # early stage (dev branch, no MR): manual & non-failing
    - if: '$CI_MERGE_REQUEST_ID == null && $CI_OPEN_MERGE_REQUESTS == null'
      when: manual
      allow_failure: true
    # Draft MR: auto & non-failing
    - if: '$CI_MERGE_REQUEST_TITLE =~ /^Draft:.*/'
      allow_failure: true
    # else (Ready MR): auto & failing
    - when: on_success

Acceptance test jobs also use a similar (but separate) common block:

# acceptance job prototype: implement adaptive pipeline rules
.acceptance-policy:
  rules:
    # exclude tags
    - if: $CI_COMMIT_TAG
      when: never
    # on production or integration branch(es): auto & failing
    - if: '$CI_COMMIT_REF_NAME =~ $PROD_REF || $CI_COMMIT_REF_NAME =~ $INTEG_REF'
    # disable if no review environment
    - if: '$REVIEW_ENABLED != "true"'
      when: never
    # on ADAPTIVE_PIPELINE_DISABLED: auto & failing
    - if: '$ADAPTIVE_PIPELINE_DISABLED == "true"'
    # early stage (dev branch, no MR): manual & non-failing
    - if: '$CI_MERGE_REQUEST_ID == null && $CI_OPEN_MERGE_REQUESTS == null'
      when: manual
      allow_failure: true
    # Draft MR: auto & non-failing
    - if: '$CI_MERGE_REQUEST_TITLE =~ /^Draft:.*/'
      allow_failure: true
    # else (Ready MR): auto & failing
    - when: on_success

From the above rules, you might notice you can easily disable the adaptive pipeline strategy (therefore enforce quality and security jobs, whatever the development stage) by setting the ADAPTIVE_PIPELINE_DISABLED variable to true.

You might also want to globally override the test & analysis and/or the acceptance jobs strategy by overriding the common block(s).

Example of custom acceptance jobs strategy

For example, the following enforces the acceptance tests whatever the development stage:

# my acceptance job strategy: always run acceptance tests on any branch
.acceptance-policy:
  rules:
    # exclude tags
    - if: $CI_COMMIT_TAG
      when: never
    - when: on_success

Advanced usage - Override YAML¶

Sometimes, configuration via variables is not enough to tweak an existing template to fit to your needs.

Fortunately, GitLab CI include feature is implemented in a way that allows you to override the included YAML code.

from GitLab documentation

The files defined in include are:

Deep merged with those in .gitlab-ci.yml.
Always evaluated first and merged with the content of .gitlab-ci.yml, regardless of the position of the include keyword.

In order to override the included templates YAML code, you'll probably have to deep dive into it and understand how it is designed.

The templates base job¶

A very important thing you should be aware of is that every template defines a (hidden) base job, extended by all other jobs. That might not be the case for templates that declare one single job.

For example the Maven template defines the .mvn-base base job.

Thus, if you wish to override something for all the jobs from a specific template, this is the right place to do the magic.

Example 1: add service containers¶

In this example, let's consider my Java project needs a MySQL database to run its unit tests.

According to the Maven template implementation, that can be done by overriding the mvn-build job as follows:

mvn-build:
  services:
    - name: mysql:latest
      alias: mysql_host
  variables:
    MYSQL_DATABASE: "acme"
    MYSQL_ROOT_PASSWORD: "root"

Those changes will gracefully be merged with the mvn-build job, the rest of it (defined by the Maven template) will remain unchanged.

Tip

More info about Service Containers

Example 2: run on private runners with proxy¶

In this example, let's consider my project needs to deploy on a Kubernetes cluster that is only accessible from my private runner (with tags kubernetes, private), and that requires an http proxy.

According to the Kubernetes template implementation, that can be done by overriding the base .k8s-base job as follows:

.k8s-base:
  # set my runner tags
  tags:
    - kubernetes
    - private
  # set my proxy configuration
  variables:
    http_proxy: "http://my.proxy:8080"
    https_proxy: "http://my.proxy:8080"

This way, all Kubernetes jobs will inherit this configuration.

Example 3: disable go-mod-outdated job¶

There are a few to be continuous jobs that can't be disabled. It is the case for example of the go-mod-outdated job from the Golang template (actually this job is a pure manually triggered job).

Let's suppose in my project I don't want this job to appear in my pipelines.

That can be done by simply overriding the go-mod-outdated job rules as follows:

include:
  - component: $CI_SERVER_FQDN/to-be-continuous/golang/gitlab-ci-golang@4.8.1

# hard disable go-mod-outdated
go-mod-outdated:
  rules:
    - when: never

This way, the job won't never appear in the project pipeline. This technique may be used to hard-disable any non-configurable to be continuous job.

Example 4: allow a test job to fail¶

All to be continuous test & analysis jobs implement a progressive strategy. With this strategy, test & analysis jobs are not allowed to fail on production and integration branches.

Why can't I configure this behavior?

to be continuous considers it's an anti-pattern to allow a test or analysis job to fail. In practice, such an in-between choice quickly becomes totally useless because no one will pay attention when it fails.

to be continuous position:

either you care about the topic addressed by the job: activate it, accept to break the pipeline when the job fails, and fixing it shall be a priority to restore the pipeline.
either you don't really care: simply disable (or don't enable) the job.

There should not be an in-between position.

Nevertheless if you want to change the strategy to allow a test or analysis job to fail, it can be done by overriding the job rules as follows (example with docker-trivy):

include:
  - component: $CI_SERVER_FQDN/to-be-continuous/docker/gitlab-ci-docker@5.7.0

# allow docker-trivy to fail
docker-trivy:
  rules:
    # next rule to preserve the Adaptive Pipeline's "early stage" behavior
    # (dev branch, no MR: manual & allow failure)
    - if: '$CI_MERGE_REQUEST_ID == null && $CI_OPEN_MERGE_REQUESTS == null'
      when: manual
      allow_failure: true
    # any other case: auto & allow failure
    - allow_failure: true