Security¶
Risks¶
CI/CD is the automation of most (if not all) process in your software lifecycle going from building to running the application in production. With so many things happening, attackers have a wide range of capabilities in case of a compromise:
- introduce dubious components (i.e. backdoor) in the application,
- leak (to the Internet) credentials granting access to internal services or infrastructures,
- access said services or infrastructures,
- impersonate an employee,
- leak intellectual property,
- ...
Every CI/CD job is executed inside a container image which contains various tools required by the job. When you are executing the job, you are trusting the people building the image and the tools.
In some cases, the image may have been corrupted by an attacker. This is what we call a supply chain attack. There are famous cases like the Codecov incident in which hackers managed to leak credentials from several services and cloud platforms.
Supply chain attacks can also occur when installing application dependencies in your CI/CD pipeline (Shai-Hulud).
Mitigations¶
Variables scoping¶
To prevent leakage, CI/CD variables has to be present only in jobs that need them. By example in the Maven template, only the mvn-sonar job actually needs the SONAR_TOKEN environment variable so others jobs (mvn-build, mvn-release...) should not access this secret.
You can find more info about how to protect your CI/CD variables in the official GitLab documentation.
Image selection¶
As much as we can, we try to select either official images (ex: Maven, Python), or at least images maintained by an active community. Each of those images can be freely overridden with the appropriate configuration variable to select fixed versions (more info here) or any alternative that would suit you more.
By default, to be continuous templates mostly use the latest tag from upstream images since it is a maintenance-less default that works for nearly everyone. However latest images are prone to supply chain attacks and are also likely to introduce breaking changes.
to be continuous is not responsible of any possible security issue from a default container image.
To mitigate the risks of using latest images, you should always use a fixed version tag (maven:3.9.1 instead of maven:latest) or build your own image.
Additional dependencies¶
However, the direct counterpart of using official images is that many default image do not ship all the required tools used by the template (think about git and curl utilities). For this reason, many templates check the presence of these additional dependencies and install them on the fly (see maybe_install_pkg function).
This is a good practice to avoid bloating the image with unnecessary tools, but it also means that you are trusting the upstream repositories of your distribution (i.e. Debian, Alpine, etc.) to not be compromised and requiring network access to download packages.
Every project is encouraged to maintain his own Docker image in order to fit to their needs, avoid extraneous downloading for additional dependencies and simplify the work of security team (it's easy to scan a Docker image). While doing so, we recommend to use a minimal distribution and root-less images to prevent some container escalation vulnerabilities against your runner provider.
Tip
Whenever building your own image or using an upstream image, you can use Renovate to watch updates for your tools, test the new version and integrate them seamlessly.
Application dependencies¶
Using lock files¶
We highly recommend using lock files (package-lock.json, yarn.lock, poetry.lock, etc.) to list all direct and transitive dependencies. These lock files should be commited to your git repositories. This ensures consistent installs between machines (including CI/CD pipelines).
When using lock files, the CI/CD pipelines based on to-be-continuous always use the dependencies versions listed in the lock file, they will never try to install newer versions of your dependencies (they use npm ci instead of npm install for example).
To add new dependencies, a developper needs to update the lock file directly on his development machine (the developper should run npm install or a similar command). Once done, this new lock file needs to be committed to the git repository.
To upgrade dependencies, we recommend using Renovate in a Gitlab CI pipeline. It will handle lock files for you.
Keep in mind that some templates might install dependencies not covered by project lock file.
Limit the risk of supply chain attack¶
When a dependency is compromised (Shai-Hulud for example), it is generally detected and removed from the repositories (NPM, PyPI, etc.) in the first hours/days after it has been published. A good practice is to configure your package managers (npm, yarn, poetry, etc.) on your development machines so that they do not download dependencies that have been published very recently. pnpm and yarn already support such settings.
Renovate also has a similar option when updating dependencies.
Vulnerability Reports (Trivy)¶
Important
When reviewing vulnerabilities from containers, keep in mind the following considerations:
- Containers are usually very short-lived in a CI/CD context.
- No direct user access is possible.
- Most job do not expose any external endpoint (i.e. HTTP server), making vulnerabilities relying on user interaction very hard if not impossible to exploit.
In short, even though risks are often reduced in a CI/CD context, this should not mean that vulnerabilities assessment are unnecessary. They remain an essential step in securing your pipeline.
Here are vulnerability reports for each default image used by to be continuous templates (generated every day):
| Template | Image Variable | Default Image | Vulnerabilities |
|---|---|---|---|
| Ansible | ANSIBLE_IMAGE | docker.io/cytopia/ansible:latest-tools |
3 High, 2 Medium, 2 Low |
| Ansible | ANSIBLE_LINT_IMAGE | docker.io/haxorof/ansible-lint:latest |
1 High, 1 Medium |
| Amazon Web Services | AWS_CLI_IMAGE | docker.io/amazon/aws-cli:latest |
|
| Azure | AZURE_CLI_IMAGE | mcr.microsoft.com/azure-cli:latest |
3 High, 2 Medium, 2 Low |
| Bash | BASH_BATS_IMAGE | docker.io/bats/bats:latest |
1 High, 1 Medium |
| Bash | BASH_SHELLCHECK_IMAGE | docker.io/koalaman/shellcheck-alpine:stable |
2 Critical, 4 High, 25 Medium, 5 Low |
| Bruno | BRU_IMAGE | docker.io/library/node:lts-alpine |
7 High, 1 Medium |
| GitLab Butler | BUTLER_IMAGE | registry.gitlab.com/to-be-continuous/tools/gitlab-butler:latest |
1 High, 1 Medium, 1 Low |
| Cloud Foundry | CF_CLI_IMAGE | docker.io/governmentpaas/cf-cli |
10 Critical, 45 High, 112 Medium, 5 Low, 1 Unknown |
| Cloud Native Buildpacks | CNB_BUILDER_IMAGE | docker.io/paketobuildpacks/builder-jammy-base:latest |
57 Critical, 162 High, 1750 Medium, 212 Low |
| Cloud Native Buildpacks | CNB_SKOPEO_IMAGE | quay.io/containers/skopeo:latest |
|
| Cloud Native Buildpacks | CNB_TRIVY_IMAGE | docker.io/aquasec/trivy:latest |
not fetched |
| Cypress | CYPRESS_IMAGE | docker.io/cypress/included:latest |
2 Critical, 68 High, 102 Medium, 254 Low, 3 Unknown |
| dbt | DBT_IMAGE | ghcr.io/dbt-labs/dbt-core:latest |
4 Critical, 281 High, 1168 Medium, 793 Low, 11 Unknown |
| Docker Compose | DCMP_IMAGE | docker.io/library/docker:latest |
7 Critical, 13 High, 15 Medium, 9 Low |
| Docker Compose | DCMP_KICS_IMAGE | docker.io/checkmarx/kics:latest |
1 Critical, 2 High, 1 Medium, 2 Low |
| Debian | DEBIAN_BASE_IMAGE | docker.io/debian |
3 Medium, 71 Low |
| Debian | DEBIAN_BUILD_IMAGE | $DEBIAN_BASE_IMAGE |
not fetched |
| Debian | DEBIAN_LINT_IMAGE | $DEBIAN_BASE_IMAGE |
not fetched |
| DefectDojo | DEFECTDOJO_BASE_IMAGE | docker.io/library/node:alpine3.11 |
1 Critical, 34 High, 3 Medium, 4 Low |
| Dependency Track | DEPTRACK_SBOM_SCANNER_IMAGE | registry.gitlab.com/to-be-continuous/tools/dt-sbom-scanner:latest |
1 Critical, 2 High, 3 Medium, 2 Low |
| Docker | DOCKER_BUILDAH_IMAGE | quay.io/containers/buildah:latest |
|
| Docker | DOCKER_DIND_IMAGE | docker.io/library/docker:dind |
7 Critical, 13 High, 15 Medium, 9 Low |
| Docker | DOCKER_HADOLINT_IMAGE | docker.io/hadolint/hadolint:latest-alpine |
2 Critical, 4 High, 25 Medium, 5 Low |
| Docker | DOCKER_IMAGE | docker.io/library/docker:latest |
7 Critical, 13 High, 15 Medium, 9 Low |
| Docker | DOCKER_KANIKO_IMAGE | registry.gitlab.com/to-be-continuous/tools/kaniko:latest |
2 Critical, 10 High, 5 Medium, 10 Low |
| Docker | DOCKER_SBOM_IMAGE | docker.io/anchore/syft:debug |
|
| Docker | DOCKER_SKOPEO_IMAGE | quay.io/containers/skopeo:latest |
|
| Docker | DOCKER_TRIVY_IMAGE | docker.io/aquasec/trivy:latest |
not fetched |
| .NET | DOTNET_IMAGE | mcr.microsoft.com/dotnet/sdk:10.0 |
not fetched |
| .NET | DOTNET_INFERSHARP_IMAGE | mcr.microsoft.com/infersharp:latest |
10 Critical, 60 High, 125 Medium, 137 Low, 10 Unknown |
| .NET | DOTNET_SBOM_IMAGE | ghcr.io/cyclonedx/cdxgen:master |
11 Critical, 211 High, 190 Medium, 15 Low, 1 Unknown |
| .NET | DOTNET_SEMGREP_IMAGE | docker.io/semgrep/semgrep:latest |
1 Critical, 3 High, 4 Medium, 1 Low |
| Google Cloud | GCP_CLI_IMAGE | gcr.io/google.com/cloudsdktool/cloud-sdk:latest |
12 Critical, 211 High, 781 Medium, 819 Low, 12 Unknown |
| Gitleaks | GITLEAKS_IMAGE | docker.io/zricethezav/gitleaks:latest |
1 High, 5 Medium, 1 Low |
| GitOps | GITOPS_IMAGE | docker.io/alpine/git:latest |
1 Critical, 3 High, 4 Medium, 1 Low |
| GitLab Package | GLPKG_IMAGE | docker.io/curlimages/curl:latest |
2 Critical, 5 High, 19 Medium |
| Go | GO_CI_LINT_IMAGE | docker.io/golangci/golangci-lint:latest-alpine |
1 High, 1 Medium |
| Go | GO_IMAGE | docker.io/library/golang:bookworm |
2 Critical, 150 High, 643 Medium, 655 Low, 9 Unknown |
| Go | GO_SBOM_IMAGE | $GO_IMAGE |
not fetched |
| Go | GO_SEMGREP_IMAGE | docker.io/semgrep/semgrep:latest |
1 Critical, 3 High, 4 Medium, 1 Low |
| Gradle | GRADLE_IMAGE | docker.io/library/gradle:latest |
22 Medium, 21 Low |
| Helmfile | HELMFILE_CLI_IMAGE | ghcr.io/helmfile/helmfile:latest |
12 Critical, 44 High, 115 Medium, 11 Low |
| Helm | HELM_CLI_IMAGE | docker.io/alpine/helm:latest |
1 Critical, 1 High, 11 Medium |
| Helm | HELM_KUBE_SCORE_IMAGE | docker.io/zegl/kube-score |
14 Critical, 78 High, 135 Medium, 8 Low |
| Helm | HELM_YAMLLINT_IMAGE | docker.io/cytopia/yamllint |
9 High, 13 Medium, 1 Low |
| Hurl | HURL_IMAGE | ghcr.io/orange-opensource/hurl:latest |
2 Critical, 4 High, 21 Medium, 3 Low |
| k6 | K6_IMAGE | docker.io/grafana/k6:latest |
1 Critical, 3 High, 2 Medium, 1 Low |
| Kubernetes | K8S_KUBECTL_IMAGE | docker.io/alpine/k8s:MUST_SET_VERSION |
not fetched |
| Kubernetes | K8S_KUBE_SCORE_IMAGE | docker.io/zegl/kube-score:latest |
14 Critical, 78 High, 135 Medium, 8 Low |
| Lighthouse | LHCI_IMAGE | docker.io/cypress/browsers:latest |
16 High, 70 Medium, 244 Low, 3 Unknown |
| GNU Make | MAKE_IMAGE | docker.io/alpinelinux/build-base |
|
| Maven | MAVEN_IMAGE | docker.io/library/maven:latest |
17 Medium, 21 Low |
| MkDocs | MKD_IMAGE | docker.io/squidfunk/mkdocs-material:latest |
3 High |
| MkDocs | MKD_LYCHEE_IMAGE | docker.io/lycheeverse/lychee:latest |
1 Critical, 2 High, 17 Medium, 76 Low, 1 Unknown |
| MobSF | MOBSF_CLIENT_IMAGE | docker.io/badouralix/curl-jq |
1 High, 1 Medium |
| MobSF | MOBSF_CODE_IMAGE | docker.io/opensecurity/mobsfscan:latest |
3 Critical, 97 High, 481 Medium, 566 Low, 14 Unknown |
| Angular | NG_CLI_IMAGE | docker.io/trion/ng-cli-karma:latest |
2 Critical, 161 High, 666 Medium, 788 Low, 13 Unknown |
| Angular | NG_SEMGREP_IMAGE | docker.io/semgrep/semgrep:latest |
1 Critical, 3 High, 4 Medium, 1 Low |
| Node.js | NODE_IMAGE | docker.io/library/node:lts-alpine |
7 High, 1 Medium |
| Node.js | NODE_SEMGREP_IMAGE | docker.io/semgrep/semgrep:latest |
1 Critical, 3 High, 4 Medium, 1 Low |
| OSS Review Toolkit (ORT) | ORT_SCANNER_IMAGE | ghcr.io/oss-review-toolkit/ort:latest |
24 Critical, 145 High, 1657 Medium, 158 Low, 1 Unknown |
| OpenShift | OS_CLI_IMAGE | quay.io/openshift/origin-cli:latest |
1 Critical, 4 High, 7 Medium, 1 Low |
| PHP | PHP_IMAGE | docker.io/library/php:latest |
38 High, 156 Medium, 594 Low, 8 Unknown |
| Playwright | PLAYWRIGHT_IMAGE | mcr.microsoft.com/playwright:latest |
27 High, 911 Medium, 175 Low |
| Postman | POSTMAN_IMAGE | docker.io/postman/newman:latest |
27 High, 34 Medium, 10 Low |
| pre-commit | PRE_COMMIT_IMAGE | docker.io/library/python:3-alpine |
1 High, 1 Medium, 1 Low |
| Puppeteer | PUPPETEER_IMAGE | ghcr.io/puppeteer/puppeteer:latest |
111 Critical, 426 High, 1001 Medium, 996 Low, 18 Unknown |
| Python | PYTHON_IMAGE | docker.io/library/python:3-slim |
3 Medium, 74 Low |
| Renovate | RENOVATE_IMAGE | docker.io/renovate/renovate:latest |
1 Critical, 6 High, 566 Medium, 64 Low |
| Robot Framework | ROBOT_BASE_IMAGE | docker.io/ppodgorsek/robot-framework:latest |
1 Critical, 10 High, 2 Medium |
| RPM | RPM_BASE_IMAGE | docker.io/fedora |
|
| RPM | RPM_BUILD_IMAGE | $RPM_BASE_IMAGE |
not fetched |
| RPM | RPM_LINT_IMAGE | $RPM_BASE_IMAGE |
not fetched |
| Rust | RUST_IMAGE | docker.io/library/rust:latest |
159 High, 449 Medium, 930 Low, 10 Unknown |
| Source-to-Image | S2I_DIND_IMAGE | docker.io/library/docker:dind |
7 Critical, 13 High, 15 Medium, 9 Low |
| Source-to-Image | S2I_SKOPEO_IMAGE | quay.io/containers/skopeo:latest |
|
| Source-to-Image | S2I_TRIVY_IMAGE | docker.io/aquasec/trivy:latest |
not fetched |
| S3 (Simple Storage Service) | S3_CMD_IMAGE | docker.io/d3fk/s3cmd:latest |
1 Critical, 2 Medium |
| Scala/SBT | SBT_IMAGE | docker.io/sbtscala/scala-sbt:17.0.2_1.6.2_3.1.3 |
56 Critical, 288 High, 417 Medium, 827 Low, 15 Unknown |
| Scala/SBT | SBT_SBOM_IMAGE | docker.io/anchore/syft:debug |
|
| semantic-release | SEMREL_IMAGE | docker.io/library/node:lts-slim |
1 Critical, 8 High, 17 Medium, 74 Low, 1 Unknown |
| SonarQube | SONAR_SCANNER_IMAGE | docker.io/sonarsource/sonar-scanner-cli:latest |
97 High, 50 Medium, 10 Low |
| Spectral | SPECTRAL_IMAGE | docker.io/stoplight/spectral:latest |
23 High, 29 Medium, 10 Low |
| Sphinx | SPHINX_IMAGE | ghcr.io/sphinx-doc/sphinx:latest |
72 Critical, 191 High, 189 Medium, 208 Low, 3 Unknown |
| Sphinx | SPHINX_LYCHEE_IMAGE | docker.io/lycheeverse/lychee:latest |
1 Critical, 2 High, 17 Medium, 76 Low, 1 Unknown |
| SQLFluff lint | SQLFLUFF_IMAGE | docker.io/sqlfluff/sqlfluff:latest |
3 Critical, 19 High, 48 Medium, 103 Low, 2 Unknown |
| Test SSL | TESTSSL_IMAGE | docker.io/drwetter/testssl.sh:latest |
|
| Terraform | TF_CHECKOV_IMAGE | docker.io/bridgecrew/checkov |
2 Critical, 15 High, 34 Medium, 117 Low, 1 Unknown |
| Terraform | TF_DOCS_IMAGE | quay.io/terraform-docs/terraform-docs:edge |
5 Critical, 10 High, 30 Medium, 1 Low |
| Terraform | TF_IMAGE | docker.io/hashicorp/terraform:latest |
2 Critical, 1 High, 3 Medium |
| Terraform | TF_INFRACOST_IMAGE | docker.io/infracost/infracost |
90 Critical, 824 High, 793 Medium, 33 Low |
| Terraform | TF_PUBLISH_IMAGE | docker.io/curlimages/curl:latest |
2 Critical, 5 High, 19 Medium |
| Terraform | TF_TFLINT_IMAGE | ghcr.io/terraform-linters/tflint:latest |
2 Critical, 5 High, 3 Medium, 1 Low |
| Terraform | TF_TFSEC_IMAGE | docker.io/aquasec/tfsec-ci |
5 Critical, 22 High, 57 Medium, 11 Low |
| Terraform | TF_TRIVY_IMAGE | docker.io/aquasec/trivy |
not fetched |
| Zola | ZOLA_IMAGE | docker.io/jauderho/zola:latest |
|
| Zola | ZOLA_LYCHEE_IMAGE | docker.io/lycheeverse/lychee:latest |
1 Critical, 2 High, 17 Medium, 76 Low, 1 Unknown |