Skip to content

Architectural Principles

This page presents fundamental to be continuous architectural principles and their justification.

Common features

Here is the list of common to be continuous features that must be implemented by every template:

Standard stages

Every template shall reuse defined generic pipeline stages.

If an additional stage seems to be required, that must be discussed with the core team first.

Sensible defaults

Examples:

  • When introducing a new toggleable feature (e.g. a new SAST job), decide wisely whether it should be enabled by default (opt-out) or disabled by default (opt-in).
  • When defining default tool CLI options, select the most appropriate options. If unsure: prefer the option that raises the bar (of code quality/best practices) rather than the conservative one

GitLab by default

When implementing a feature requiring an external service (ex: a packages registry) and if GitLab provides such a service, then:

  • Everything must be done to configure the feature to use GitLab's service by default (zero config).
  • If not possible, at least all the required documentation shall be added to the README.md to explain what has to be done to configure the GitLab service properly.

Examples:

  • The Docker template uses GitLab's container registry by default, although any other container registry can be used by configuration.
  • The Python template uses GitLab's PyPI registry by default, although any other PyPI registry can be used by configuration.
  • The Maven template can't use GitLab's Maven registry by default, but documents what has to be done in its README.md.

The right template perimeter

to be continuous provides templates organized around languages and technologies.\ But sometimes it's not very easy to determine what a template should do and where a new template should be created for additional features.\ to be continuous templates are organized around the input file(s) and/or a CLI tool that the job apply to.

Examples:

  • The Maven template relies on the mvn tool and pom.xml in the repo, similarly the Gradle template relies on the gradletool and the build.gradle file.
  • The Python template relies on the presence of Python source code in the repository, but still supports various dependency management and build tools.
  • Kubernetes relies on kubectl, but also embeds additional tools related to Kubernetes manifests.
  • The Docker template provides a coherent set of tools to build & test container images from a Dockerfile. For instance it might (and it will soon) support various tools to bake the container image from the Dockerfile (e.g. Docker-in-Docker, kaniko, podman or buildah).

No nested includes

Nested includes shall not be used in to be continuous templates for the following reasons:

  • it would prevent to be continuous templates from being remote included (b.t.w. this is the technique used by R2Devops, also featuring to be continuous templates)
  • it would prevent to be continuous templates from being included from another root path than /to-be-continuous (it may happen in some organizations when the groups hierarchy is locked by governance rules)
  • should the nested includes be versioned or not? both options have serious drawback (dependency hell vs. non-deterministic)

Standard environments

Every infrastructure(-as-code) and deployment template shall support 4 kinds of environments (each being optional):

Environment Type Description Associated branch(es)
Review Those are dynamic and ephemeral environments to deploy your ongoing developments.
It is a strict equivalent of GitLab's Review Apps feature.		 All development branches (non-integration, non-production)
Integration A single environment to continuously deploy your integration branch. The integration branch (develop by default)
Staging A single environment to continuously deploy your production branch.
It is an iso-prod environment, meant for running the automated acceptance tests prior to deploying to the production env.		 The production branch (main or master by default)
Production Well.. the prod! The production branch (main or master by default)

Container images

Explicit official image registry

Not specifying explicitly the official image registry in the image name enables a supply chain attack by pushing a malicious image with the same name in alternate registries and expect a mirror will pick one of the alternate registries before the official one.

In order to protect against this kind of attacks, to be continuous always use fully qualified image names (i.e. including the registry).

Latest version by default

As stated in our documentation, to be continuous templates use required tools as container images. And when available, the latest version is used.

Why latest?

  • there is no good default for everyone: every project should select a specific version that fits their needs
  • the latest version has more chances to be up-to-date with security patches (at least for an official and maintained project)

Test alongside build

For built languages, we've decided that build and test should stand in the same GitLab job (at least by default). This choice is for performance reasons, as testing often requires to build first, and doing the 2 steps in separate jobs might involve redoing part of the build 2 times, even when cache is properly implemented.

Tools reports

to be continuous templates features many tools that produce reports (testing tools, SAST, DAST, linters, vulnerability scanners...).

Report formats

We decided that those tools:

Report files naming convention

The output report file path must comply to the following convention:

<project_root_dir>/reports/<job_name>.<format_name>.<extension>

Examples:

  • $NG_WORKSPACE_DIR/reports/ng-e2e.xunit.xml is the end-to-end test execution report produced by the Angular template (ng), using the xUnit/JUnit format (XML)
  • $CYPRESS_PROJECT_DIR/reports/cypress-*.xunit.xml is the test execution report produced by the Cypress template, using the xUnit/JUnit format (XML)
  • $PYTHON_PROJECT_DIR/reports/py-coverage.cobertura.xml is the coverage report produced by the Python template (py), using the Cobertura format (XML)
  • reports/docker-sbom-*.cyclonedx.json is the SBOM report produced by the Docker template, using the CycloneDX format (JSON)
  • reports/docker-hadolint-*.codeclimate.json is the Hadoling report produced by the Docker template, using the CodeClimate format (JSON)
  • reports/docker-hadolint-*.native.json is the Hadoling report produced by the Docker template, using the Hadolint native format (JSON)
    ⚠ native should be used as format name when it is the native tool's format

Dotenv artifacts

to be continuous templates are composable. In other words they cooperate gracefully with others to minimize the amount of integration work. It is common that a job produces output that could be used by downstream jobs, possibly from other templates. Those output can be artifacts (ex: result of a build), reports (ex: SAST tools reports consumed by the SonarQube template or DefectDojo), and sometimes it could be a dynamic variable.

For this last need, to be continuous uses the dotenv report format to propagate output environment variables downstream.

Examples:

  • every deployment template propagates the deployed environment as a $environment_url, that can be used by any acceptance testing template, whichever the deployment technology or the testing tools
  • the Docker and Cloud Native Buildpack templates both propagate several variables of the built container image (container registry, full url with tag, full url with digest, tag only...) that can be used in downstream templates to pull this image

Non-backward compatible GitLab features

Some GitLab features are not backward compatible. For instance features that introduce a new keyword that would break the .gitlab-ci.yml validation on previous versions of GitLab.

In that case, to be continuous templates shall observe a reasonable rollout time before implementing them, not to break projects using them on a self-managed server.

For security patches, GitLab maintains the two previous major.minor versions. We decided to observe the same delay before implementing a non-backward compatible feature. I.e. wait that the GitLab version that introduced the feature is the oldest version maintained by GitLab.

In practice, this is about 3 months.

Package publish

Each template capable of publishing (versioned) packages, shall implement it according to the common publish design.

Decorelating the publish job from the release process ensures that all publish-capable templates will always take care of publishing their own package(s), whichever the technique that was used to trigger the release (manual, semantic-release or any other alternative).