Skip to content

Template Development Guidelines

This page references GitLab CI development guidelines for people that wish to contribute with templates of their own.

to be continuous architectural principles

As a template developer, you should be aware of to be continuous architectural principles as they generally apply to all templates.

Naming conventions

prefix job names

In order to prevent job names overlap between several templates, each template shall prefix its jobs with a prefix.

For readability reason in the graphical representation of the pipeline, that prefix shall be as short as possible (5 letters max).

Example

mvn for Maven, cf for Cloud Foundry, os for OpenShift, k8s for Kubernetes, ng for Angular

jobs naming

Jobs shall adopt kebab-case convention: lowercase words separated by hyphens.

Example

xxx-build, xxx-test, xxx-code-quality

variables naming

Global variables used and defined by a template shall adopt SCREAMING_SNAKE_CASE convention: uppercase words separated by underscores, and shall be prefixed with the template name or the name of the related tool.

Example

MAVEN_CLI_OPTS, DOCKER_CLAIR_ADDR, SONAR_HOST_URL

It also may happen that some templates evaluate and export (dynamic) local variables. They shall adopt snake_case convention: lowercase words separated by underscores.

Example

hostname variable dynamically evaluated and exported by OpenShift template during deployment jobs

Job prototype

Every template shall define a (hidden) base job with name .<prefix>-base, extended by all other jobs, that defines:

  • the default image to use (variabilized)
  • common cache policy

Example

# xxx job prototype
.xxx-base:
  image: $XXX_IMAGE
  services:
    # this image performs usage tracking
    - name: "registry.gitlab.com/to-be-continuous/tools/tracking:master"
      command: ["--service", "xxx", "1.0.0"] # 2nd arg: template name; 3rd arg: template version 
  # Cache downloaded dependencies and plugins between builds.
  # To keep cache across branches add 'key: "$CI_JOB_NAME"'
  cache:
    key: "$CI_COMMIT_REF_SLUG-xxx"
    paths:
      - binaries/

ℹ the hidden base job is not required for templates implementing only one job.

Make everything overridable

It is not so easy for a project that uses a template to override/change a job implementation, so the best way to make our templates flexible is to use global variables that can be overridden.

The most obvious example is the docker image used by the jobs, that shall be variabilized. But command line arguments shall be defined globally too, and overridable by users.

Example

variables:
  # Docker image is variabilized and can be overridden
  XXX_IMAGE: xxx:latest
  # my build arguments are variabilized and can be overridden
  XXX_BUILD_ARGS: 'build --with-default-args'

# xxx job prototype
.xxx-base:
  image: $XXX_IMAGE
  services:
    # this image performs usage tracking
    - name: "registry.gitlab.com/to-be-continuous/tools/tracking:master"
      command: ["--service", "xxx", "1.0.0"] # 2nd arg: template name; 3rd arg: template version 
  # Cache downloaded dependencies and plugins between builds.
  # To keep cache across branches add 'key: "$CI_JOB_NAME"'
  cache:
    key: "$CI_COMMIT_REF_SLUG-xxx"
    paths:
      - binaries/

xxx-build:
  extends: .xxx-base
  stage: build
  script:
    - xxx $XXX_BUILD_ARGS

Kicker descriptor

Kicker is an online configurer that helps generating .gitlab-ci.yml files interactively.

In order to operate, it needs a JSON file exposed by each template that describes:

  • general information about the template (name, description, kind, ...)
  • general configuration variables (name, description, default value, ...)
  • each supported job (description, variables, ...)

This is described in a kicker.json file, located in each template project root directory.

In order to edit it efficiently, you should configure your JSON editor to use the following JSON schema:

https://gitlab.com/to-be-continuous/kicker/raw/master/kicker-schema-1.json

Guidelines per template type

Build & Test templates

Build & Test templates depend on the language/build system and are in charge of building, (unit) testing and providing all language specific code analysis tools (SAST, dependency check, ...).

They shall implement the following jobs:

  • a build job (mandatory) mapped on the build stage that builds the code (if applicable for the language) and runs the unit tests.
    Unit Tests should support the following GitLab CI features:
  • any number of code analysis jobs (optional) mapped on the test stage that allow triggering language specific code analysis tools (SAST, dependency check, ...)
    • if those jobs are time consuming they shall be triggered manually
    • they shall be enabled/disabled by configuration (ex: a job is disabled by default and is enabled when the SOMEJOB_ENABLED variable is set)
    • whenever possible, code analysis on non-master branches should be a partial code analysis (only on changes in the branch)
  • a publish job (optional) mapped on the publish stage that can be triggered manually to perform a publish of the built artifacts on a package repository (ex: Artifactory, NPM or Maven repository)

Code Analysis templates

Code Analysis templates provide code analysis tools (SAST, dependency check, ...) not dependent on any specific language or build tool (ex: SonarQube, Checkmarx, Coverity).

They shall implement one or several code analysis jobs mapped on the test stage that allow triggering code analysis

  • if those jobs are time consuming they shall be triggered manually
  • they shall be enabled/disabled by configuration (ex: a job is disabled by default and is enabled when the SOMEJOB_ENABLED variable is set)
  • whenever possible, code analysis on non-master branches should be a partial code analysis (only on changes in the branch)

Packaging templates

Packaging templates provide tools allowing to package the code into a specific executable/distributable package (ex: Docker, YUM, DEB, ...)

They shall implement the following jobs:

  • one or several linter jobs (optional) mapped on the build or test stage that perform lint analysis on the packaging descriptors and scripts
  • one or several build jobs (mandatory) mapped on the package-build stage that build the package
  • one or several test jobs (optional) mapped on the package-test stage that test and analyse the built package
  • one or several publish jobs (mandatory) mapped on the publish stage that publish the built package to a compatible package repository

Deploy & Run templates

Deploy & Run templates depend on the hosting (cloud) environment and are in charge of deploying the code to the hosting environment.

They shall implement the following jobs:

  • one generic hidden deployment job (mandatory), in charge of deploying the code to some parameterized environment
    • must be enabled/disabled by configuration
    • must use the resource_group feature to prevent multiple pipelines deploying to the same environment at the same time
    • must propagate the $CI_ENVIRONMENT_URL variable as a environment_url.txt artifact file (legacy way)
    • must propagate deployed environment information as dotenv artifacts (the dotenv way):
      • environment_type: the type of environment (review, integration, staging or production),
      • environment_name: the application name,
      • environment_url: set to $CI_ENVIRONMENT_URL.
  • one generic hidden cleanup job (mandatory), in charge of deleting some parameterized environment
  • one review job (mandatory), in charge of deploying the code on a dynamic review environment
    • extends the generic deployment job
    • mapped on the deploy stage
    • must be executed on non-master, non-develop branches only
    • must reference the cleanup-review job (see below) in its environment:on_stop event
  • one cleanup-review job (mandatory) in charge of deleting the corresponding dynamic review environment
    • extends the generic cleanup job
    • mapped on the deploy stage
    • must be executed on non-master branches only
    • must be associated to the environment:action:stop event
  • one integration job (mandatory) in charge of deploying the code on the integration environment
    • extends the generic deployment job
    • mapped on the deploy stage
    • must be executed on develop branch only
  • one staging job (mandatory) in charge of deploying the code on the staging environment
    • extends the generic deployment job
    • mapped on the deploy stage
    • must be executed on master branch only
  • one production job (mandatory) in charge of deploying the code on the production environment
    • extends the generic deployment job
    • mapped on the production stage
    • must be executed on master branch only

Acceptance templates

Acceptance templates provide acceptance tools (functional testing, performance testing, DAST).

They shall implement one or several acceptance jobs mapped on the acceptance stage that run the acceptance tool.

Those jobs should implement the following features:

  • tests report integration using JUnit test report
  • auto-evaluating the environment url to test based on the presence of $environment_url (possibly propagated by an upstream deployment template using dotenv artifacts) or a environment_url.txt file.