GitLab CI template for GitLab Butler¶

Automate cleaning your projects with GitLab Butler.

Usage¶

This template can be used both as a CI/CD component or using the legacy include:project syntax.

Use as a CI/CD component¶

Add the following to your .gitlab-ci.yml:

include:
  # 1: include the component
  - component: $CI_SERVER_FQDN/to-be-continuous/gitlab-butler/gitlab-ci-butler@1.0.0
    # 2: set/override component inputs
    inputs:
      # ⚠ this is only an example
      group-path: "group/sub-group"
      pipelines-delete-older-than: "90"

Use as a CI/CD template (legacy)¶

Add the following to your .gitlab-ci.yml:

include:
  # 1: include the template
  - project: "to-be-continuous/gitlab-butler"
    ref: "1.0.0"
    file: "/templates/gitlab-ci-butler.yml"

variables:
  # 2: set/override template variables
  # ⚠ this is only an example
  BUTLER_GROUP_PATH: "group/sub-group"
  BUTLER_PIPELINES_DELETE_OLDER_THAN: "90"

Configuration¶

The GitLab Butler template uses some global configuration.

Input / Variable	Description	Default value
`image` / `BUTLER_IMAGE`	The Docker image used to run GitLab Butler	`registry.gitlab.com/to-be-continuous/tools/gitlab-butler:latest`
`BUTLER_TOKEN`	A GitLab access token to allow GitLab Butler crawl your projects	none
`group-path` / `BUTLER_GROUP_PATH`	GitLab group to process	`$CI_PROJECT_NAMESPACE`
`pipelines-delete-older-than` / `BUTLER_PIPELINES_DELETE_OLDER_THAN`	The max age (in days) after which pipelines are deleted	`90`

This template will help you to use GitLab Butler from a GitLab project to automate cleaning your projects. On the contrary to other to-be-continuous templates, this one should be used in a separate project that will be in charge of crawling all your other projects.

Upon including the template, carefully follow GitLab Butler's documentation to configure the tool accordingly. Pay attention to the following: - GitLab Butler requires that you declare a BUTLER_TOKEN variable with an access token. - Define the GitLab group that contain projects to be cleaned with BUTLER_GROUP_PATH. - Configure a scheduled pipeline to run cleaning at regular interval.

Default GitLab Butler configuration¶

This template is designed to be run on GitLab.

Defaults values are set to manage GitLab Butler behavior:

GitLab Butler variable	Value
`BUTLER_GITLAB_API`	GitLab source API url (defaults to `$CI_API_V4_URL`)
`BUTLER_EXCLUDE`	project/group path(s) to exclude (coma separated list)
`BUTLER_PIPELINES_DELETION_LIMIT`	maximum number of pipelines deleted per project (default = no limit)
`BUTLER_PIPELINES_KEEP_PER_BRANCH`	number of pipelines to keep per branch (⚠ branch MUST still exist)
`BUTLER_PIPELINES_KEEP_PER_TAG`	number of pipelines to keep per tag (⚠ tag MUST still exist)
`BUTLER_PIPELINES_KEEP_PER_MR`	number of pipelines to keep per merge request (⚠ MR MUST still exist)
`BUTLER_PIPELINES_DELETE_OLDER_THAN`	max age (in days) after which pipelines are deleted (unless they are kept by a keep rule)

Dry-run implementation details¶

Depending on the source of a pipeline, the template will either perform cleaning or perform a dry run (to preview the behavior of GitLab Butler with logs, without making any changes to your projects).

The real cleaning is triggered only when:

the pipeline is started from the web interface (menu Build > Pipelines, click Run pipeline),
or the pipeline is started from a scheduled run (menu Build > Pipeline schedules).

In any other case (regular Git commit or else) the template will perform a dry run.

Dry-run behavior is enabled/disabled with the BUTLER_DRY_RUN environment variable. Please don't try to override it, or you might break the default template implementation.

Removing backlog¶

If you use GitLab Butler on groups/projects that had an extensive use of CI/CD pipelines for a long time, the tool will try to remove a large number of pipelines on the first run.

In that case, you can use BUTLER_PIPELINES_DELETION_LIMIT variable to limit number of pipelines deleted during one run. The remaining will be removed on later runs.

Per example, if your project has accumulated 5_000 pipelines over 3 years, using BUTLER_PIPELINES_DELETION_LIMIT=300 will remove at most 300 pipelines per run. With a daily scheduled execution, GitLab Butler will clean the buildup in a few days.