GitLab CI template for Source-to-Image¶

This project implements a GitLab CI/CD template to build reproducible container images from your source code with Source-to-Image (S2I).

Usage¶

This template can be used both as a CI/CD component or using the legacy include:project syntax.

Use as a CI/CD component¶

Add the following to your .gitlab-ci.yml:

include:
  # 1: include the component
  - component: $CI_SERVER_FQDN/to-be-continuous/s2i/gitlab-ci-s2i@1.1.1
    # 2: set/override component inputs
    inputs:
      builder-image: "fabric8/s2i-java:latest-java11" # ⚠ this is only an example

Use as a CI/CD template (legacy)¶

Add the following to your .gitlab-ci.yml:

include:
  # 1: include the template
  - project: 'to-be-continuous/s2i'
    ref: '1.1.1'
    file: '/templates/gitlab-ci-s2i.yml'

variables:
  # 2: set/override template variables
  S2I_BUILDER_IMAGE: "fabric8/s2i-java:latest-java11" # ⚠ this is only an example

Understanding the S2I template¶

Global configuration¶

The S2I template uses some global configuration used throughout all jobs.

Input / Variable	Description	Default value
`dind-image` / `S2I_DIND_IMAGE`	The Docker image used to run the Docker daemon	`registry.hub.docker.com/library/docker:dind`
`skopeo-image` / `S2I_SKOPEO_IMAGE`	The container image used to run skopeo	`quay.io/skopeo/stable:latest`
`version` / `S2I_VERSION`	The target S2I version to install/use	`latest`
`platform` / `S2I_PLATFORM`	The target S2I platform to install/use	`linux-386`

Images¶

The S2I template builds a container image that may be pushed as two distinct images, depending on a certain workflow:

snapshot: the image is first built and pushed to some container registry as the snapshot image. It can be seen as the raw result of the build, but still untested and unreliable.
release: once the snapshot image has been thoroughly tested (both by package-test stage jobs and/or acceptance stage jobs after being deployed to some server), then the image is pushed one more time as the release image. This second push can be seen as the promotion of the snapshot image being now tested and reliable.

In practice:

the snapshot image is always pushed by the template (pipeline triggered by a Git tag or commit on any branch),
the release image is only pushed:
- on a pipeline triggered by a Git tag,
- on a pipeline triggered by a Git commit on master.

The snapshot and release images are defined by the following variables:

Input / Variable	Description	Default value
`snapshot-image` / `S2I_SNAPSHOT_IMAGE`	S2I snapshot image	`$CI_REGISTRY_IMAGE/snapshot:$CI_COMMIT_REF_SLUG`
`release-image` / `S2I_RELEASE_IMAGE`	S2I release image	`$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME`

As you can see, the S2I template is configured by default to use the GitLab container registry. You may perfectly override this and use another container registry, but be aware of a few things:

the S2I_SNAPSHOT_IMAGE requires a container registry that allows tag overwrite,
the S2I_RELEASE_IMAGE may use a container registry that doesn't allow tag overwrite, but:
1. you should avoid overwriting a Git tag (at it will obviously fail while trying to (re)push the image),
2. you have to deactivate publish on main (or master) branch by setting the $S2I_PROD_PUBLISH_STRATEGY variable to none (as it would lead to the main tag being overwritten).

Registries and credentials¶

As seen in the previous chapter, the S2I template uses by default the GitLab registry to push snapshot and release images. Thus it makes use of credentials provided by GitLab itself to login (CI_REGISTRY_USER / CI_REGISTRY_PASSWORD).

But when using other registry(ies), you'll have also to configure appropriate credentials.

Using the same registry for snapshot and release¶

If you use the same registry for both snapshot and release images, you shall use the following configuration variables:

Input / Variable	Description
`S2I_REGISTRY_USER`	container registry username for image registry
`S2I_REGISTRY_PASSWORD`	container registry password for image registry

Using different registries for snapshot and release¶

If you use different registries for snapshot and release images, you shall use separate configuration variables:

Input / Variable	Description
`S2I_REGISTRY_SNAPSHOT_USER`	container registry username for snapshot image registry
`S2I_REGISTRY_SNAPSHOT_PASSWORD`	container registry password for snapshot image registry
`S2I_REGISTRY_RELEASE_USER`	container registry username for release image registry
`S2I_REGISTRY_RELEASE_PASSWORD`	container registry password for release image registry

Secrets management¶

Here are some advices about your secrets (variables marked with a ):

Manage them as project or group CI/CD variables:
- masked to prevent them from being inadvertently displayed in your job logs,
- protected if you want to secure some secrets you don't want everyone in the project to have access to (for instance production secrets).
In case a secret contains characters that prevent it from being masked, simply define its value as the Base64 encoded value prefixed with @b64@: it will then be possible to mask it and the template will automatically decode it prior to using it.
Don't forget to escape special characters (ex: $ -> $$).

Jobs¶

`s2i-build` job¶

This job builds the image and publishes it to the snapshot repository.

It is bound to the package-build stage, and uses the following variables:

It uses the following variable:

Input / Variable	Description	Default value
`root-dir` / `S2I_ROOT_DIR`	Relative path to the application source code base directory in your repository	`.`
`builder-image` / `S2I_BUILDER_IMAGE`	The S2I builder image used to build your application image	none (required)
`build-extra-flags` / `S2I_BUILD_EXTRA_FLAGS`	S2I build extra flags	none

This job produces output variables that are propagated to downstream jobs (using dotenv artifacts):

Input / Variable	Description	Example
`s2i_image`	snapshot image name with tag	`registry.gitlab.com/acme/website/snapshot:main`
`s2i_image_digest`	snapshot image name with digest (no tag)	`registry.gitlab.com/acme/website/snapshot@sha256:b7914a91...`
`s2i_repository`	snapshot image bare repository (no tag nor digest)	`registry.gitlab.com/acme/website/snapshot`
`s2i_tag`	snapshot image tag	`main`
`s2i_digest`	snapshot image digest	`sha256:b7914a91...`

They may be freely used in downstream jobs (for instance to deploy the upstream built image, whatever the branch or tag).

`s2i-publish` job¶

This job pushes (promotes) the built image as the release image using skopeo.

Input / Variable	Description	Default value
`publish-args` / `S2I_PUBLISH_ARGS`	Additional `skopeo copy` arguments	(none)
`prod-publish-strategy` / `S2I_PROD_PUBLISH_STRATEGY`	Defines the publish to production strategy. One of `manual` (i.e. one-click), `auto` or `none` (disabled).	`manual`
`release-extra-tags-pattern` / `S2I_RELEASE_EXTRA_TAGS_PATTERN`	Defines the image tag pattern that `$S2I_RELEASE_IMAGE` should match to push extra tags (supports capturing groups - see below)	`^v?(?P<major>[0-9]+)\\.(?P<minor>[0-9]+)\\.(?P<patch>[0-9]+)(?P<suffix>(?P<prerelease>-[0-9A-Za-z-\\.]+)?(?P<build>\\+[0-9A-Za-z-\\.]+)?)$` (SemVer pattern)
`release-extra-tags` / `S2I_RELEASE_EXTRA_TAGS`	Defines extra tags to publish the release image (supports capturing group references from `$S2I_RELEASE_EXTRA_TAGS_PATTERN` - see below)	(none)
`semrel-release-disabled` / `S2I_SEMREL_RELEASE_DISABLED`	Set to `true` to disable semantic-release integration	none (enabled)

This job produces output variables that are propagated to downstream jobs (using dotenv artifacts):

Input / Variable	Description	Example
`s2i_image`	release image name with tag	`registry.gitlab.com/acme/website:main`
`s2i_image_digest`	release image name with digest (no tag)	`registry.gitlab.com/acme/website@sha256:b7914a91...`
`s2i_repository`	release image bare repository (no tag nor digest)	`registry.gitlab.com/acme/website`
`s2i_tag`	release image tag	`main`
`s2i_digest`	release image digest	`sha256:b7914a91...`

They may be freely used in downstream jobs (for instance to deploy the upstream built image, whatever the branch or tag).

Using extra tags¶

When publishing the release image, the S2I template might publish it again with additional tags (aliases):

the original published image tag (extracted from $S2I_RELEASE_IMAGE) must match $S2I_RELEASE_EXTRA_TAGS_PATTERN (semantic versioning pattern by default),
extra tags to publish can be defined in the $S2I_RELEASE_EXTRA_TAGS variable, each separated with a whitespace.

the S2I template supports group references substitution to evaluate extra tags:

$S2I_RELEASE_EXTRA_TAGS_PATTERN supports capturing groups:
- v([0-9]+)\.([0-9]+)\.([0-9]+) has 3 (unnamed) capturing groups, each capturing any number of digits
- v(?<major>[0-9]+)\.(?<minor>[0-9]+)\.(?<patch>[0-9]+) has 3 named capturing groups (major, minor and patch), each capturing any number of digits
$S2I_RELEASE_EXTRA_TAGS supports capturing group references from $S2I_RELEASE_EXTRA_TAGS_PATTERN:
- \g1 is a reference to capturing group number 1
- \g<major> is a reference to capturing group named major

the default value of $S2I_RELEASE_EXTRA_TAGS_PATTERN matches and captures all parts of a standard semantic versioning-compliant tag:

the major group captures the major version
the minor group captures the minor version
the patch group captures the patch version
the prerelease group captures the (optional) pre-release version (including the leading -)
the build group captures the (optional) build version (including the leading +)
the suffix group captures the (optional) entire suffix (including pre-release and/or build)

Example: publish latest, major.minor and major aliases for a SemVer release:

variables:
  # ⚠ don't forget to escape backslash character in yaml
  S2I_RELEASE_EXTRA_TAGS: "latest \\g<major>.\\g<minor>\\g<build> \\g<major>\\g<build>"

With this contiguration, the following extra tags would be published:

original tag	extra tags
`main`	none (doesn't match `$S2I_RELEASE_EXTRA_TAGS_PATTERN`)
`some-manual-tag`	none (doesn't match `$S2I_RELEASE_EXTRA_TAGS_PATTERN`)
`1.2.3`	`latest`, `1.2`, `1`
`1.2.3-alpha.12`	`latest`, `1.2`, `1`
`1.2.3+linux`	`latest`, `1.2+linux`, `1+linux`
`1.2.3-alpha.12+linux`	`latest`, `1.2+linux`, `1+linux`

`semantic-release` integration¶

If you activate the semantic-release-info job from the semantic-release template, the s2i-publish job will rely on the generated next version info.

the release will only be performed if a semantic release is present
the tag will be based on SEMREL_INFO_NEXT_VERSION, it will override S2I_RELEASE_IMAGE by simply substituting the tag, or adding a tag when there's none.

For instance, in both cases:

S2I_RELEASE_IMAGE: "registry.gitlab.com/$CI_PROJECT_NAME"
S2I_RELEASE_IMAGE: "registry.gitlab.com/$CI_PROJECT_NAME:$CI_COMMIT_REF_NAME"

The published container image will be registry.gitlab.com/$CI_PROJECT_NAME:$SEMREL_INFO_NEXT_VERSION and all subsequent jobs relying on the s2i_image variable will be provided with this tag.

When semantic-release detects no release (i.e. either the semantic-release template is misconfigured, or there were simply no feat/fix commits), the s2i-publish job will report a warning and no image will be pushed in the release registry. In such a case, the s2i_image remains unchanged, and will refer to the snapshot image. Any subsequent job that may deploy to production (Kubernetes or Openshift), should thus be configured not to deploy in this situation. Refer to deployment template for more information.

Finally, the semantic-release integration can be disabled with the S2I_SEMREL_RELEASE_DISABLED variable.