GitLab CI template for sbt

This project implements a GitLab CI/CD template to build, test and analyse your sbt-based projects.

Languages supported by this build tool are :

• Scala
• Scala.js
• Scala-native
• Java

Usage

This template can be used both as a CI/CD component or using the legacy include:project syntax.

Use as a CI/CD component

Add the following to your .gitlab-ci.yml:

include:
  # 1: include the component
  - component: $CI_SERVER_FQDN/to-be-continuous/sbt/gitlab-ci-sbt@1.7.1
    # 2: set/override component inputs
    inputs:
      image: "registry.hub.docker.com/sbtscala/scala-sbt:17.0.2_1.6.2_3.1.3" # ⚠ this is only an example

Use as a CI/CD template (legacy)

Add the following to your .gitlab-ci.yml:

include:
  # 1: include the template
  - project: 'to-be-continuous/sbt'
    ref: '1.7.1'
    file: '/templates/gitlab-ci-sbt.yml'

variables:
  # 2: set/override template variables
  SBT_IMAGE: "registry.hub.docker.com/sbtscala/scala-sbt:17.0.2_1.6.2_3.1.3" # ⚠ this is only an example

Project prerequisites

dependencies resolution

In order to configure your project to use a repository cache/proxy such as artifactory or nexus, add those two files into your project root directory :

• .sbtopts with this content :

  -Dsbt.override.build.repos=true
  -Dsbt.repository.config=.sbtrepositories
  

• .sbtrepositories with such content :

  [repositories]
  local
  mirror_of_central: https://mavenrepo.acme.host/maven-proxy-asis-apache-releases
  confluent-releases : https://mavenrepo.acme.host/maven-proxy-asis-confluent-releases
  

If some repositories requires credentials, add a file credentials.sbt (you can choose the basename you want) to your project home directory with such content :

credentials ++= {
  List("MAVEN_REPOSITORY_HOST", "MAVEN_REPOSITORY_USERNAME", "MAVEN_REPOSITORY_PASSWORD").flatMap(sys.env.get) match {
    case List(host, user, password) =>
      streams.value.log.info(s"private/internal dependencies resolution credentials for host '$host' user '$user' has been configured")
      Some(Credentials("Artifactory Realm", host, user, password))

    case _ =>
      streams.value.log.warn(
        "private/internal dependencies resolution may fail as some environment variables are missing : MAVEN_REPOSITORY_USERNAME / MAVEN_REPOSITORY_PASSWORD / MAVEN_REPOSITORY_HOST"
      )
      None
  }
}

Which uses three environment variables MAVEN_REPOSITORY_HOST, MAVEN_REPOSITORY_USERNAME, MAVEN_REPOSITORY_PASSWORD from which sbt will extract credentials information.

Note that thanks to those configurations files, your favorite IDE will be able to automatically take the right repository configuration without any specific configuration.

Global configuration

The sbt template uses some global configuration used throughout all jobs.

Input / Variable	Description	Default value
image / SBT_IMAGE	The Docker image used to run sbt set the version required by your project	registry.hub.docker.com/sbtscala/scala-sbt:17.0.2_1.6.2_3.1.3
opts / SBT_OPTS	Global sbt options	-Dsbt.global.base=sbt-cache/sbtboot -Dsbt.boot.directory=sbt-cache/boot -Dsbt.coursier.home=sbt-cache/coursier -Dsbt.ci=true -Dsbt.color=always
cli-opts / SBT_CLI_OPTS	Additional sbt options used on the command line	--batch

As you can see, your sbt cache policy is preconfigured to use local cache in sbt-cache directory (not to download dependencies over and over again).

If you have a good reason to do differently, you'll have to override the SBT_OPTS variable as well as the cache policy.

Jobs

sbt-build job

The sbt template features a job sbt-build that performs build and tests at once. This stage is performed in a single job for optimization purpose (it saves time) and also for test jobs dependency reasons (some test jobs such as SONAR analysis have a dependency on test results).

It uses the following environment variable:

Input / Variable	Description	Default value
build-args / SBT_BUILD_ARGS	sbt arguments for the build job packaging	clean package
test-args / SBT_TEST_ARGS	sbt arguments for the build job test phase	coverage test coverageAggregate

Note: - Always keep the clean goal to generate bytes code without any instrumentation coming from the test phasis.

About Code Coverage

With its default arguments, the GitLab CI template for sbt forces the use of sbt-coverage to compute code coverage during unit tests execution.

But this requires that you add this plugin to your project:

Add the plugin in your project/plugins.sbt:

addSbtPlugin("org.scoverage" % "sbt-scoverage" % "2.0.1") // Check for newer releases : https://search.maven.org/artifact/org.scoverage/sbt-scoverage

sbt-sbom job

This job generates a SBOM file listing all dependencies using syft.

It is bound to the test stage, and uses the following variables:

Input / Variable	Description	Default value
sbom-disabled / SBT_SBOM_DISABLED	Set to true to disable this job	none
sbom-image / SBT_SBOM_IMAGE	The syft image used for SBOM analysis	registry.hub.docker.com/anchore/syft:debug
sbom-opts / SBT_SBOM_OPTS	Options for syft used for SBOM analysis	dir:sbt-cache/coursier --catalogers java-cataloger

In addition to logs in the console, this job produces the following reports, kept for one week:

Report	Format	Usage
reports/sbt-sbom.cyclonedx.json	CycloneDX JSON	Security & Compliance integration

publish jobs

The sbt template supports publishing:

• snapshot artifacts (on any branch change),
• and/or release artifacts (on versioned tags such as x.y.z)

The expected behavior of publish jobs is controlled with the SBT_PUBLISH_MODE variable, supporting one of the following values:

• none (default): Publishing is disabled.
• snapshot: Auto-publishes snapshot artifacts on any branch change.
• ontag: Auto-publishes snapshot artifacts on any branch change, and publishes release artifacts as soon as a new release tag is committed. In this mode it is up to you to manage release numbering and tagging operations.
• release: Auto-publishes snapshot artifacts on any branch change, and implements a full publishing workflow with automatic release numbering and git tags management (see release job below for details).

Repository authentication

Your publication repository(ies) may require authentication credentials to publish artifacts.

You shall handle them in the following way:

1. define all required credentials as: lock: project variables,
2. define the target repositories and the credentials with a dedicated sbt file such as publish.sbt in your project home directory

Example:

Define the following environment variables :

Input / Variable	Description
maven-repository-publish-snapshot-url / MAVEN_REPOSITORY_PUBLISH_SNAPSHOT_URL	Destination repository to publish snpashot artifacts
maven-repository-publish-release-url / MAVEN_REPOSITORY_PUBLISH_RELEASE_URL	Destination repository to publish release artifacts
MAVEN_REPOSITORY_PUBLISH_USERNAME	Repository Username
MAVEN_REPOSITORY_PUBLISH_PASSWORD	Repository Password

credentials ++= {
  val publishURLProperty = if (isSnapshot.value) "MAVEN_REPOSITORY_PUBLISH_SNAPSHOT_URL" else "MAVEN_REPOSITORY_PUBLISH_RELEASE_URL"

  List(publishURLProperty, "MAVEN_REPOSITORY_PUBLISH_USERNAME", "MAVEN_REPOSITORY_PUBLISH_PASSWORD").flatMap(sys.env.get) match {
    case List(url, username, password) =>
      val host = new java.net.URL(url).getHost
      streams.value.log.info(s"publish/release credentials for host '$host' user '$username' has been configured")
      Some(Credentials("Artifactory Realm", host, username, password))

    case _ =>
      streams.value.log.warn(
        "publish/release may fail as some environment variables are missing : MAVEN_REPOSITORY_PUBLISH_USERNAME / MAVEN_REPOSITORY_PUBLISH_PASSWORD / MAVEN_REPOSITORY_PUBLISH_SNAPSHOT_URL / MAVEN_REPOSITORY_PUBLISH_RELEASE_URL"
      )
      None
  }
}

publishTo := {
  val publishURLProperty = if (isSnapshot.value) "MAVEN_REPOSITORY_PUBLISH_SNAPSHOT_URL" else "MAVEN_REPOSITORY_PUBLISH_RELEASE_URL"

  val mayBeTarget =
    if (isSnapshot.value) sys.env.get(publishURLProperty).map(url => "snapshots on artifactory".at(url))
    else sys.env.get(publishURLProperty).map(url => "releases on artifactory".at(url))

  mayBeTarget match {
    case None         => streams.value.log.warn(s"publish repository target url not given ($publishURLProperty)")
    case Some(target) => streams.value.log.info(s"publish to ${target.name} -> $target")
  }
  mayBeTarget
}

// To avoid error such as "java.net.ProtocolException: Too many follow-up requests: 21"
updateOptions := updateOptions.value.withGigahorse(false)

sbt-release job

This job is disabled by default and can be activated by setting the SBT_PUBLISH_MODE to release.

It is based on the sbt-release plugin. You'll have to add it in your project/plugins.sbt:

// Check for newer releases : https://search.maven.org/artifact/com.github.sbt/sbt-release
addSbtPlugin("com.github.sbt" % "sbt-release" % "1.1.0") 

SCM authentication

A sbt release involves some Git push operations. For this you shall use a SSH key.

We recommend you to use a project deploy key with write access to your project.

The key should not have a passphrase ( see how to generate a new SSH key pair).

Specify $GIT_PRIVATE_KEY as protected project variable with the private part of the deploy key.

-----BEGIN 0PENSSH PRIVATE KEY-----
blablabla
-----END OPENSSH PRIVATE KEY-----