GitLab CI template for Cloud Foundry¶

This project implements a GitLab CI/CD template to deploy your application to a Cloud Foundry platform as well as to manage service instances that can be bound to those applications.

Usage¶

This template can be used both as a CI/CD component or using the legacy include:project syntax.

Use as a CI/CD component¶

Add the following to your .gitlab-ci.yml:

include:
  # 1: include the component
  - component: "gitlab.com/to-be-continuous/cloud-foundry/gitlab-ci-cf@4.6.2"
    # 2: set/override component inputs
    inputs:
      # ⚠ this is only an example
      url: https://api.cloud-foundry.acme.host
      org: MyProject

Use as a CI/CD template (legacy)¶

Add the following to your gitlab-ci.yml:

include:
  # 1: include the template
  - project: "to-be-continuous/cloud-foundry"
    ref: "4.6.2"
    file: "/templates/gitlab-ci-cf.yml"

variables:
  # 2: set/override template variables
  # ⚠ this is only an example
  CF_URL: "https://api.cloud-foundry.acme.host"
  CF_ORG: "MyProject"

Understand¶

This chapter introduces key notions and principle to understand how this template works.

Managed deployment environments¶

This template implements continuous delivery/continuous deployment for projects hosted on a Cloud Foundry platform.

It allows you to manage automatic deployment & cleanup of standard predefined environments. Each environment can be enabled/disabled by configuration. If you're not satisfied with predefined environments and/or their associated Git workflow, you may implement you own environments and workflow, by reusing/extending the base (hidden) jobs. This is advanced usage and will not be covered by this documentation.

The following chapters present the managed predefined environments and their associated Git workflow.

Review environments¶

The template supports review environments: those are dynamic and ephemeral environments to deploy your ongoing developments (a.k.a. feature or topic branches).

When enabled, it deploys the result from upstream build stages to a dedicated and temporary environment. It is only active for non-production, non-integration branches.

It is a strict equivalent of GitLab's Review Apps feature.

It also comes with a cleanup job (accessible either from the environments page, or from the pipeline view).

Integration environment¶

If you're using a Git Workflow with an integration branch (such as Gitflow), the template supports an integration environment.

When enabled, it deploys the result from upstream build stages to a dedicated environment. It is only active for your integration branch (develop by default).

Production environments¶

Lastly, the template supports 2 environments associated to your production branch (master or main by default):

a staging environment (an iso-prod environment meant for testing and validation purpose),
the production environment.

You're free to enable whichever or both, and you can also choose your deployment-to-production policy:

continuous deployment: automatic deployment to production (when the upstream pipeline is successful),
continuous delivery: deployment to production can be triggered manually (when the upstream pipeline is successful).

Supported authentication methods¶

The Cloud Foundry template supports basic authentication only (user/password login). Those credentials - as most of the configuration parameters - can be defined globally and overridden per environment.

Deployment context variables¶

In order to manage the various deployment environments, this template provides a couple of dynamic variables that you might use in your hook scripts, deployment manifests and other deployment resources:

${environment_type}: the current deployment environment type (review, integration, staging or production)
${environment_name}: a generated application name to use for the current deployment environment (ex: myapp-review-fix-bug-12 or myapp-staging) - details below

Generated environment name¶

The ${environment_name} variable is generated to designate each deployment environment with a unique and meaningful application name. By construction, it is suitable for inclusion in DNS, URLs... It is built from:

the application base name (defaults to $CI_PROJECT_NAME but can be overridden globally and/or per deployment environment - see configuration variables)
GitLab predefined $CI_ENVIRONMENT_SLUG variable (sluggified name, truncated to 24 characters)

The ${environment_name} variable is then evaluated as:

<app base name> for the production environment
<app base name>-$CI_ENVIRONMENT_SLUG for all other deployment environments
${environment_name} can also be overriden per environment with the appropriate configuration variable

Examples (with an application's base name myapp):

`$environment_type`	Branch	`$CI_ENVIRONMENT_SLUG`	`$environment_name`
`review`	`feat/blabla`	`review-feat-bla-xmuzs6`	`myapp-review-feat-bla-xmuzs6`
`integration`	`develop`	`integration`	`myapp-integration`
`staging`	`main`	`staging`	`myapp-staging`
`production`	`main`	`production`	`myapp`

Deployment method¶

The Cloud Foundry template supports a versatile way to deploy your application, based on an app manifest you have to provide in your project.

The template processes the following steps:

optionally executes the cf-pre-push.sh script in your project to perform specific environment pre-initialization (for e.g. create required services),
optionally executes the cf-pre-start.sh script in your project.
This hook script is only necessary if your application has dependencies that cannot be described in the manifest and which require the application to exist on the platform (for e.g. mapping internal routes).
When found, the cf push operation is executed with the --no-start option, then this hook script is executed, and finally your app is cf start-ed.
looks for your Cloud Foundry app manifest file, performs variables substitution and cf push it,
look for an env-specific manifest-$environment_type.yml in your project (e.g. manifest-staging.yml for staging environment),
fallbacks to default manifest.yml.
optionally executes the cf-readiness-check.sh to wait & check for the application to be ready (if not found, the template assumes the application was successfully started).
optionally executes the cf-post-push.sh script in your project to perform specific environment post-initialization stuff,
optionally executes the cf-post-bluegreen.sh after a blue-green deployment, to perform specific environment post-initialization.

Zero-downtime deployment¶

Historically Cloud Foundry did not provide any feature for deploying a new version without downtime. Therefor template implemented a blue-green method. It can be enabled with the CF_XXX_ZERODOWNTIMEvariable and it is enabled by default for production.

Drawbacks:

This solution is complex, runs a full copy of the application and so requires to double the allocated resources (cpu, ram and disk)

Starting with CF CLI v7, a new built-in feature called rolling depoyment was added. It allows to rollout instances one by one instead of switching all instances at once.

This feature can be enabled with rolling-strategy input or CF_ROLLING_STRATEGY variable. Enabling it as a few implications:

All environments will use this strategy. So $CF_XXX_ZERODOWNTIME is ignored.
The cf-pre-start.sh hook is only called during the first deployment as --no-start is incompatible with the strategy.
During the next deployments, two versions of your application will accept requests concurrently. It may have a impact on:
Single instance application which will temporarily become concurrent,
Atomic actions like database migrations which might break the previous version.
Deployment won't stop if cf-readiness-check.sh hook fails. So the health-check-http-endpoint should check all services availability.
The cf-post-bluegreen.sh hook is not called as it is not a blue-green deployment.

Cleanup method¶

The Cloud Foundry template also supports a method for deleting your environments (actually only review environments).

The template processes the following steps:

optionally executes the cf-pre-cleanup.sh script in your project to perform specific environment pre-cleanup,
looks for your Cloud Foundry app manifest file, performs variables substitution and cf delete it,
look for an env-specific manifest-$environment_type.yml in your project (e.g. manifest-staging.yml for staging environment),
fallbacks to default manifest.yml.
optionally executes the cf-post-cleanup.sh to perform specific environment post-cleanup (for e.g. delete bound services).

Cleanup job limitations¶

When using this template, you have to be aware of one limitation (bug) with the cleanup job.

By default, the cleanup job triggered automatically on branch deletion will fail due to not being able to fetch the Git branch prior to executing the job (sounds obvious as the branch was just deleted). This is pretty annoying, but as you may see above, deleting an env may require scripts from the project...

So, what can be done about that?

if your project doesn't require any delete script (in other words a simple cf delete is enough to clean-up everything): you could simply override the cleanup job Git strategy to prevent from fetching the branch code:
```
cf-cleanup-review:
  variables:
    GIT_STRATEGY: none
```
in any other case, we're just sorry about this bug, but there is not much we can do:
remind to delete your review env manually before deleting the branch
otherwise you'll have to do it afterwards from your computer (using cf CLI) or from the Cloud Foundry console.

Using variables¶

You have to be aware that your deployment (and cleanup) implementation have to be able to cope with various environments, each with different application names, exposed routes, settings, ... Part of this complexity can be handled by the lookup policies described above (ex: one manifest per env) and also by using available environment variables:

deployment context variables provided by the template:
${environment_type}: the current environment type (review, integration, staging or production)
${environment_name}: the application name to use for the current environment (ex: myproject-review-fix-bug-12 or myproject-staging)
${hostname}: the environment hostname, extracted from the current environment url (after late variable expansion - see below)
${tmp_environment_name}: the application current name being used during the deployment in this environment (different from ${environment_name} during a blue/green deployment for instance)
${domain}: the Cloud Foundry domain being used during the deployment in this environment
${routepath}: the Cloud Foundry route path being used during the deployment in this environment
any GitLab CI variable
any custom variable (ex: ${SECRET_TOKEN} that you have set in your project CI/CD variables)

While your scripts may simply use any of those variables, your app manifest can be variabilized too using Cloud Foundry variables replacement:

((environment_type)): the environment type (review, integration, staging or production)
((environment_name)): the application target name to use in this environment
((tmp_environment_name)): the application current name being used during the deployment in this environment (different from ((environment_name)) during a blue/green deployment for instance)
((hostname)): the current hostname being used during the deployment in this environment
((domain)): the Cloud Foundry domain being used during the deployment in this environment

You may optionally provide your own variable files (in your project) that will be used in addition to the above variables:

look for an env-specific cf-vars-$environment_type.yml (e.g. cf-vars-staging.yml for staging environment).
or default cf-vars.yml.

Your cf-vars-$environment_type.yml or cf-vars.yml files may contain variable patterns such as ${MY_SECRET}. If so, those patterns will be evaluated (replaced) with actual environment values. Beware that those values can be leaked by the cf push output. Multiline variables must be surrounded by double quotes (").

Routes management¶

Deployment jobs support several ways of managing mapped routes.

Here is the general recommendation (for each one of your environments):

if the application is mapped to one single route:
do not declare any route in the manifest,
set the xxx-host-name input or $CF_xxx_HOST_NAME variable to override the hostname to use (or leave unset to use default),
set the xxx-domain input or $CF_xxx_DOMAIN variable to define the domain to use (or leave unset if you wish to use the default CF domain),
set the xxx-route-path input or $CF_xxx_ROUTE_PATH variable to define the route path to use (or leave unset if you wish to use none).
if the application is mapped to several routes:
declare the routes in your manifest, possibly using ((environment_name)) and ((hostname)) variables,
set the $CF_xxx_HOST_NAME variable to override the hostname to use (or leave unset to use default),
xxx-domain input or $CF_xxx_DOMAIN variables won't be used,
xxx-route-path input or $CF_xxx_ROUTE_PATH variables won't be used.

Service instances management¶

Deployment jobs also provide a means to manage the lifecycle of service instances along with application deployment. This means that a service instance or the information to access an external service instance (aka "user provided service" in CloudFoundry), can be provisioned before the application is deployed (if it does not exist, it is created). Furthermore, in case of cleanup, they are deleted after the application is stopped/deleted itself. Concerning deletion, there is an exception: in case of production environment, automatic deletion is not supported (for safety reason).

For that purpose, the project has only to supply one json file per service instance that describes its characteristics. In order to identify such files, they should all have the same suffix: cf-service.json.

There are to cases to locate these files.

There is a sub-directory with the name of the type of environment to be managed by the pipeline (i.e. review, integration, staging, or production) present in the $CF_SCRIPTS_DIR directory. In that case, the files are looked up in this directory and only in this one.
There is no such sub-directory and in that case, the files are looked up directly in the $CF_SCRIPTS_DIR directory. This means that these files (and the service instances they represent) are probably shared by all types of environment.

Concerning the format of such a file describing a service instance it is quite straightforward. This is a json record with the following fields:

cfServiceName: this is the name of the service instance to be created
cfServiceOffering: this is the service offering to be used to create the service instance
cfServiceBroker: this is the name of the broker to be used in case there are several service offerings with the same name
cfServicePlan: this the service plan to be used within this service offering
cfServiceArgs: this is a json record that contains the parameters to be passed to the service offering to parameterize the creation

Let us take an example:

{
  "cfServiceName": "logarythm_drain_catalog",
  "cfServiceOffering": "logarythm_drain_prod",
  "cfServicePlan": "httpdrain",
  "cfServiceArgs": {
    "cloudid": "stg_ods_m_00.aed.lizard.o6o3knwikgjewx4il6rq",
    "component": "int",
    "env": "ep1"
  }
}

It describes a service instance of the logarythm_drain_prod service created with the httpdrain plan. It is named logarythm_drain_catalog. Finally, it provides the parameters to be used at creation time in the form of a json record.

The equivalent cf command to create this service instance is the following:

cf create-service logarythm_drain_prod httpdrain logarythm_drain_catalog -c '{"cloudid": "stg_ods_m_00.aed.lizard.o6o3knwikgjewx4il6rq","component":"int","env":"ep1"}'

Not only service instance from service of the Cloud Foundry marketplace can be managed. User provided services are also supported. For that purpose, the same file structure is used. To define a user provided service instance, the same file format is used while positioning both cfServiceOffering and cfServicePlan fields to the specific value CUPS ou cups. Then the arguments passed through the cfServiceArgs field are provided to the application when binding to this service instance.

Let us take an example:

{
  "cfServiceName": "my-ups",
  "cfServiceOffering": "CUPS",
  "cfServicePlan": "CUPS",
  "cfServiceArgs": {
    "user": "service-user",
    "password": "service-password",
    "url": "https://my-service-endpoint.domain.com"
  }
}

The equivalent cf command to create this user provided service instance is the following:

cf cups my-ups -p '{"user":"service-user","password":"service-password","url":"https://my-service-endpoint.domain.com"}'

Then at service binding, the application gets the three credential parameters as specified in the descriptor file, that are user, password and url.

Deployment output variables¶

Each deployment job produces output variables that are propagated to downstream jobs (using dotenv artifacts):

$environment_type: set to the type of environment (review, integration, staging or production),
$environment_name: the application name (see below),
$environment_url: set to $CI_ENVIRONMENT_URL.

Those variables may be freely used in downstream jobs (for instance to run acceptance tests against the latest deployed environment).

You may also add and propagate your own custom variables, by pushing them to the cloudfoundry.env file in your deployment scripts.

Configuration reference¶

Secrets management¶

Here are some advices about your secrets (variables marked with a ):

Manage them as project or group CI/CD variables:
masked to prevent them from being inadvertently displayed in your job logs,
protected if you want to secure some secrets you don't want everyone in the project to have access to (for instance production secrets).
In case a secret contains characters that prevent it from being masked, simply define its value as the Base64 encoded value prefixed with @b64@: it will then be possible to mask it and the template will automatically decode it prior to using it.
Don't forget to escape special characters (ex: $ -> $$).

Global configuration¶

The Cloud Foundry template uses some global configuration used throughout all jobs.

Input / Variable	Description	Default Value
`cli-image` / `CF_CLI_IMAGE`	The Docker image used to run CF CLI commands set the version required by your Cloud Foundry server	`registry.hub.docker.com/governmentpaas/cf-cli`
`manifest-basename` / `CF_MANIFEST_BASENAME`	CF manifest file basename (without extension nor env suffix)	`manifest`
`url`/ `CF_URL`	Default CF API url	has to be defined
`CF_USER`	Default CF user name	has to be defined
`CF_PASSWORD`	Default CF user password	has to be defined
`org`/ `CF_ORG`	Default CF organization for project	has to be defined
`base-app-name` / `CF_BASE_APP_NAME`	Base application name	`$CI_PROJECT_NAME` (see GitLab doc)
`default-domain` / `CF_DEFAULT_DOMAIN`	Default CF domain (only define if you want to use a different domain from CF default)	none
`default-route-path` / `CF_DEFAULT_ROUTE_PATH`	Default CF route path	none
`default-push-args` / `CF_DEFAULT_PUSH_ARGS`	Additional arguments for cf push command (only define if you want has a specific need not med by the template)	none
`scripts-dir` / `CF_SCRIPTS_DIR`	Directory where CF scripts (manifest, hook scripts) are located	`.` (root project dir)
`rolling-strategy` / `CF_ROLLING_STRATEGY`	Use Cloud Foundry native zero-downtime deployment strategy. See Zero-downtime deployment	`false`

Review environments configuration¶

Review environments are dynamic and ephemeral environments to deploy your ongoing developments (a.k.a. feature or topic branches).

They are disabled by default and can be enabled by setting the review-space input or CF_REVIEW_SPACE variable (see below).

Here are variables supported to configure review environments:

Input / Variable	Description	Default value
`review-space` / `CF_REVIEW_SPACE`	CF space for `review` env	none (disabled)
`review-url`/ `CF_REVIEW_URL`	CF API url for `review` env (only define if different from default)	`$CF_URL`
`CF_REVIEW_USER`	CF user name for `review` env (only define if different from default)	`$CF_USER`
`CF_REVIEW_PASSWORD`	CF user password for `review` env (only define if different from default)	`$CF_PASSWORD`
`review-org` / `CF_REVIEW_ORG`	CF organization for `review` env (only define if different from default)	`$CF_ORG`
`review-app-name` / `CF_REVIEW_APP_NAME`	Application name for `review` env	`"${CF_BASE_APP_NAME}-${CI_ENVIRONMENT_SLUG}"`
`review-domain` / `CF_REVIEW_DOMAIN`	CF domain for `review` env	`$CF_DEFAULT_DOMAIN`
`review-host-name` / `CF_REVIEW_HOST_NAME`	Application host name for `review` env	`"${CF_BASE_APP_NAME}-${CI_ENVIRONMENT_SLUG}"` (ex: `myproject-review-fix-bug-12`)
`review-zerodowntime` / `CF_REVIEW_ZERODOWNTIME`	Enables zero-downtime deployment on `review` env	`false`
`review-environment-scheme` / `CF_REVIEW_ENVIRONMENT_SCHEME`	The review environment protocol scheme	`https`
`review-environment-domain` / `CF_REVIEW_ENVIRONMENT_DOMAIN`	The review environment domain	none
`review-route-path` / `CF_REVIEW_ROUTE_PATH`	CF route path for `review` env	`$CF_DEFAULT_ROUTE_PATH`
`review-push-args` / `CF_REVIEW_PUSH_ARGS`	Additional arguments for push command	`$CF_DEFAULT_PUSH_ARGS`
`review-retired-app-suffix` / `CF_REVIEW_RETIRED_APP_SUFFIX`	If set, the app old version is not deleted/overriden but renamed with this suffix	none
`review-domain-tmp` / `CF_REVIEW_DOMAIN_TMP`	The review environment domain for the temporary blue-green app	`$CF_REVIEW_DOMAIN`

Integration environment configuration¶

The integration environment is the environment associated to your integration branch (develop by default).

It is disabled by default and can be enabled by setting the integ-spaceinput or CF_INTEG_SPACE variable (see below).

Here are variables supported to configure the integration environment:

Input / Variable	Description	Default value
`integ-space` / `CF_INTEG_SPACE`	CF space for `integration` env	none (disabled)
`integ-url` / `CF_INTEG_URL`	CF API url for `integration` env (only define if different from default)	`$CF_URL`
`CF_INTEG_USER`	CF user name for `integration` env (only define if different from default)	`$CF_USER`
`CF_INTEG_PASSWORD`	CF user password for `integration` env (only define if different from default)	`$CF_PASSWORD`
`integ-org` / `CF_INTEG_ORG`	CF organization for `integration` env (only define if different from default)	`$CF_ORG`
`integ-app-name` / `CF_INTEG_APP_NAME`	Application name for `integration` env	`"${CF_BASE_APP_NAME}-integration"`
`integ-domain` / `CF_INTEG_DOMAIN`	CF domain for `integration` env	`$CF_DEFAULT_DOMAIN`
`integ-route-path` / `CF_INTEG_ROUTE_PATH`	CF route path for `integration` env	`$CF_DEFAULT_ROUTE_PATH`
`integ-push-args` / `CF_INTEG_PUSH_ARGS`	Additional arguments for push command	`$CF_DEFAULT_PUSH_ARGS`
`integ-host-name` / `CF_INTEG_HOST_NAME`	Application host name for `integration` env	`"${CF_BASE_APP_NAME}-integration"`
`integ-zerodowntime` / `CF_INTEG_ZERODOWNTIME`	Enables zero-downtime deployment on `integration` env	`false`
`integ-environment-url` / `CF_INTEG_ENVIRONMENT_URL`	The integration environment url including scheme (ex: `https://my-application-integration.nonpublic.domain.com`). Do not use variable inside variable definition as it will result in a two level cascade variable and gitlab does not allow that.	none
`integ-retired-app-suffix` / `CF_INTEG_RETIRED_APP_SUFFIX`	If set, the app old version is not deleted/overriden but renamed with this suffix	none
`integ-domain-tmp` / `CF_INTEG_DOMAIN_TMP`	The integration environment domain for the temporary blue-green app	`$CF_INTEG_DOMAIN`

Staging environment configuration¶

The staging environment is an iso-prod environment meant for testing and validation purpose associated to your production branch (main or master by default).

It is disabled by default and can be enabled by setting the staging-spaceinput or CF_STAGING_SPACE variable (see below).

Here are variables supported to configure the staging environment:

Input / Variable	Description	Default value
`staging-space` / `CF_STAGING_SPACE`	CF space for `staging` env	none (disabled)
`staging-url` / `CF_STAGING_URL`	CF API url for `staging` env (only define if different from default)	`$CF_URL`
`CF_STAGING_USER`	CF user name for `staging` env (only define if different from default)	`$CF_USER`
`CF_STAGING_PASSWORD`	CF user password for `staging` env (only define if different from default)	`$CF_PASSWORD`
`staging-org` / `CF_STAGING_ORG`	CF organization for `staging` env (only define if different from default)	`$CF_ORG`
`staging-app-name` / `CF_STAGING_APP_NAME`	Application name for `staging` env	`"${CF_BASE_APP_NAME}-staging"`
`staging-domain` / `CF_STAGING_DOMAIN`	CF domain for `staging` env	`$CF_DEFAULT_DOMAIN`
`staging-route-path` / `CF_STAGING_ROUTE_PATH`	CF route path for `integration` env	`$CF_DEFAULT_ROUTE_PATH`
`staging-push-args` / `CF_STAGING_PUSH_ARGS`	Additional arguments for push command	`$CF_DEFAULT_PUSH_ARGS`
`staging-host-name` / `CF_STAGING_HOST_NAME`	Application host name for `staging` env	`"${CF_BASE_APP_NAME}-staging"`
`staging-zerodowntime` / `CF_STAGING_ZERODOWNTIME`	Enables zero-downtime deployment on `staging` env	`false`
`staging-environment-url` / `CF_STAGING_ENVIRONMENT_URL`	The staging environment url including scheme (ex: `https://my-application-staging.nonpublic.domain.com`). Do not use variable inside variable definition as it will result in a two level cascade variable and gitlab does not allow that.	none
`staging-retired-app-suffix` / `CF_STAGING_RETIRED_APP_SUFFIX`	If set, the app old version is not deleted/overriden but renamed with this suffix	none
`staging-domain-tmp` / `CF_STAGING_DOMAIN_TMP`	The staging environment domain for the temporary blue-green app	`$CF_STAGING_DOMAIN`

Production environment configuration¶

The production environment is the final deployment environment associated with your production branch (main or master by default).

It is disabled by default and can be enabled by setting the prod-spaceinput or CF_PROD_SPACE variable (see below).

Here are variables supported to configure the production environment:

Input / Variable	Description	Default value
`prod-space` / `CF_PROD_SPACE`	CF space for `production` env	none (disabled)
`prod-url` / `CF_PROD_URL`	CF API url for `production` env (only define if different from default)	`$CF_URL`
`CF_PROD_USER`	CF user name for `production` env (only define if different from default)	`$CF_USER`
`CF_PROD_PASSWORD`	CF user password for `production` env (only define if different from default)	`$CF_PASSWORD`
`prod-org` / `CF_PROD_ORG`	CF organization for `production` env (only define if different from default)	`$CF_ORG`
`prod-app-name` / `CF_PROD_APP_NAME`	Application name for `production` env	`$CF_BASE_APP_NAME`
`prod-domain` / `CF_PROD_DOMAIN`	CF domain for `production` env	`$CF_DEFAULT_DOMAIN`
`prod-route-path` / `CF_PROD_ROUTE_PATH`	CF domain for `production` env	`$CF_DEFAULT_DOMAIN`
`prod-push-args` / `CF_PROD_PUSH_ARGS`	Additional arguments for push command	`$CF_DEFAULT_PUSH_ARGS`
`prod-host-name` / `CF_PROD_HOST_NAME`	Application host name for `production` env	`$CF_DEFAULT_ROUTE_PATH`
`prod-deploy-strategy` / `CF_PROD_DEPLOY_STRATEGY`	Defines the deployment to production strategy. One of `manual` (i.e. one-click) or `auto`.	`manual`
`prod-zerodowntime` / `CF_PROD_ZERODOWNTIME`	Enables zero-downtime deployment on `production` env	`true`
`prod-environment-url` / `CF_PROD_ENVIRONMENT_URL`	The production environment url including scheme (ex: `https://my-application.public.domain.com`) Do not use variable inside variable definition as it will result in a two level cascade variable and gitlab does not allow that.	none
`prod-retired-app-suffix` / `CF_PROD_RETIRED_APP_SUFFIX`	If set, the app old version is not deleted/overriden but renamed with this suffix	none
`prod-domain-tmp` / `CF_PROD_DOMAIN_TMP`	The production environment domain for the temporary blue-green app	`$CF_PROD_DOMAIN`

`cf-cleanup-all-review` job¶

This job allows destroying all review environments at once (in order to save cloud resources).

It is disabled by default and can be controlled using the cleanup-all-review input or $CLEANUP_ALL_REVIEW variable:

automatically executed if $CLEANUP_ALL_REVIEW set to force,
manual job enabled from any master branch pipeline if $CLEANUP_ALL_REVIEW set to true (or any other value),

The first value force can be used in conjunction with a scheduled pipeline to cleanup cloud resources for instance everyday at 6pm or on friday evening.

The second one simply enables the (manual) cleanup job on the master branch pipeline.

Anyway destroyed review environments will be automatically re-created the next time a developer pushes a new commit on a feature branch.

in case of scheduling the cleanup, you'll probably have to create an almost empty branch without any other template (no need to build/test/analyse your code if your only goal is to cleanup environments).

Variants¶

Vault variant¶

This variant allows delegating your secrets management to a Vault server.

Configuration¶

In order to be able to communicate with the Vault server, the variant requires the additional configuration parameters:

INPUT / Variable	Description	Default value
`TBC_VAULT_IMAGE`	The Vault Secrets Provider image to use (can be overridden)	`registry.gitlab.com/to-be-continuous/tools/vault-secrets-provider:master`
`vault-base-url` / `VAULT_BASE_URL`	The Vault server base API url	none
`vault-oidc-aud` / `VAULT_OIDC_AUD`	The `aud` claim for the JWT	`$CI_SERVER_URL`
`VAULT_ROLE_ID`	The AppRole RoleID	must be defined
`VAULT_SECRET_ID`	The AppRole SecretID	must be defined

Usage¶

Then you may retrieve any of your secret(s) from Vault using the following syntax:

@url@http://vault-secrets-provider/api/secrets/{secret_path}?field={field}

With:

Name	Description
`secret_path` (path parameter)	this is your secret location in the Vault server
`field` (query parameter)	parameter to access a single basic field from the secret JSON payload

Example¶

include:
  # main template
  - project: "to-be-continuous/cloud-foundry"
    ref: "4.6.2"
    file: "/templates/gitlab-ci-cf.yml"
  # Vault variant
  - project: "to-be-continuous/cloud-foundry"
    ref: "4.6.2"
    file: "/templates/gitlab-ci-cf-vault.yml"

variables:
  # audience claim for JWT
  VAULT_OIDC_AUD: "https://vault.acme.host"
  # Secrets managed by Vault
  CF_USER: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/my-app/cf/noprod?field=user"
  CF_PASSWORD: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/my-app/cf/noprod?field=password"
  CF_PROD_USER: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/my-app/cf/prod?field=user"
  CF_PROD_PASSWORD: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/my-app/cf/prod?field=password"
  VAULT_BASE_URL: "https://vault.acme.host/v1"
  # $VAULT_ROLE_ID and $VAULT_SECRET_ID defined as a secret CI/CD variable

Examples¶

Ajax front-end¶

Context¶

Review environment enabled,
Continuous deployment to production,
internal route for review envs,
internet route for production.

`.gitlab-ci.yml`¶

include:
  - project: "to-be-continuous/cloud-foundry"
    ref: "4.6.2"
    file: "/templates/gitlab-ci-cf.yml"

# Global variables
variables:
  # specific project variables
  # TODO
  # Cloud Foundry CI template variables
  CF_URL: "https://api.cloud-foundry.acme.host"
  CF_ORG: "MyProject" # one single Organization for all spaces
  # CF_USER and CF_PASSWORD are defined as protected project CI/CD variables
  CF_REVIEW_SPACE: "Integration"
  CF_REVIEW_ENVIRONMENT_DOMAIN: "apps.cloud-foundry.acme.host" # intranet route
  # CF_STAGING_SPACE not defined: no staging environment; continuous deployment
  CF_PROD_SPACE: "Production"
  # CF_DEFAULT_DOMAIN not defined: use CF default domain by default
  CF_PROD_DOMAIN: "acme.com" # on prod: use my own internet domain
  CF_PROD_ENVIRONMENT_URL: "https://frontend.myproject.acme.com" # internet route
  CF_PROD_DEPLOY_STRATEGY: "auto"

build:
  stage: build
  script:
    - echo "build the front-end (TODO)"
  only:
    refs:
      - branches

manifest generation stuff¶

When deploying to review environment, the application name is generated (after the project and branch name). So the project should either not specify the application name in the manifest, or use a specific manifest-review.yml with variabilized ((environment_name)).

Spring Boot back-end¶

Context¶

uses a MySQL service,
Review environment enabled,
Continuous delivery: automatic deployment to staging env, automatic functional tests on staging, and manual deployment to production.

`.gitlab-ci.yml`¶

include:
  - project: "to-be-continuous/cloud-foundry"
    ref: "4.6.2"
    file: "/templates/gitlab-ci-cf.yml"

# Global variables
variables:
  # specific project variables
  # TODO
  # Cloud Foundry CI template variables
  CF_URL: "https://api.cloud-foundry.acme.host"
  CF_ORG: "MyProject" # one single Organization for all spaces
  # CF_USER and CF_PASSWORD are defined as protected project CI/CD variables
  CF_REVIEW_SPACE: "Integration" # enables review environments
  CF_REVIEW_ENVIRONMENT_SCHEME: "https"
  CF_REVIEW_ENVIRONMENT_DOMAIN: "apps.cloud-foundry.acme.host" # intranet route
  CF_STAGING_SPACE: "Preprod" # enables staging environment
  CF_STAGING_ENVIRONMENT_URL: "https://backend-staging.apps.cloud-foundry.acme.host" # intranet route
  CF_PROD_SPACE: "Production" # enables prpoduction environment
  # CF_xxx_DOMAIN not defined: use CF default domain for review and staging; routes are declared explicitly for production
  CF_PROD_ENVIRONMENT_URL: "https://backend.myproject.acme.com" # internet route

build:
  stage: build
  script:
    - mvn $MAVEN_CLI_OPTS clean package
  artifacts:
    paths:
      - target/*.jar
      - manifest.yml
  only:
    refs:
      - branches

manifest generation stuff¶

`manifest.yml`¶

---
applications:
  - path: target/myproject-backend-1.0.0-SNAPSHOT.jar
    memory: 768m
    instances: 1
    services:
      - ((environment_name))-db

This file will be used for review and staging environments, and uses variables for the MySQL service name (built from ((environment_name))).

The manifest doesn't need to specify the application name because it is explicitly set by the template deployment scripts.

It doesn't declare any route, therefore delegates the routes management to the template:

review apps will be mapped to default CF domain with "${$CI_PROJECT_NAME}-${CI_ENVIRONMENT_SLUG}" hostname
staging app will be mapped to default CF domain with "${$CI_PROJECT_NAME}-staging" hostname

`manifest-production.yml`¶

---
applications:
  - path: target/myproject-backend-1.0.0-SNAPSHOT.jar
    memory: 2G
    instances: 3
    routes:
      # internet route
      - route: backend.myproject.acme.com
      # intranet route
      - route: myproject-backend.apps.cloud-foundry.acme.host
    services:
      - myproject-backend-db

This file will be used for production env only.

The manifest doesn't use variables as it is only used for production env.

It also uses specific scalability settings (instances & memory).

hook scripts¶

`cf-pre-push.sh`¶

This script should ensure the required database is created, naming the database service after the application target name ($environment_name).

#!/bin/bash
dbname="${environment_name:-myproject-backend}-db"
echo "maybe create database '$dbname'..."
cf service $dbname || cf create-service c-mysql 100mb $dbname

`cf-readiness-check.sh`¶

This script - when found by the template - is used to wait & check for the application to be ready.

It uses the provided $hostname and $domain variables to build absolute urls to the application.

It is supposed to exit with status 0 on success (the template will go on with deployment), or any non-0 value in case of error (the template will stop and as much as possible revert the ongoing deployment).

#!/bin/bash
for attempt in {1..5}
do
  echo "Testing application readiness ($attempt/5)..."
  if curl --fail --silent --insecure --write-out "\n--> response status: %{http_code}\n" https://$hostname.$domain/actuator/health
  then
    echo "[INFO] healthcheck responsed: success"
    exit 0
  fi
  sleep 3
done

echo "[ERROR] max attempts reached: failed"
exit 1

`cf-post-cleanup.sh`¶

This script should cleanup the database, naming the database service after the application target name ($environment_name).

#!/bin/bash
dbname="${environment_name:-myproject-backend}-db"
echo "maybe delete database '$dbname'..."
cf delete-service $dbname -f

GitLab CI template for Cloud Foundry¶

Usage¶

Use as a CI/CD component¶

Use as a CI/CD template (legacy)¶

Understand¶

Managed deployment environments¶

Review environments¶

Integration environment¶

Production environments¶

Supported authentication methods¶

Deployment context variables¶

Generated environment name¶

Deployment method¶

Zero-downtime deployment¶

Cleanup method¶

Cleanup job limitations¶

Using variables¶

Routes management¶

Service instances management¶

Deployment output variables¶

Configuration reference¶

Secrets management¶

Global configuration¶

Review environments configuration¶

Integration environment configuration¶

Staging environment configuration¶

Production environment configuration¶

cf-cleanup-all-review job¶

Variants¶

Vault variant¶

Configuration¶

Usage¶

Example¶

Examples¶

Ajax front-end¶

Context¶

.gitlab-ci.yml¶

manifest generation stuff¶

Spring Boot back-end¶

Context¶

.gitlab-ci.yml¶

manifest generation stuff¶

manifest.yml¶

manifest-production.yml¶

hook scripts¶

cf-pre-push.sh¶

cf-readiness-check.sh¶

cf-post-cleanup.sh¶

`cf-cleanup-all-review` job¶

`.gitlab-ci.yml`¶

`.gitlab-ci.yml`¶

`manifest.yml`¶

`manifest-production.yml`¶

`cf-pre-push.sh`¶

`cf-readiness-check.sh`¶

`cf-post-cleanup.sh`¶