CLCI Gitlab CI¶
Gitlab CI is the default CI solution for projects hosted on Gitlab. While in-depth knowledge of how Gitlab CI works is not needed to get very basic pipelines set up using this project, it can be useful when attempting to build more complicated pipelines. See https://docs.gitlab.com/ee/ci/ for more documentation. The repository for this project is located at https://gitlab.common-lisp.net/clci/gitlab-ci.
Gitlab CI ultimately requires one or more YAML files that describe the jobs to perform. This project contains the following files:
definitions.gitlab-ci.yml: Basic definitions that are used to build up more complicated jobs and pipelines.
linux-test-pipeline.gitlab-ci.yml: A pipeline that runs tests on Linux systems.
guarded-linux-test-pipeline.gitlab-ci.yml: A pipeline that includes
linux-test-pipeline.gitlab-ci.ymlif allowed by the value of
clpm-dep-update-pipeline.gitlab-ci.yml: A pipeline that updates dependencies managed by CLPM.
guarded-clpm-dep-update-pipeline.gitlab-ci.yml: A pipeline that includes
clpm-dep-update-pipeline.gitlab-ci.ymlif allowed by the value of
release-pipeline.gitlab-ci.yml: A pipeline that cuts releases when tags are pushed.
guarded-release-pipeline.gitlab-ci.yml: A pipeline that includes
release-pipeline.gitlab-ci.ymlif allowed by the value of
$PIPELINE_TYPEand the commit is a tag that looks like a release.
Currently, this project only works on Linux runners (and may Mac runners if you configure them correctly). However, first-class Windows and MacOS support is on the roadmap. This support is likely to arrive because this project is an offshoot of another (private) project that does have Windows and MacOS support. We mostly just need to get some runners for testing.
This section describes how to set up Gitlab CI for a CL-based project using these helpers. Gitlab CI has a dizzying amount of flexibility, so this quickstart will quickly get you something that works and will be sufficient for many use cases. But you may want to dive deeper into this project or Gitlab CI at some point to perform more advanced tasks.
This section describes setup that you must perform other than modifying your
In order to use the test pipeline, you must provide a method to run your tests (this project does not attempt to guess how they should be run). You must provide either a Makefile target to run your tests or write a script for the CI helpers to run.
Create a file called
scripts/ci-test.lisp (alternatively, set
$CLCI_TEST_SCRIPT to point to another file). This file will be loaded (with
cl:load) into a CL process that has ASDF available (and your chosen
dependency manager, if applicable). This file must either enter the
debugger (by signaling an uncaught error) or exit with a non-zero exit
(uiop:quit 1)) if your tests fail.
For most projects, the contents of this file should be as simple as:
NOTE: If you have a
Makefile in your repo, the helpers will instead run
make test (after setting some environment variables). If you wish to disable
this behavior, set
In order to use the release pipeline, you should create a
in your repo. The first level heading (line starting with a single
be your project name. The second level headings (starting with
start with the version number and the contents should describe the changes to
your project. See the
CHANGELOG.md file in this repo for an example.
Additionally, you must protect your release tags. Go to "Settings > Repository > Protected Tags" and protect
Then, when you're ready to make a release, push a tag of the form
Update CLPM bundle
If you are using CLPM for your dependency management, you can use these helpers to periodically update your lock file for you.
To do this, you should create a bot for your project that can open merge requests. Go to "Settings > Access Tokens" and create a token with
api permissions. Then create a CI variable ("Settings > CI/CD > Variables") called
CLCI_MR_TOKEN, whose value is the token you got from the first step.
Last, you should create a scheduled job to check for updates to your
dependencies. Go to "CI/CD > Schedules". Create a new scheduled job for your
default branch that sets
Copy the following to
.gitlab-ci.yml in your repo. Be sure to read the
comments, as they give some hints on customizations that may be appropriate for
include: project: 'clci/gitlab-ci' ref: v2-stable file: - guarded-linux-test-pipeline.gitlab-ci.yml - guarded-release-pipeline.gitlab-ci.yml # Uncomment if you are using CLPM and want to automatically update your # bundle. # # - guarded-clpm-dep-update-pipeline.gitlab-ci.yml variables: # Uncomment if you want to use Quicklisp as your dependency manager during # tests. # # CLCI_DEPENDENCY_MANAGER: "quicklisp" # # Uncomment if you have Git submodules that you want the runner to # automatically init and update for you. Submodules are sometimes used by # projects to bundle their test dependencies. # # GIT_SUBMODULE_STRATEGY: recursive # # Uncomment these lines if you want to test against Allegro, you have read # the Allegro express license # <https://franz.com/ftp/pub/legal/ACL-Express-20170301.pdf>, *and* your use # of Allegro Express does not violate the license. Alternatively, uncomment # and provide your own Docker image (or runner) that has Allegro installed # with your actual license. # # CLCI_TEST_ALLEGRO: "yes" # I_AGREE_TO_ALLEGRO_EXPRESS_LICENSE: "yes" # This section is not strictly required, but prevents Gitlab CI from launching # multiple redundent pipelines when a Merge Request is opened. workflow: rules: - if: '$CI_PIPELINE_SOURCE == "merge_request_event"' - if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS' when: never - if: '$CI_COMMIT_BRANCH' - if: '$CI_COMMIT_TAG'
This project is still very new, so I can't really say that these questions are frequently asked, but I imagine they would be :).
How do I install a non-CL dependency before running tests?
First, make sure your dependency isn't already installed. The default Linux Docker images are based on the buildpack-deps image, which includes a lot of commonly used libraries.
If you really need to install something before your tests run, add the
following to your
.common:test:clci: before_script: - apt-get update - apt-get install -y foo bar baz
How do I use CLPM as my dependency manager?
Just commit your
clpmfile.lock (in the root of your repo) without setting
$CLCI_DEPENDENCY_MANAGER and the helpers will take care of the rest.
How do I test using dependencies from Ultralisp?
quicklisp. Then set
How do I test against both Quicklisp and Ultralisp?
Add the following to your
.common:test:clci: parallel: matrix: - CLCI_QL_DIST_NAME: [quicklisp, ultralisp]
How do I test against both the ASDF bundled with the implementation and the latest released ASDF?
Add the following to your
.common:test:clci: parallel: matrix: - CLCI_ASDF_VERSION: [REQUIRE, latest]
How do I generate documentation (or any other CL-based task)?
Create a CL file in your repo that does what you want and enters the debugger
or exits with a non-zero status if it fails. Then add a job in your
generate docs: extends: - .clci sbcl - .clci DEPENDENCY_MANAGER script variables: CLCI_SCRIPT: path/to/script.lisp artifacts: paths: - doc/
DEPENDENCY_MANAGER is replaced with
asdf (no dependency manager),
quicklisp. In the future, a more flexible job may be added that
chooses the correct dependency manager based on the value of
$CLCI_DEPENDENCY_MANAGER, but that will require a non-trivial amount of
refactoring to support.
Your script will then be loaded into a CL process that has ASDF and your dependency manager (if applicable) available.
Detailed Documentation ¶
More detailed documentation for these helpers can be found at: