Orso Labs

Building Go code, with and without Go modules, with Concourse


The example code is available in directory build-golang of repository concourse-pipelines, separated in 2 branches:

The directory structure:

├── README.adoc
├── ci
│   ├── build-golang-pipeline.yml
│   ├── build-task.yml
│   ├── build.sh
│   ├── unit-task.yml
│   └── unit.sh
├── cmd
│   └── cake         <1>
│       └── main.go
├── go.mod           <2>
├── go.sum           <2>
├── hello            <3>
    ├── hello.go
    └── hello_test.go

Without Go modules

We begin with a minimal pipeline, nothing special (the pipeline in the sample repository has also the build job shown in the image above):

- name: concourse-pipelines
  type: git
    uri: https://github.com/marco-m/concourse-pipelines.git
    branch: golang-pre-modules
    paths: [build-golang/*]     <1>

- name: unit
  - get: concourse-pipelines
    trigger: true
  - task: unit
    file: concourse-pipelines/build-golang/ci/unit-task.yml

In the task configuration we use the optional task input path to create a directory structure compliant with GOPATH:

platform: linux

  type: registry-image
  source: {repository: golang}

- name: concourse-pipelines
  path: gopath/src/github.com/marco-m/concourse-pipelines <1>

  path: gopath/src/github.com/marco-m/concourse-pipelines/build-golang/ci/unit.sh <2>

Note: Concourse enforces the input path to be relative, this is why we cannot specify a path like /go/src/.... On the other hand, this is not a problem, as we will see in a moment.

Finally the run script:

export GOPATH=$PWD/gopath           <1>
export PATH=$PWD/gopath/bin:$PATH   <2>

cd gopath/src/github.com/marco-m/concourse-pipelines/build-golang <3>

echo "Fetching dependencies..."
go get -v -t ./...        <4>

echo "Running tests..."
go test -v ./...          <5>

Optimization: task caching

As-is, the run script keeps downloading over and over the same dependencies.

We can use the task cache feature to cache the dependencies and so speed-up the build.

In the task configuration we add:

- path: depspath/
- path: gopath/pkg/

and in the run script we prepend the cache directories to the environment variables:

export GOPATH=$PWD/depspath:$PWD/gopath
export PATH=$PWD/depspath/bin:$PWD/gopath/bin:$PATH

(check the full source in the sample repo in case of doubt)

I took the trick about the 2-component GOPATH and depspath from the booklit project, which is a relatively simple Golang project built with Concourse, that I use as my reference. In case you are confused as I was the first time I saw it, depspath is not a special name, any name would do. It is simply the fact that

  1. It is the first component of GOPATH
  2. It is the cache

that makes it work :-)

Note also that with Go modules this 2-component path is not needed.

With Go modules

NOTE: This is not a tutorial about Go modules. Please refer to the official documentation at Go Modules.

Everything becomes simpler.

We create the Go module go.mod if needed (don’t forget to commit it to git, together with go.sum, that will be created automatically on first test/build):

$ cd build-golang
$ go mod init github.com/marco-m/concourse-pipelines/build-golang

The task file becomes a classic Concourse task file:

platform: linux

  type: registry-image
  source: {repository: golang}

inputs:  <1>
- name: concourse-pipelines

caches:  <2>
- path: gopath/

  path: concourse-pipelines/build-golang/ci/unit.sh

Also the run script becomes simpler:


set -e -u -x

export GOPATH=$PWD/gopath           <1>
export PATH=$PWD/gopath/bin:$PATH

cd concourse-pipelines/build-golang <2>

echo "Running tests..."             <3>
go test -v ./...

That’s it, happy building!


#concourse #cicd #golang #go-modules