Integrating a Cloudsmith repository with a Drone CI pipeline

At Cloudsmith, we strive hard to ensure that private Cloudsmith repositories work with any build and release process that our customers use. Our mission is to be the universal package management solution that any modern DevOps workflow requires.  Thanks to the power of the Cloudsmith CLI and API, it’s very easy to integrate our platform with any of the popular build tools out there, and in this blog, we will take a look at how you can set up your Drone CI pipelines to push the results of your builds to your own private repository.

What is Drone CI

Drone CI is an open-source Continuous Integration platform. It has a container-first approach which means that you utilize Docker containers to execute the steps in your build pipelines.

Installing and setting up Drone itself is beyond the scope of this blog, but it’s easy – the components of a Drone build environment are provided as Docker images so you can get set up and running very quickly.

A Drone environment consists of two main pieces of infrastructure. The Drone Server and Drone Runners.

The Drone Server

The Drone server is the centralised place where you manage and observe your build pipelines and configure settings and secrets. It is installed via a lightweight Docker image that you can deploy on a cloud instance, a container service or even directly on your own hardware. It’s also available as a cloud-native solution that is free to use for open-source projects.

For this tutorial, I have installed the Drone server directly on an AWS EC2 instance.

Drone Runners

The Drone runners are standalone daemons that poll the drone server for pipelines to execute. In this tutorial, I will make use of the Docker runner that I have installed on my local machine, but you could equally install it on a cloud instance or container service also.  There are additional types of runners that are optimized for different use cases such as a stateful pipeline that needs to store files on the host machine between pipeline executions, but we will look at a simple build here that doesn’t require anything stateful.

Let’s get started!

Once you have set up the drone server, and one or more runners, you then connect the drone server to your source repositories using OAuth. I’ve already set this up and you can see the source repositories that the Drone server now has access to:

Let’s go ahead and activate it for my demo source repository:

You can see some of the main settings here, including the name of the drone configuration file from the source repo (.drone.yml)

The Drone configuration file is a YAML file that you place at the root of your source repository and it is used to define your pipeline steps. It’s actually a superset of the docker-compose file format, and it’s designed to be easily readable by anyone looking at your source.

The next thing we need to do is add a couple of secrets to the environment for this build pipeline.

1. The Cloudsmith API Key. We will need this as we need to authenticate push the package that we build to our Cloudsmith repository
2. Docker Credentials. We are going to use our own Docker image to execute this build, and this image is stored in a private Cloudsmith Docker registry.  We can get these credentials from our ~/.docker/.config.json file after we do a normal `docker login`

It is, of course, a great idea to store these secrets encrypted on the Drone server as we are doing here. You do NOT want to check secrets like these into your source as part of any configuration files.

Once we have everything activated as above, we can now build the project.

I have a simple “Hello World” example project in C, that I want to build into a Debian package. Let’s put together a drone configuration file for this project. It starts off by defining the kind as a pipeline, and the runner type as a docker runner, and we have just named it Cloudsmith Demo:

kind: pipeline

type: docker

name: CloudsmithDemo

Next, we just specify the OS and architecture and we then get into defining the steps that will execute when this pipeline runs:

platform:

os: linux

arch: amd64

After this, we start to declare our steps. We just need a single step here, a build and push package step, and we also declare the docker image that we will use to execute this step:

steps:

- name: Build and Push Package

image: docker.cloudsmith.io/demo/examples-repo/drone-base:latest

This is a docker image that I have prepared already, and just so you can see what is required for this build, the Dockerfile looks like this:

# Pull from base Ubuntu image

FROM ubuntu

# Do system updates and install dependencies

RUN apt-get update

RUN apt-get -y install git ruby ruby-dev rubygems build-essential python3-pip

RUN gem install --no-document fpm

RUN pip3 install cloudsmith-cli

RUN apt-get clean

It contains the tooling required to compile the C source, and then package it into a Debian package.  It also contains the Cloudsmith CLI, which we will use to push the package to our Cloudsmith repository.

You could install all the tooling using additional commands in the step, and forgo the use of your own Docker image. But it’s cleaner, and also makes better use of drone’s “container first” approach to build your own image with any pre-requisites for the pipeline, and use that to execute the step.

Next, we set up our Cloudsmith API Key as an environment variable, from our drone server:

environment:

CLOUDSMITH_API_KEY:

from_secret: CLOUDSMITH_API_KEY

We then execute the ‘make’ command to compile our C source, use ‘fpm’ to package this into a Debian package, and then use the ‘cloudsmith push deb’ command to push the built package to our Cloudsmith repo:

commands:

- make

- fpm -f -s dir -t deb -v 1.0.1 -n cloudsmith-drone-demo .

- cloudsmith push deb demo/examples-repo/debian/buster cloudsmith-drone-demo_1.0.1_amd64.deb

Finally, we also pull in the secret we added to our Drone server that will allow Docker to pull our Docker image with our prerequisites from our private Cloudsmith Docker registry:

image_pull_secrets:

- dockerconfigjson

So, putting all this together, our complete .drone.yml config files looks like this:

---

kind: pipeline

type: docker

name: CloudsmithDemo

platform:

os: linux

arch: amd64

steps:

- name: Build and Push Package

image: docker.cloudsmith.io/demo/examples-repo/drone-base:latest

environment:

CLOUDSMITH_API_KEY:

from_secret: CLOUDSMITH_API_KEY

commands:

- make

- fpm -f -s dir -t deb -v 1.0.1 -n cloudsmith-drone-demo .

- cloudsmith push deb demo/examples-repo/debian/buster cloudsmith-drone-demo_1.0.1_amd64.deb

image_pull_secrets:

- dockerconfigjson

Now, all that we need to do is to make a change to our source and push it to our source repository to start this build.

If we check our Drone server after pushing a change to our source, we can see that our pipeline has started to execute:

Let’s have a look and see what happened.  It started by cloning our source repo,

And then our “build and push package” step executed:

You can see that it has pulled the docker image we are using to execute this step, and then executed ‘make’ and ‘fpm’ to build our Debian package. Finally, it pushed the resulting Debian package to our private Cloudsmith repository.  That’s the pipeline execution finished successfully.

If we now go to our Cloudsmith repository, we can see our new cloudsmith-drone-demo Debian package – the result of our automated build pipeline, present as expected in our own private repository:

Summary

It’s easy to automate pushing the results of your Drone pipeline builds to a private Cloudsmith repository. With the simple addition of the Cloudsmith CLI (or a Docker container that has the Cloudsmith CLI), you can be up and running in just a few minutes, and benefit from the security, control and centralised management that a universal package management solution like Cloudsmith provides.

Liked this article? Don\'t be selfish (:-), share with others:   Tweet