The Circle Is Now Complete

Can you believe it's summer-time already? It's hard for us to tell from the cave that we're coding in day and night, but the office thermostat indicates that it is probably sunny way up there, above the ground. It does mark a fantastic occasion to bring you news of some juicy pieces of new functionality to help get you automated, now with even more secure delivery of your software.

We're extremely pleased to announce one of our features that we believe brings us full circle in terms of achieving data portability in and out of the service: Webhooks. This isn't a fancy term for a new Spider-Man gadget. Where the RESTful API (and CLI) allow you to easily get data into the system, the webhooks allows us to push data about events to you. Now you can do clever things like react to a package being promoted to a production repository, in order to drive your pipelines to start a deployment.

So yes, the circle is now complete for data portability. Which is great, of course, but we know what you're thinking: Hurry up and get NPM / Docker / NuGet / Alpine / Bower completed (delete as appropriate), and you'd be right. We're working hard on it indeed, in between working on raising funds to speed things up, and we'll hopefully have even more exciting announcements next time.

But enough about that for now: To the product updates!

Webhooks

TL;DR: You can now have Cloudsmith inform you, your cats and your servers about events that happen, such as new packages being synchronised. Drive your pipelines or ChatOps with it!

Where: The "Webhooks" sub-menu of each repository.

Webhooks - What are they good for? Either for driving automation by pushing data to your pipelines, or by integrating with your chat tools to provide slick ChatOps. Our webhooks support events that occur in a repository, such as when packages have been uploaded, are synchronising, have synchronised or have failed. We plan to add more events in the future, and to have them creatable at the the namespace (user/org) level, or site-wide on Enterprise edition.

For pipeline automation, you might utilise webhooks by setting up one to tell a CI/CD tool such as Jenkins, or perhaps Spinnaker (if you're being fancy) that it is time to deploy when a synchronised package is moved to your production repository. In this way, you can control the flow from development to production by limiting who (or what) has the authority to move packages to the production repository, and thus, to deploy publicly.

For ChatOps, you can utilise webhooks to send a message to a chat tool such as Slack when each type of event occurs. You'll format this in such a way so as to present critical information to your team, such as what the package is, where it is located, who uploaded it, and how to access it, etc. If you're really fancy, you'll have a Slack integration that lets users interact with Cloudsmith via slash commands, for super slick bi-directional DevOps goodness.

Creating a webhook is simple and only requires an endpoint to send it to. If you're looking to create a Slack integration, we highly recommend using the Handlebars Template payload format. Handlebars is a mini templating language that you'll use to craft the output of the webhook. More help is available on-site, but if you're stuck, just yell. In the near-future we'll offer more concrete documentation, and beyond that we'll add more tightly integrated plugins for things like Slack that do the hard work for you.

Composer Package Repository

TL;DR: We've added support for Composer packages (in the form of .phar, .tgz, .zip, etc.), so that you can have your PHP packaging with all the usual Cloudsmith goodness.

Where: The "Upload Packages" sub-menu of each repository.

As defined by Wikipedia: "Composer is an application-level package manager for the PHP programming language that provides a standard format for managing dependencies of PHP software and required libraries". Cloudsmith now provides PHP as a first-class package repository type with support for most of the metadata that a typical composer.json file exports. Get started by uploading a Composer-compatible .phar package built with phar-composer or similar.

If you're using the UI to upload packages then Composer support is immediately available (as per image above). If you're uploading via the CLI you'll need to download and install the latest (0.6.1) version of the cloudsmith-cli application. Assuming you've followed the usual instructions for installation, you can do this with a simple invocation of pip to update your CLI:

pip install --upgrade cloudsmith-cli

Social Authentication

TL;DR: You can now signup and login using third party social authentication credentials from providers such as Amazon, GitHub, GitLab, Google and Microsoft.

Link: https://cloudsmith.io/user/login/

This one is probably the most obvious because it's hard to miss, but we've added the capability to login via social providers (as you can see below) such as Amazon, GitHub, GitLab, Google and Microsoft. In the future we'll add additional support for more complicated and Enterprise-level authentication integrations, such as being able to use a third party AD service for orgs. For now though, please enjoy the convenience of jumping services.

Since you've already got an account, you'll need to login first and then manage your social accounts in your user settings. You will then be able to connect each of the supported social providers to your account so that you can login with them next time. In the future we'll offer slicker association from login by asking you to re-enter your password if you try to login with a social account but you already have a (previously unconnected) Cloudsmith account.

Two-Factor Authentication

TL;DR: You can now use two-factor authentication to protect your logins, your Organisations, and your packages with an additional level of security.

Link: https://cloudsmith.io/user/settings/security/

Two times the security. Twice as secure. Right? Well, that only matters if the base level of security is strong to begin with. At Cloudsmith security is one of our most paramount concerns, and we utilise our collective years across different disciplines such as financial technology and Internet startups to apply this to package management. You can see this in the architectural DNA of the service, such as how we process packages away from the front-end, through to utilisation of front-end security techniques, such as the use of Content Security Policy (CSP), HTTP Strict Transport Security (HSTS), etc.

We now have support for two-factor authentication via a TOTP (Time-based One-time Password Algorithm) device, such as Google Authenticator, LastPass Authenticator, etc. Once you've completed enrolment (i.e. registration of your device with us), you will be challenged to authenticate via the device after social or password-based login. You do this by entering in a 6-digit pin that your device presents. If you forget your 6-digit pin, we also offer a recovery service using disposable tokens.

If you're the owner of an Organisation (as defined by Cloudsmith) you can also force enforce enrolment of two-factor for everyone in the Org. A flag that denotes 2fa within the members list will tell you if the member has two-factor enabled or not. If you enforce enrolment and a User hasn't yet enroled, they will not be able to access any of the pages for the Organisation (e.g. they can't view or manipulate packages). If you are security conscious, please consider enabling this.

Emails Management

TL;DR: You can now add and manage multiple email addresses per User account, in addition to managing your subscriptions to the kind of emails we love to send you to keep you in the loop.

Link: https://cloudsmith.io/user/settings/emails/

It's often the case that one ends up juggling multiple email accounts and using them at the same time to access different services. This is often the case if you belong to multiple organisations and each provides you with an email address that is specific to it. To help manage and facilitate this, you can now add multiple email addresses to your Cloudsmith User Account. Once an email is verified you'll be able to login with it. You can also specify your primary email address, which is the one that we'll email you with when contacting you.

In addition to that, we're proud to say that we're fully compliant with the latest privacy laws in the EU (the GDPR, which you've likely heard about more often than you'd like to). As such, you can now manage your email communication preferences and choose whether you want to opt-in or opt-out for emails. You're reading the newsletter right now, so if you opt-out of that we'll not send you it anymore; but then you'll not know about new features like this. :-)

Vendor-Friendly Entitlements

TL;DR: You can now limit downloads to specific packages or download paths, by maximum number of downloads or clients, or by start or expiry dates. Oh, and support for user-based metadata.

Where: The "Entitlements" sub-menu of private repositories.

We have built Cloudsmith around servicing a trifecta of business use-cases, sometimes called the "3 'D's of Package Management": Dependencies for development, Deployment for operations, and Distribution for Vendors. The third use-case, distribution for Vendors, is all about empowering Users to securely distribute their software to the world at high-speed. It doesn't matter whether the software is free, open-source, or proprietary and paid.

If you're selling software and distributing it via Cloudsmith, you'll likely have a license that is associated per Customer and which dictates their terms of usage. Associating the license with an entitlement allows you to control and track downloads of the software specifically for that license. For example, you could only allow the Customer to download specific packages, between August 1st 2018 and August 1st 2019, up to a maximum of 10 downloads from one location (i.e. a single client).

Each entitlement can have different restrictions, and you can use the freeform (JSON-based) metadata to add your own information into the entitlement. Support is already available via the UI and via the API, but we'll be adding full support for the enhanced functionality to the next release of the CLI.

Full Boolean Search

TL;DR: You can now search for packages using full boolean queries (with AND/OR/NOT support), along with additional attributes such as distribution, format, status, etc.

Where: The search bar on top of the packages list in repositories.

You may or may not recall that in a previous newsletter we said that we'd be gradually rolling out more FOPS support across the site - In case you've forgotten or have never heard of it before, FOPS is our endearing term for Filtering, Ordering, Pagination and Search support across the website. This is something that we'd like to make universal across the UI, the API and the CLI, and we're gradually rolling out support where we can. We're still not pleased with our current FOPS level, so we're continuing to add this in-between other feature releases.

This month's flavour is the addition of Full Boolean Search to the packages search functionality. Previously we only supported an implicit AND query (i.e. all terms must be present). Now you can take advantage of the full boolean query parser to build complex queries with any combination of AND, OR and NOT. You can use parentheses to group terms if required. As part of this functionality, we have also added support for other search terms, such as distributions (e.g. 'el' for enterprise linux), format (e.g. 'deb' for debian packages), status (e.g. 'in_progress' for packages in progress). Help is available by hovering on the question mark.

Support is immediately available via the UI, API and the CLI. You might be excited to know that in the near-future we'll be rewriting the CLI actions that target packages to use the search functionality instead of having to use things like auto-generated slugs (which we appreciate is painful and hard to predict).

Other Minor Changes

In addition to the above, the following changes have made it in:

• You can now set email communication preferences.
• Each repository page now has additional intro/help text.
• The Entitlements list now reflects additional info for restrictions.
• Downloads and clients are now tracked per entitlement token.
• Access logs related to packages now link back to package.
• Many fixes and enhancements across the user interface.

Hit Me With The Full Changelog

You can see the full log of changes for the latest releases at:

v0.30: https://help.cloudsmith.io/blog/v030x
v0.29: https://help.cloudsmith.io/blog/v029x
v0.28: https://help.cloudsmith.io/blog/v028x
v0.27: https://help.cloudsmith.io/blog/v027x

What's Coming Up Next/Soon?

We're full steam ahead on improving the service, and we're likely to have a bumper pack month or two coming up. Expect big changes as we on-board more developers.

With that said, here are some little slices of awesome from the roadmap (no particular order):

• NPM Packages: A fully-fledged NPM repository (soon).
• Better Documentation: A manual on how to use this beast.
• Upstream Proxying: Utilising us as a package proxy cache.
• Package Redeployment: Overwrite (redeploy) your packages.
• CLI Upgrades: Adding new entitlements/webhooks support.
• Landing Redesign: Better information on what/why we do this.
• Multiple Actions: UI for affecting multiple packages at once.
• Docker Packages: A fully-fledged Docker repository.

If you feel particularly strongly about any of those, or if you've got suggestions of your own, don't forget to vote and submit ideas for the roadmap. You'll need a Trello account to add your thumbs up, but it's free and easy to signup. No excuses!

Ran Out Of Trial?

If you've run out of your trial period, but want to try some of the latest premium features, no problem. Reply to let us know and we'd be delighted to offer you an extension. In fact, we offer this to any user who has missed their chance to trial the service properly, for whatever reason. Just ask!

Company Update

You want to see us succeed, right? Of course you do! To do that we have a humble favour to ask - As a startup who is currently in the throes of an investment round, we can do with all the help that we can get to make things more awesome. The more traction that we can build, the stronger and better the service that we'll be able to build for you. If you can help us by promoting us through word-of-mouth, by writing an article or featuring us, or simply by providing a qualified link back to us, we'd be eternally grateful.

As you might have experienced, we do our best to support all of our users in any way that we can. If that means helping you write some code to automate or integrate, we're happy to do so. Some of these things would be hard to scale, but transparency and open/honest friendliness is something that we build into our company ideals, and is something we'd like to continue doing as we grow larger. Getting you automated in the easiest and most secure possible way is the reason that we built Cloudsmith.

Let's do this together - If you help, please let us know. :-)

That's A Wrap!

Other than that, as always we hope that you enjoy the latest functionality, and if you've got any questions about any of the above, or need help from integration to use-case guidance, please don't hesitate to reply or contact us on the usual channels.

So what are you waiting for? Get started at Cloudsmith now.

Remember: Be Awesome & Automate Everything.

Liked this article? Don\'t be selfish (:-), share with others:   Tweet