To NuGet and Beyond Webinar [On-demand Session]

Watch our To NuGet & Beyond webinar with Adil Leghari, Dan McKinney, Paddy Carey and Mikey Lombardi.

We break down the NuGet ecosystem in Cloudsmith's first ever hosted webinar! Read the video transcript below, and make sure to read our blog article: What is NuGet.

Video Transcription

Introductions

Adil Leghari:

Good afternoon, good evening, depending on where you are in the world. We appreciate you taking the time out of your day to join us here and welcome to our webinar entitled To NuGet and Beyond. Now we here Cloudsmith know that as a package format, NuGet is a core technology for the Microsoft developer ecosystem, and we want to ensure that we're fully meeting the needs of our customers when it comes to NuGet packaging.

And we also want to make sure that we're treating our Windows users like First-Class citizens on our platform. So to that end, today we'll be talking about some exciting new improvements we've made to our NuGet repository platform, here at Cloudsmith, but also our new support for NuGet Upstreams. Sound good?

OK, so before we get ahead of ourselves, some introductions are in order. Firstly, who the heck are we? So, Dan, I'ma let you go first.

Dan McKinney: Yeah, thanks very much, Adil. So yes, hi everybody. I'm Dan McKinney, I work in developer relations here at Cloudsmith. So it's my role to sort of educate and help and assist in any way that I can. So hoping to get some good questions today and to show you some of the sort of unique features of Cloudsmith and how we've brought those features to the NuGet community and the NuGet ecosystem in general.

Adil Leghari:

Awesome, awesome. So I'm Adil Leghari, I'm a solutions architect, manager here at Cloudsmith. I'm assistant turned solution architect and turned manager, and I've spent the better part of two decades here wrangling windows boxes.

So over the past five or so years, I've gotten more active in the PowerShell & DevOps community. I'm a speaker and author and a blah blah blah blah blah, you know that sort of thing. But I think more important for today, I've also spent the past three years in the packaging management space, here Cloudsmith and also previously a Chocolaty; hello ChocoFam. So I guess you could say I know a thing or two about new packaging. Both of us are very pleased to have joining us today as our surprise guest speaker, my dear friend and colleague in the PowerShell Dev Community, Michael Lombardi, Mikey, over to you.

Mikey Lombardi:

Hey, how's it going? So I have been in the PowerShell Windows Dev Ops space for around roughly a decade and spent a lot of time doing talks and trying to get people to write docs and script code a little bit more cleanly, elegantly.

And then in the last couple of years of my entire professional life, doing nothing but open source development and documentation, writing previously at Puppet, currently at Microsoft, a huge amount of fun to maintain Chocolaty packages and PowerShell modules. So packaging has stayed near and dear to me because it's the best way to get what I wrote or what other people wrote to any system. Everything else is just so much pain.

**Adil Leghari:**Yeah, no, for sure. By now, I think also you're doing some game dev stuff, right? Which you'll be looking at in the future for packaging?

Mikey Lombardi:

Yeah, doing some interesting work in Golang to build some CLI apps and frameworks and tools. And I'm very excited actually to use Cloudsmith to deliver those things in a much less frustratingly, complex way. I don't want to figure out packaging from the ground up if I can help it. And the raw feeds seem like they're going to solve exactly my problem for me. So I'm very excited about that.

Adil Leghari:

Awesome. We're excited to have you, Mikey. I just want to put a caveat upfront that your views and opinions do not reflect those of your employer.

But whereas Dan and I we are, we are speaking for our employers.

Now, for those of you who may not be familiar with Cloudsmith and what we do here, Dan, why don't you walk us through a quick overview?

Overview of Cloudsmith

Dan McKinney:

Yes, absolutely. Well, I would hope that we do have some Cloudsmith users, but for those that aren't yet Cloudsmith users, Cloudsmith is a global cloud native and universal package management as a service package management solution. And so what we really provide is the ability to sort of build a single source of truth for all your software, artifacts, images, packages.

And although this is a very NuGet-focused webinar, you know, we are fully universal. We support over 28 package formats in multi-format repositories as well, so we are a completely global distributed, web application. There's nothing to install. You don't have to patch anything or update anything. It's a managed platform. We have over 225 points of presence on our package delivery network, as we call it, so great for distributed teams or software distribution even.

Adil Leghari:

An so just to clarify, if I were just a random user, a free user even, if I upload any package up into Cloudsmith, it is available in 225 different points of presence.

Dan McKinney:

Yeah, exactly. You know, it's cached at the edge of the package delivery network. So, you know, performant global access. You don't have to worry about replication or clustering or high availability or anything like that. That's the whole managed platform for you.

Well, of course, we are heavy on security, lots of really, you know, granular access controls and permissions and the ability to restrict and allow access and a very fine-grained manner. And for private repositories, of course, you can have public and open source repositories as well. And yes, as I said, universal. So, you know, we love all package formats equally, but with a special focus on NuGet today.

So actually, Adil, I think this is a very good point now that I've just explained, yes?

For our first poll, so really, we'd like to know more about our users or people who are potentially considering using Cloudsmith. What is your need? What problem are you trying to solve in package management? What is your use case? So we have some options in our poll. You know, A to distribute software externally or B to manage growing complexity within an organization which is very popular now.

Actually, we see a lot of this or a very hot topic at the moment, software supply chain security to supply security or software supply chain. Or maybe you just want to centralize package management on one sort of platform. And for those of us, for those of you who have just joined the webinar and missed the previous slide, what is Cloudsmith? That's a perfectly legitimate answer in this case. So yes, and we'll leave that up just for a second and people can answer and we'll look at them at the end.

Adil Leghari:

SSounds good, Dan, and I do want a plug because we have this here, and I think Alan, our CEO, I'm sure is watching, did remind us yesterday that we have all these great resources and we sometimes don't even plug them enough internally. So let's do that now. So I will mention that securing the software supply chain is one of those big SBOM buzzwords that are out there now. We did try and demystify this a little bit. We had a webinar last week with the Linux Foundation and Dan Lorenc over at Chainguard, who's of Sigstore and Cosign fame.

So feel free to go to our website and check that out as well. It's available under resources.

But yeah, generally, we talk a lot about software supply chain security, which is a very hot topic of conversation, right, Mikey?

Mikey Lombardi:

Yeah, just a little bit of a nightmare to catch up to for everybody who pretended that it didn't matter forever.

Agenda Overview

Adil Leghari:

So that's kind of like we talked to not talk specifically about everything you wanted to know about the supply chain, but we're afraid that so I'm plugging another webinar during this webinar. But hey, it's all good content, right? So just frames of reference for you for later.

OK, so let's talk you through a little bit of the agenda now that you know who we are and what Cloudsmith is just you know what you're getting yourself into today. So specifically, what I'll talk about, first of all, is what is NuGet, as a packaging format, as a technology, and how it enables folks on the .Net dev ecosystem.

And then we'll talk about specific NuGet flavors, so the communities and tooling around different use cases for the technology. So in NuGet, that's mainly three groups the C#/.NET folks using Visual Studio, the PowerShell dev community and of course, the Chocolaty dev community as well. So we'll highlight use cases, and I'll also focus a little bit on the differences because the packages are very discrete and different.

There's some complexity in there that we want to help and solve. So that's the next point. We will talk about how Cloudsmith helps solve and make simple some of these complexities for you in the NuGet ecosystem and some of the solutions we have around that. Then I will demo and show off a little bit of this actually live for you so you can understand some of the stuff that we've been working on here, especially to cater to .NET dev specifically. So I'll show off some of the easy stuff you can do with the package installs and stuff like that.

But I'll also touch on, you know, some of the work that we're doing with PowerShell modules, et cetera, making the Cloudsmit CLI available and in places like the Chocolaty Community Repository. So then after that, Dan is going to really dive into, but wait, there's more! All the additional platform features that you get, are just parts of being a Cloudsmith customer. So there's a lot more there that we can touch on, but we'll try and limit it to the time that we have here today, and then we'll leave it open for a Q&A at the end as well because I'm sure there'll be some questions. And will have Mikey chiming in with anything coming from a Cloudsmith customer or user perspective as well.

What is NuGet?

Adil Leghari: Any time he sees fit as well, so sound good? Let's get into the meat of the talk show. So first of all, what is NuGet, right? NuGet is the official package management system the .NET development community, right? Those are NuGet packages. And so that includes a platform and tooling to help .NET developers create, publish and consume and share this reusable code. Why reinvent the wheel if you don't have to? A lot of NuGet devs will build,

let's say you write some code to essentially solve a problem, and then now you can have that library packaged as a NuGet package and other developers now, when they need to solve that same problem, they can just pull down your library, install that NuGet package and then they're ready to go with the tools available in that package to make their development and workflow a lot more efficient. So it encourages the reusability of code and the functionalization of those things. And so the smallest unit of that is a NuGet package.

So a little background for you. NuGet was created in 2010 under the name “NuPack”, I believe, according to Rob from Chocolatey, I believe it was “Nu” before that? He mentioned it once. So I think it was “Nu”, firstly, that it was talked about but referred to as “NuGet” or “NuPack” earliest and then also then moving to NuGet as a format by the Outercurve Foundation. The Outercurve Foundation is now very much a precursor to the .NET Foundation set up by Microsoft.

Some people, you know a thing or two about, Mikey. So of course, NuGet packages themselves, what do they contain? What is the NuGet package? So it's actually kind of very universal and multi-use format. So it can be very different depending on the type of package you have, and we'll highlight some of that here. But the main components sort of to boil it down, are a library sort of like a compiled piece of code, or a binary, that actually stores some of the functions and other features of the other that you're trying to use.

So in the case of .NET/C#, it can be a DLL in the case of a Chocolatey package that can be a binary itself, EXEs or MSIs. And along with that is related files so often describing how to use or install the package itself in the case of Chocolate Factory, this is actually a PowerShell script that does the actual installation upgrade or uninstall of the package. So that's part of related files there.

And then we have a manifest file, the Nuspec, if you will, right?

This is the metadata of about the package version title author information like good details to have there, including dependency format as well if you have any dependencies for that package are defined here in the nuspec. OK, so let's talk about NuGet flavors and communities. And again, I'm going to get into some of this stuff, and I just want to do a quick call out here that there's no quiz later, so you don't have to remember all of this. But a big part of what is NuGert, NuGet flavors and communities, tooling, we tackle a lot of this in a blog that I published last week.

So that blog will go and cover this in detail, too as well. If you don't want to remember all of it now, I'll do a quick summary for you, for the different communities,

NuGet Flavors: Communities and Tooling

.NET (C#) Community

Adil Leghari: So let's summarize some of these use cases, right? So there are three big groups here, firstly, let's talk about the .NET developer community, now this is largely C# folks, but also F# to a small extent. The VB, you know, the hangers-on that are out there still running, VB. So the organization behind this is very much the .NET Foundation now as in NuGet.org, of course, and Microsoft that sponsor the .NET Foundation.

And so in addition to that, of course, the language of choice for this is largely C#, so C# devs that are working, you know, in Visual Studio, et cetera, but of course, some F# and VB too. The tool of choice is ubiquitous here, so Visual Studio is the tool that is used most popularly as an IDE an “integrated development environment”, so Visual Studio is what he was to write that code and has some integration built right in for NuGet packaging.

And of course, NuGet CLI as well. So NuGet.exe can be used by devs when they're looking to do something or preferring to do it from the command line or also when they need it in a pipeline. So it's not interactive.

The public repository of choice, of course, is the NuGet Gallery. The NuGet Gallery is probably the largest public repository of NuGet packages. It contains almost 300,000 NuGet packages. So for the sake of our discussion today, it is the largest. So that's a lot of packages that are up there.

And so these packages now have moved to NuGet format V3, just a heads up. So NuGet v3 is the newer format. NuGet.org moved from V2 to V3, largely citing concerns around scaling and the availability of NuGet.org. So they had to solve some problems there, and so they upgraded to NuGet V3. And so that's really a re-artifacting of the whole format. So, very much not compatible with the other ones in that sense.

So in addition to that, largely the composition of these are our libraries, so compiled code that other developers can use so they can quickly import that individual studio and then it's ready to go for them. They can just start using that from the repository and start using the IntelliSense and the functions right from that code built into Visual Studio.

PowerShell Community

Adil Leghari:

The second group is a group we know very well, right, Mikey? So the PowerShell developer community, right?

So this is not a “formal group”, per se. I mean, of course, PowerShell team shout out to Mike Greene and Steve Lee over at Microsoft do work over there and do push the technology. But largely it's the community of folks here in the PowerShell community, and we show up a lot on Slack and Discord and places like that. And of course, Microsoft is behind this technology. The “Shell Father” himself, Jeffrey Snover, who wrote it back in the day or who invented the format.

Mikey Lombardi:

It's worth calling out that the language is open source and has been for half a decade now. It's Microsoft, but it is all developed out in the open and has been open source for a while. Which is great because people like me and you get to go and bother them with our stuff all the time.

Adil Leghari:

Yeah, we can happily submit our suggestions and we get a response back, say we await your PR, right?

I will call that out a little bit here in the tools of choice too. Obviously, the language is PowerShell, but but largely here, you know, there's two main ways that folks interact in the PowerShell format. One, is powershell.exe. So the original Windows PowerShell does come bundled in, and for a lot of sysadmins, of course, they know on every Windows OS, right? So it's in the Windows ecosystem, it comes built into the box, so it's easy to use.

But largely Mikey, like you called out PowerShell, the pwsh.exe is now cross-platform and open source. So this has made it almost ubiquitous. Now it's across all technologies and a lot of OSS adoption has increased, once it was open source there.

The open source version of Microsoft PowerShell is the very popular format now and ever growing and where a lot of the code now, the functionality lands, right? According to Microsoft, I believe Windows PowerShell is a feature complete, right? It’s just more bug fixes and security updates. So all this new development is occurring on PowerShell cross-platform.

What are we referring to is as now? Should we call it Microsoft PowerShell officially?

Mikey Lombardi:

It’s just PowerShell.

Adil Leghari:

Yeah. Okay, let's let's stick with that. And so, that's obviously increased for PowerShell devs, the availability and the accessibility of the code. Right now, of course, the big, big public repository is the PowerShell Gallery. The PS Gallery hosts a lot of the packages, over 9600 unique packages.

The great thing about it is that it's an easy format to be able to push up there. Once you do have your manifest and other things you find, you can push any PowerShell module up there. Now we know, Mikey, you and I, that sometimes what will have to end up happening, at no fault of the team, is the fact that availability can sometimes be an issue, right? Which is where it's helpful to have your own internal or private repositories, which we can get into.

Mikey Lombardi:

Yes, totally, and running services is hard, and calling out to public services from every server is maybe not your favorite thing in the World. Certainly, not your security team's favorite thing.

Adil Leghari:

Exactly. And it helps to have that level of vetting and internal verification, which an internal repository affords. So we'll touch on tha int a little bit. Now, just a quick point here, so PowerShell modules themselves, they are just files, scripts and manifests in a folder essentially once they get on to the box in the endpoint.

So NuGet here, is actually used as a delivery mechanism, so it's what's used to package the actual module itself. So when it's uploaded into a NuGet repository and when it's pulled down from a NuGet repository is when nuget.exe is used. So NuGet actually does under the hood pack and unpack that package for you into a PowerShell module. So it's a delivery mechanism, so it doesn't actually live in that format. And importantly, this NuGet V2 not NuGet V3.

Mikey Lombardi:

It's also great for dependency management. That’s really hard to figure out on your own. And version pinning and all that stuff. So you get that for free just by using the NuGet package format.

Adil Leghari:

Exactly, that version pining and dependency is one of the great features of NuGet for sure.

Chocolatey Dev Community

Adil Leghari:

So great segway, here to the Chocolatey Dev Community.

This is a community, Mikey, also near and dear to our hearts, fellow choco-fam from back in the day. So it's definitely alive and well. And a lot of sysadmins now use Chocolatey as an easy way to be able to install any package under the sun on all your endpoints. I know, Stevie is probably in the chat right now, he is a Senior SA and Manager over at Chocolatey. He's helped me a lot with the PS Cloudsmith module, so shout out to Stevie, I appreciate all you do. For sure he is very active with this and he's a big, big advocate for it.

So Chocolaty Software, of course, is the organization that was founded by Rob Reynolds. Hi, Rob if you’re here listening.

The language of choice is, of course, is the Chocolatey CLI so choco.exe. Obviously PowerShell is also under the hood, using PowerShell scripts to do the install of the applications. The tools of choice is the Chocolatey CLI of course, and that's choco.exe and also Chocolatey GUI, there is a GUI format as well.

Gary Park over at Chocolatey has done a lot of hard work around creating GUI to be able to install packages for those who prefer a graphical user interface. Now for PowerShell and for Chocolatey, I will call out in terms of tools of choice, a lot of folks who are in the dev community have obviously moved to Visual Studio Code, which is the popular IDE for many language formats, not just PowerShell stuff. But but it's definitely useful here. PowerShell has extensions built in for VS Code, of course, Chocolatey does as well, thanks, Gary.

So along that line, composition of Chocolatey packages, this is where they're different, right? Chocolatey packages consist of binaries, and these binaries are .exe’s around the size of the actual application installers themselves. And those are attached PowerShell scripts, which allow you to install, uninstall or upgrade the application. So a lot of sysadmins end up using this as a way to install packages in any format and the Chocolatey Community Repository, of course, contains so many packages out there. I think there's over seven thousand? I'll get called out in the chat.

But around 7000 let's say packages that are up there, so that every piece of software under the sun you can package under there and is available now. If you are interested in Chocolatey stuff, we do, of course, support Chocolatey packages natively in our NuGet Repositories solution. But if you're interested in using Chocolatey internally as a business or as an organization, they do offer Chocolatey for Business, which is a business offering.

I encourage you to take a look at that. It'll make it a lot easier where you can right click any MSI//EXE quickly, generate a Chocolatey package in less than five seconds. It's like my Choco Fam is showing here, right?

Mikey Lombardi:

Well, yeah, there's two things about Chocolatey, right? The first is as you kind of obliquely talked about there.

Adil Leghari:

Sorry, Ryan said, over 9000 packages, 9000. Now, over 9000!

Mikey Lombardi:

I don't know if anybody watching has ever tried to figure out how to silently install something repeatedly? That process is not fun, particularly on Windows, Chocolatey extracts that. MSIs can have any number of different flags and features and settings. Chocolaty one, lets you do that in a very reliable, repeatable way. But two, Chocolatey for Business cuts down that time even further by just automatically doing it, which is great.

I've argued everywhere I've ever worked, we should be using Chocolatey to install stuff because it's just way, way easier. And I firmly believe in packaging internal apps, stuff that never leaves the organization, packaging and deploying it with the same exact tooling that you're already using for everything else. Why not?

Adil Leghari:

And all the more reason why you would need an internal repository for that, right? So that's where we come in as Cloudsmith. If you are a Choco dev out there or using Choco as an organization, please reach out to us because Cloudsmith can help you with the repository piece of this. It’s just basically another service you don't need to manage yourself. We abstract away a lot of the complexity for you.

So great opportunity for our next poll question. How are you using NuGet packaging today? Are you using it the .NEt/C# World? Are you in Visual Studio as a dev? Are you using it with PowerShell? Are you using Chocolaty NuGet packages? Or just a mix of all of it? Or none of the above? There's no right or wrong answers here, but it helps us to get an understanding of our audience. So feel free to vote in the poll if you can, and we'll take a look at those results later. If we have time.

And while folks are voting, this is a good segway for me. Mikey, why don't you regale us with how you plan to use Cloudsmith in some of your gaming stuff?

Cloudsmith Use Case: Game Development with Mikey Lombardi

Mikey Lombardi:

So the first thing to say is that I'm a fool, and I decided that what I should do is take a tabletop skirmish war game and what's the best way to play that? Obviously, that'd be like a tool like GUI or whatever. That's not what I'm doing. I'm doing a text based user interface.

If you remember ye olden days when you would start a game in a terminal, and then it would prompt you with text? Exactly, like Zork. That's obviously how you play a war game, that's actually a clever idea. So I started going down that road and I realized very quickly two things. One, I needed to make this whatever this, this framework ends up being extensible by default because solving extensibility after the fact is awful. I've been there. I never want to do it again.

Shout out to Dave Armstrong, who I think might be watching. We worked together on a similar problem space, also using Go when we were all together at Puppet, so I already knew that I wanted to do that.

And then I very quickly realized, there's not really a package format for your games modules. Like, what does that look like? And so I panicked because I did not want to invent the package format or try to munch something weird to fit. I didn't want you to need anything that couldn't live in the binary. It had to just work from inside of that. And so, I DM’d Adil, I said, Hey, is there anything for this? And he said, yeah, you can use raw feeds for that, which just lets me treat, in this case, it'll be a TAR.GZIP right? I don't know what I'll actually call it, but that's functionally what it will be. And then I can treat those as versioned artifacts in a feed. So that's the one thing.

The other thing that got me really excited after I started looking is the multi-format repository idea is being able to package my app for Debian, for RPM, for Chocolatey, for Docker image, for Scoop, for anything and everything. And then I can just have one URI that I send people to and say, If you want to install it, use your tool pointed at this at this link, that link stays the same regardless of what tools, I don't have to maintain it. And then that just works, which seems really, really good because I also didn't want to figure out how to maintain a feed for Debian and a feed for RPM and a feed for NuGet, the idea of that terrified me.

So I'll be using Cloudsmith extensively for all of my installable open source stuff and for the modules and plugins and whatnot.

Adil Leghari:

And we appreciate your continued patronage here. Yeah, the check is in the mail. Thanks. Yeah, it's great. I mean, this is the thing. This is what we get off on, folks. So you enjoy this more than anything is the idea of being able to solve problems for you.

I think that that's the biggest thing. That's a reason why it was invented right back in the day? Lee and Alan, Dan, you were there.

Dan McKinney:

That is exactly what it is, to make things easier and solve problems. That's exactly what it is, and package management is a tricky problem to solve, as Mikey said, it really is. So that's the whole purpose.

Adil Leghari:

And that was kind of the origin story, right? A little bit, Dan, just to back up, I think, you know, Lee and Allen were actually just trying to solve an actual problem.

Dan McKinney:

That's exactly what that was.

You know, it was built from a pain point that our founders experienced themselves. That was the genesis of Cloudsmith you know, so it's pinpoint we're familiar with. There's no truer explanation than that.

Adil Leghari:

It’s very much that, for devs by devs.

Mikey Lombardi:

And that's the same thing with both Chocolatey and NuGet, right? People who are writing .Net code wanted to share the code in some way that didn't involve copying stuff from repository folders and Chocolaty. Folks were like installing software “onesie twosie”, and having to manage everything differently and having a different API for everything is not fun. I don't like that. What if I could standardize?

I think most good software comes from a developer trying to solve the problem for themselves and then realizing, oh, actually, this solves a problem for a lot of people. Let me just make this available!

Adil Leghari:

And pretty much every PowerShell modules are the same, and .NET libraries are just trying to solve problems. Yeah, and it's like, wait, other people will have the same problem?

Let me develop out loud. Thank you for both for your input to that.

I think it's very interesting to see how these real world problems bubble up and how we try and solve them with the formats that we have.

NuGet: Each Flavor is Unique

Adil Leghari:

So let's touch a little bit back on NuGet here. I will outline some of the differences here. Each flavor of NuGet, if you will, is unique. Each format has its own different composition, as we talked about, with Chocolatey packages with the binaries, right? And PowerShell scripts.

And then the PowerShell modules just be the PowerShell itself, and the NuGet libraries being a little different for .NET.

So along those lines, for every package that you download from these different feeds, you can't just use a NuGet feed from NuGet.org, a V3 feed to point to from a PowerShell module install, or a Chocolatey package install, because it just won't work, right? And Pester is a great example for this. Pester is a testing framework for those who don't know for PowerShell. Pester exists in the NuGet Gallery. PowerShell gallery and the Chocolatey community repository, in all three places.

But it's not interoperable, right? Each package is packaged a little bit differently as far as NuGet packages go, and in that different format, so it's where the complexity comes in and some of the challenges around using different kinds of feeds, right? And so also to add to this, as we mentioned before, NuGet.org was citing performance concerns and scalability of course

moved to V3, which is a rearchitecting of the format. Chocolatey and PowerShell are still on NuGet V2 and they don't necessarily play nice together, so it's not interoperable in that way.

Now along those lines, let's talk a little bit about how Cloudsmith makes this simple for you. Dan, over to you.

How Cloudsmith Helps

Support for ALL NuGet Formats

Dan McKinney:

Yeah, absolutely. Well, you know, as you just said Adil. There's complexities in NuGet and in its use across these three different use cases. So not only do you have V2 and V3, but partial modules, NuGet packages and Chocolatey packages. So the different use cases, different purposes. So how Cloudsmith really helps is we unify this.

And I mean, we do this across all other package formats as well. But specifically for NuGet, you can very quickly create a repository on Cloudsmith, and that's our universal term, of course, is a NuGet feed. And you can very quickly push packages and they’re available.

We support V2 for Powershell, we support V3 for NuGet, we support Chocolatey and it's all abstracted away for you. You only have to focus then on your artifact, really, and we like to oversimplify this two packages go up, packages come down. You know, and I wish it was, but that's the end goal to make it that simple. So supporting all those formats and the variations within is no small task. It really isn't.

I have to give sort of a massive shoutout to the Cloudsmith engineering team and especially David Schmitt, especially David. But you know, in many ways, sort of, you know, it's only out of their hard efforts that we're able to stand up here and tell everybody about this. So absolutely, they deserve the credit for that. And it isn't a small task. It was a lot of work. I could see the amount of work that it was. It was. It was monumental. So that's just one part of it.

So, you know, having that breadth of support that enables you to, you know, serve all those use cases and solve the same pain points, whether you are a Chocolatey user or you're a PowerShell module user or you’re a NuGet. That's not to be underestimated, really.

Upstream Support

**Dan McKinney:**The second thing is, and this is very important, is upstream support. So what do I mean by upstreams? Well, I mean public packaged sources. Typically the public main central repositories for each format.

So we support upstream at Cloudsmith, which means you can still use your Cloudsmith repository as the central location for your packages and for packages that are not present there, you can configure an upstream source that Cloudsmith will then go off and fetch packages from, and we will proxy and also cache the package afterward, If you so choose, which is a very important use case, we'll talk about it in more detail later, because it touches on several things, it touches on availability, it touches on security as well, and that's effectively an isolation layer, so we can revisit that when we get through the demo.

Code and Performance Improvements

**Dan McKinney:**Finally, just general code and performance improvements around our NuGet support. So we have supported NuGet itself for a while now.

We are now bringing upstreams, which is available, and it's actually just been made available to everybody this week, fresh hot off the press, as the saying goes. And we've also brought a host of other performance improvements to our NuGet back end, which again, has that knock on effect of being useful and performant than for all of the users of our three use cases. So I don't like to overstate things. I really don't go in for it, for that kind of thing, but it has been a really big effort by the whole entire Cloudsmith team, NuGet has been a focus for months now to get this ready and to get it out there in the hands of users.

When we really want people to try it out, it's so fast, you know, you just get up and running in a minute on Cloudsmith, you can sign up, create a repository, start pushing packages, start consuming packages and we would love the community to get involved and tell us what they think.

I mean, of course, I would say that as somebody that works in Developer Relations, but they really do mean it because we need that feedback. We need that feedback very much. We would love people to sign up, you know, activate your free trial, get in and start pushing and pulling packages. So yeah, that's how we can help, and that's what we've been doing.

Adil Leghari:

Yeah, along those lines, I just want to like bookmark that again to say, you know, there is a 14 day free trial, but you also have, you know, you can even get availability of all the advanced features right off of something like a team velocity ultra sort of the higher packages.

But not only that, we do openly support free tier and open source users and we treat you like a full customer. In fact, you know, that's where a lot of folks got started even Mikey, when you started looking, the open source here for the open source stuff you're doing. We love open source here, so we're big supporters, so please just come on and use the product and test it out and let us know, give us that feedback because that's exactly how we get better at it.

Full disclosure in full transparency around that getting better is, I want to give a huge shout out to our internal customers, right?

The folks who were actively involved in the private beta. So one specific name comes to mind to shout out, is one of our customers, Chris Hunt. Thank you so much for all your efforts. I know I asked you for permission before to shout you out, so I know it's okay. But really, you know, Chris has been in talks with us before its summit and other places. Chris has been integral in our internal testing of this topic. So thanks so much, Chris, for your help with that. And that's very much, folks, how we do it.

So please come on to the platform, try it out and see if it works for you and test out those multi-format repos in the upstream. Specifically, because upstreams is new and we value that feedback, and that's exactly how we make the product. So we appreciate you helping us do that. Awesome. Okay. Well, then without further ado, I should probably hop in the demo and actually show off some of this live.

Live Demo

Dan McKinney:

So yes, Adil just showed he pulled a Chocolatey package that was not present in the Cloudsmith repository. I know I'm conscious of time. I've only got a few minutes, so I'm going to move as fast as possible. So let's just take a few minutes to talk about a few things that we saw that maybe weren't immediately apparent. So let's talk about upstreams. I'll share my screen in a moment and we'll look at the Cloudssmith repository and the upstream is configured and why we do it the way we do it at Cloudsmith; what the reasoning is for that right? Why is it different?

We'll also talk very briefly about the concepts that we have of a package promotion workflow and upstream are a very important part of that. So that's why we're starting with upstreams. We'll take a brief look at sort of our multi-format repo support and specifically how this applies to NuGet and some other formats as well. And we will then talk very quickly if we have a few minutes about the access controls and the security and entitlements and things like that.

Live Demo

Adil Leghari:

So the fun part of this is we actually have the ability to define a raw package format. So what do you get with raw? Raw just means that I have my own package format up there. Of course, if it’s something like PiPy Python package or Conda or something like that, we do support 28+ formats. So you do have that option there. But raw is essentially just saying I have some sort of file I want to put up here. So it actually is very flexible in what you can define it as, that you can use it as anything.

We rolled some security features in there for you. But we do sign every package with a GPG key, either when we generate for you or one you provide privately in a public/private key pair if you will. And so that that is available to you as a way to do that, right Dan? Have I done a decent job of answering?

Dan McKinney:

No, that's absolutely correct. And in fact, if I'm not mistaken here, Mikey said earlier that that's what he's going to use for his modules or his games. So, yeah, effectively, that's a custom package format for you, Mikey, you know?

Mikey Lombardi:

Well, and not just that, but another use case that sprang to mind while we were talking is something that PowerShell has that a lot of things don't which is updatable help, right?

So PowerShell gives you referenced docs in the terminal, but for binaries, how do you get updated reference docs? You kind of can't, right? They’re part of the binary, that's how they get it. So it occurred to me was that I can just bundle up my markdown docs and then make those a raw format package and then just say, get the latest version of the docs for this thing, and then it'll just know how to find it and pull it the same as it already does everything else. So I'll probably implement that too.

Dan McKinney:

That's right, because you can. This is probably relevant to the question as well, because you can version those raw artifacts as well. So you can request the latest, you know, of a raw artifact, which is exactly the way you think about packaging anyway.

So you know that should give you that flexibility for any files effectively and all those other Cloudsmith features like access controls, fine grained access controls, and entitlement tokens, which are read-only access keys, you can create for a repository, they can create them programmatically.

They all apply to raw files as well, so they apply to every package format on Cloudsmith. So once you get into that workflow, you'll realize the sort of possibilities that are there with a platform like this. A very good point actually Chris, just raised in the chat, package tags. With package tags, you can add custom tags to package anything you want. So we automatically add some tags when we process a package, so we will create tags based on the metadata in a package.

We do this differently for different package formats, but you can create custom tags as well and then use those when you're creating access tokens so you can create an access token that only allows access to a certain tag or group of tags. Or, you know, even a group, you know, except one tag so you can do negation like that and everything. It's very, very, very flexible. It's an ideal way to provide fine-grained access.

Mikey Lombardi:

So a really good iteration on that is one of the other things raw formats would let me do is upload a config file

and then I can say that the only people who can pull this config are whatever application team owns that config, right? All that just natively works. That's packaged, and you can promote that. You can promote a config file in very much the same way as a package or whatever else.

Dan McKinney:

You sure can and sort of related to that Mikey, specifically thinking of the Chocolaty use case.

I know Chocolatey is used in systems administration, building machines, building sets of applications for different groups within an organization. Well, you could tag certain applications as being for the marketing team or certain applications being for the product team. And then when you create those access tokens on Cloudsmith, you just grant access to the groups of applications. And that means that it simplifies the rollout of those apps and you don't have to reinvent the wheel in setup.

So, yeah, honestly, there's a lot to it. But honestly, I could talk another hour on this really, really.

Adil Leghari

I would like even just for a moment, for you to show entitlement tokens as a UI, just to understand how the granularity you can get with that. Before you do that, though, I will put in a quick plug because this is a question that I've gotten before from folks and an ask from some of our users like Chris specifically did bring this up earlier.

I will throw this up real quick on the screen if it works. So one of the big asks was, hey, how do I install Cloudsmith CLI? You know, sort of tooling around coming in this access in windows itself. Because already, I mean, of course, I have Python on a lot of places like macOS and Linux. But I mean, I don't have all this available to me on a Windows box. So what's the easiest way to sort of Occam's razor shortest path between good points to be able to do this?

So to this end, we have actually created a Cloudsmith-CLI chocolatey package up on the Chocolatey Community Repository. It is available right now to install if you do add the version number because it is currently under moderation. And that's my fault. That's on me. I got some great feedback from Paul. Shout out to Paul Broadwith over at the moderation team at Chocolatey this morning, so thanks Paul, for clarifying. I will fix those things and this will be available where you don't need to add the version number probably later in the next few days.

But now you can just do a Choco install of Cloudsmith-CLI and it's going to handle the dependency of Python and all that stuff for you as well. So another thing that we worked on recently just to make this easier for our users to be able to install Cloudsmith CLI in the Microsoft ecosystem. So just trying to make it easy for folks to do so. I do want to quickly get that plugin there because I did have gotten that question before. And now I think we can hop over and you can share your screen and talk about entitlements a little bit. Folks, I know we've gone over, but hopefully, this is valuable.

Dan McKinney

It will only take a moment. So let me just make sure I'm sharing that screen. There we go and show that you should be able to give it a moment to sort of share the screen for you.

Mikey Lombardi:

Yeah, well, that's coming up. One of the things that you pointed out, I think this is going to show is previously the way that I would have had to solve having marketing have their apps and having devs have their apps and stuff would be multiple feeds. Yeah, being able to not do that sounds great!

Dan McKinney:

Yeah, exactly that. That's exactly right. And traditionally that's what people did, Mikey. They had multiple feeds, they had multiple repositories. But then that becomes a management burden that becomes a management overhead. And that's it. We're all about simplifying things a plasma. And so that's exactly what it's like. So let's just very quickly touch on those access tokens that I mentioned and how you can use them with tags.

So you will see in this package list, we have some automatically created tags here. So a lot of these packages have been tagged latest. And I should also mention that latest is determined by the version and we support semantic versioning. So it's not just this was the most recently uploaded package, it's proper latest, rather than just, you know, a recent upload. So you'll see as well that I have some packages here that have little tags like this one here has a tag that says Pro and I have a maybe another one on the next page. There's a tag that says basic, and there's a tag that says Pro again, so you can add anything.The tags can be any, anything that you want, anything that makes sense to you and where they come in really handy is with these entitlement tokens.

So entitlement tokens and I have very unimaginatively named some demo customer and customer and test token, I really should have named those better Adil. So you can create as many of these as you want. You can create them via the API, you can create them via the CLI, and you can also update them, revoke them, disable them and things like that. I'm showing this via the Gooey, but at scale you'll do this via our API.

So let's just take a look at what is a token? Well, as I said, it's a read-only access key for this repository. But what makes it really special are these two sets of restrictions, visibility and usage. And you can also add custom metadata as well, which is very helpful to when you're pulling information on tokens from our API. But the restrictions are the key part, so visibility restrictions you can restrict by search and you can use the same sort of syntax that you use when searching the repositories in the Gooey and when using a search query in the API.

So you can restrict on package name or types or number of downloads or anything. And you can also anchor to the start of the end and do fuzzy matching, and it's really quite a sophisticated search. Including ranges of versions, you know, and all this kind of stuff.

I've restricted this token so that it's only available for the package formats MPM. and Docker. So if you go to use this token to pull a NuGet package, you'll get nothing. In return, you'll get a 404.

But this is also a fully boolean search syntax here. So you can chain these conditions together to make extremely unique tokens. In fact, you know, you could create a unique view on the repository for every token you make.

If you want that, every token could be personalized and effectively give you a subset of the packages in the repository. So that's what controls access.

And then usage, that's pretty, pretty straightforward. You can set automatic, you know, start and expiry dates for a token. You can set a maximum number of downloads or maximum number of client IPs, and that's really good to prevent token misuse. A;lso, it's good to prevent maybe a runaway CI process that's sitting on a retry and just pulling packages continuously using up all your bandwidth, things like that. Those built-in controls are very effective. So that is entitlement tokens.

But one thing I should say before we finish there is that we use entitlement tokens in our logs. So when you look at the client logs for Cloudsmith repository and you see all access and to that repository, it'll be attributed back to the token that was used. So that gives you really good analytics on package usage because that's the other side of package management.

Mikey Lombardi:

Yes, I did not know about this at all. Well, I knew about the entitlement tokens but I didn't know about the tagging. I just guess I never noticed that. But like, that was really cool.

And so I or they will say, that's cool, too. Yep. Absolutely.

Adil Leghari: So I will say this is this was action-packed with with with knowledge here. So I definitely want to say for folks who could join us for the whole time. Thank you so much for your time.

I know that Dan could probably have gone on a lot longer on this as we want to want to have him do at some point. But in the interest of your time, I do want to wrap this up a little bit, hopefully, we answered some of your questions, gave you some value out of this talk. And even if you're watching this back later on the platforms, then please do feel free to reach out. Feel free to set up a trial.

So that's the big part is you can start a free 14 day free trial of any of our types of plans. We give you all the features for free for 14 days after that. You also have the ability to have a free plan or if you're an open source user, we have open source plans to support you as well. So feel free to hop on and just give us some feedback. Try out these NuGet feeds and upstreams because the only way we get better is we get that feedback from you, We treat you like a full customer.

Anyway, you are joining us, so thank you so much for your time and on behalf of my colleagues here, Dan McKinney and Mike Lombardi. Thank you so much for joining us. We know your time is valuable and we'll see you on the next webinar!

Liked this article? Don\'t be selfish (:-), share with others:   Tweet