Using Azure DevOps pipelines for cloud-hosted POP Forums

Last week I did my first production deployment of the cloud-hosted version of POP Forums. I'm really excited to offer a version that others can use without setting it up themselves (I'm really excited about the performance, too!). If you've followed me for any length of time, you know that I've been maintaining POP Forums as an open source project since around 2003. The new cloud-hosted version is about 95% the same code, with a few substitutions in the dependency injection container to facilitate multi-tenancy, especially in the asynchronous Azure Functions to make sure that they're acting on behalf of the right tenant. What I'm excited to share is that it's so easy to use the output of the open source project and get those bits into the commercial product.

Any time a commit or merge is made to the master branch of the open source project, a build is triggered in Azure DevOps. Beyond the actual building and running the unit test suite, there are several important steps:

• dotnet pack: All of the assemblies are wrapped up in Nuget packages. POP Forums has a Razor View library that has all of the server-side UI bits and controllers, so as demonstrated in the sample project, you can run the thing strictly from packages.
• nuget push: The packages are all pushed to MyGet.
• gulp: The tasks in the gulpfile.js run to minify and package the scripts and css.
• npm publish: The output of the front-end packaging is pushed to MyGet.
• All of this is pushed out to a CI build site, where I can verify the changes. From commit/push to on-air takes about three and a half minutes.

This was all happening even before I decided to light up a hosted version of the app, because I was using it to power the forums on CoasterBuzz. Keep in mind that all of this was free to set up. GitHub is of course free for open source, and when you have a team of two, Azure DevOps is free as well. You can only run one build job at a time, but again, that's not a constraint.

As you may have guessed, the cloud-hosted POP Forums app references the packages on MyGet so it has the very latest bits ahead of the typical releases. The source for the commercial app is stored in a git repository in Azure DevOps, along side of its build pipelines and work items. Pushing to the master branch has a similar outcome, in that the new bits land on a development copy of the app, with configuration for fake payments and everything. It was here that I was able to test the recurring billing and maintenance tasks over the last few months.

The hosted app doesn't take a ton of additional schema to work, but there have been quite a few changes to it in the weeks leading up to release. For those, I used DbUp. I am seriously enamored with this library. It's always made me uncomfortable that most database versioning works on the premise of calculating the diff between old state and new state, and then generating SQL scripts to bridge the gap. If things get into a weird state in any one environment, your CI, QA or staging updates then are not necessarily exactly what will happen in your production deployment, and that could have unintended consequence. DbUp simply executes the exact same scripts at every update, in order, so your lower deployments are in fact a rehearsal of the production deployment.

The shorter story is that the DbUp project is a console app, so I ship that in the artifact, and a PowerShell call (on a Linux build agent!) runs it as part of the deployment pipeline.

Azure DevOps has an artifact-based deployment mechanism, so when I'm satisfied with the build that was deployed to the dev environment, I can specify manually specify that build be deployed to the production deployment. You can set up gated deployments as well, meaning that a user with the right permission has to click a button after a verified deployment to a lower environment, but I don't typically deploy to production as often.

Some other observations:

• Did I mention that all of this automation is free? I doubt the cost to Microsoft is significant, and when someone in my career stage gets all of this for free and applies it to a day job environment with 40+ people in there, they certainly make it up with licenses.
• Because the back-end code is all .Net Core, I can build on Linux build agents. When I switched from Windows-based agents, I shaved about a minute off of build times. (Not exactly related, but it should not be surprising that scaling app services up or out is also faster on Linux.)
• The longest time of the build is package pulling (NuGet and npm). You can get around this if you host your own agents (not free), but since the free agents are ephemeral, they're going to pull every time. I'm surprised that they haven't found a way to cache for you, since it would be in their own interest to not tie up the free compute and incur the bandwidth cost of every pull.
• The set up time for all of this is a fraction of what it used to be. I felt like I had to relearn all of this once a year, but it has been predictable and easy the last few years.

Overall, I'm really satisfied with the experience of using Azure DevOps for pipelines. Getting the value that you're creating in front of customers fast is easier (and cheaper) than ever. Go visit meta.popforums.com to see the hosted forums in action!