Cloud Drops - What is Continuous Integration (CI)?

You may have heard about Software Code or Infrastructure as Code. Well, in this video, we’ll be talking about pipelines as code. If you’re familiar with tools like Azure DevOps, Circle CI, GitHub, GitLab and Jenkins, you may notice a trend where these platforms are allowing you to define your pipelines as code. Throughout this video, we’ll be defining a multi-stage pipeline in Azure DevOps, and picking up some tips along the way.

Recently on Twitter, I was asked by @thegraycat on whether I knew of any resources to manage pipelines in version control. I sent across several top of mind thoughts over Twitter, but it got me thinking that there may be others with the same question and it could make a good blog post. So here we are, as I talk through some of my considerations for pipelines as code.

I’ve recently released a few Cloud Drops episodes on Git related content. The Git Behind the Scenes video was incredibly well received. I’m also aware from my day-to-day discussions that there’s a mix of experiences with Git, so also made a Git 101 Video. In this Cloud World that we live in, version control is an important concept beyond the ’traditional’ developers. Infrastructure Engineers can now version control their Infrastructure as Code, or maintenance scripts. Data Scientists can version control their experiments and tests. And of course, developers can version control the code for their software. I also consider version control as a gateway or first step into the world of DevOps. Typically when you think about build and release pipelines, you are triggering based upon some version control event (e.g. a commit to a particular branch, a merge of a pull request, etc.). Over the past few years, I’ve seen a trend where organisations are looking to automate quickly, rather than relying on the traditional hands-on-keyboard approach which can be error-prone and time consuming. Whether we’re talking in this context about Infrastructure as Code, Application Code, database schemas as code, data science experiments or any other representation as code, it doesn’t matter. Typically the roads lead back to the same place, to version control. So in this blog post, I’ll be covering the fundamentals of Git and how to get started. For anyone that is particularly inclined, there will also be some information on what’s happening behind the scenes when you work through these fundamental concepts.

Have you ever had a scenario where you need to maintain versions of a set of files? Or have you needed to collaborate on files with colleagues in some way? That’s a common scenario for developers, infrastructure engineers or data scientists/developers. It’s an increasingly common problem. Some people solve this using file shares or FTP Servers with numerous files, v1, v2, v2-final, but that doesn’t scale and is quite a messy approach. Surely there is a better way? That’s where a Version Control System can really help you. One such option is Git. Git is a distributed version control system, which means that rather than relying on a central location to host and store the entire set of files and history, each machine has a full version of the codebase and history locally. This means each user can be productive locally and independently on their own machine. Git is also optimised to be very lightweight and perfomant.