Automation and Orchestration had become the focus for DevOps culture.
Bringing automation to a company isn't an easy task; it requires coordination among teams and changing how we understand and interact with the application lifecycle.
Docker was responsible to start changing this mentality (monolith to API), ending large hours of bootstrapping nodes, getting the packages and configuring system files, or some entries in a relational database. Building a new application now just requires running a build command and waiting a couple of minutes to complete, uploading an image to your Docker registry, and finally deploying it in your infrastructure with the possibility of having the same configuration running over different environments.
GitOps is a culture that is changing the DevOps paradigm. 10% is on tools and 90% on philosophy. IaaC must be the objective for a DevOps or SRE department.
But what does GitOps mean?
Kubernetes started allowing to change the way that industry used to deploy and build infrastructure taking care of this, the GitOps concept appears. GitOps challenges you to automate the entire infrastructure as much as possible, accounting for traceability, versioning, and observability. A Git repository becomes the single source of truth for infrastructure.
Our development and infrastructure lifecycle will depend on our branching model. Some companies prefer using the master branch as a source of truth, working with short-lived branches for the daily work over small increments.
Some others will prefer working with Milestones that fit their sprint and opening branches every time a hotfix has been released (similar to GitFlow)
Independently of the branching model, implementing the GitOps philosophy will take your SCM as the central piece of the infrastructure.
Before starting to implement GitOps, you have to start creating declarative infrastructure, based on facts instead of by a set of instructions.
Those are a set of rules that we need to follow:
#1. Infrastructure as Code: Building declarative infrastructure instead of instructions guarantees that always the same infrastructure will be deployed and only can be altered via environment-specific declarations. Kubernetes is just another good example of how declarative infrastructure can be defined. Altering the stack behavior sometimes is needed when the pipeline needs to pass from different environments (for example, from stage to production)
#2. The canonical desired system state versioned in Git: There are two different states for a Cluster: The desired cluster status and the canonical cluster status. Writing Infrastructure as Code creates some deltas of time where the deployed version differs from what is in the repositories. One of the key facts about GitOps is having the SCM as a central piece of your infrastructure, where we can version the changes and alter the cluster status by approving and merging a Pull Request. Automated delivery pipelines roll out changes to your infrastructure when changes are made to Git.
Tip: It is a good practice having two repositories, one for your application and another for the manifest where every commit to your app can result in a deployment.
#3. Approved changes that can be automatically applied (like CI/CD): Changes can be made only a Pull Request.
But... in DRPs?
DRPs happen. Some companies where there are very hard guidelines in the base to reduce the MTTR or MTTA but independently of the nature of the incident, it takes human intervention to fix the issue. But...when we talk about infrastructure incidents things become worse, because metrics are important in all levels. GitOps culture comes to the rescue: Having a versioned repository with all the changes and using a well know provider like GitHub helps DevOps or SRE team to reduce this time, it’s reduced only to detect which commit breaks a component, removing this component of the repository HEAD and wait until the cluster matches the desired status. But isn’t all about applications or high-level components, it’s about having your entire infrastructure automated, if a blackout in a datacenter happens and you lose your instances, deploying another cluster in another geographically location must be possible. Traceability is important but it’s much better when it is tied with suitable tests.