Automate infrastructure tagging with Yor and GitLab CI

Matt Johnson

Yor + GitLab logos

About a month or so ago, we introduced an addition to our growing lineup of open source tools. We noticed a common theme in community conversations: “tagging resources is problematic.” Manually adding resource tags while trying to maintain consistency across teams, and keeping tag information updated when things change, was a problem perfectly suited for automation.

Enter Yor. 

Yor is an open-source tool that simplifies and automates tagging of infrastructure as code (IaC) resources, enabling a whole host of “code to cloud” tracking and auditing. With Yor, you can:

  • Easily trace which block of code created which resource.
  • Easily trace where in a codebase a cloud resource came from.
  • Get “at a glance” information for owners and previous editors of a cloud resource.
  • Build custom logic for adding team, department, and security permissions tags via simple YAML configuration.

Designed to be hands-off, Yor runs in your CI pipeline so that all cloud resources have the tags you need when you need them. 

How this works within GitLab CI

Before we go any further, it’s important to note that Yor is fully extensible and supports any CI pipeline you need such as GitHub Actions. It can also run as a pre-commit hook and a standalone CLI. 

Now, let’s take a look at such a setup within GitLab CI! For this setup, we’ll need two things:

1. A personal access token for your user, with commit/write access to the repository

GitLab does not grant CI pipelines write access to the source repository by default. Since we want Yor to commit our new tags automatically, we’ll need to provide this via a secret. This can be accomplished via GitLab. 

Personal access tokens

We’ll then store the token as a secret in our GitLab CI variables, along with three other key configuration items for the pipeline:

  • AUTO_COMMITTER_EMAIL: Whatever e-mail address you want associated with the git commit of new tags.
  • AUTO_COMMITTER_NAME: Whatever author’s name you want associated for the git commit of new tags.
  • GITLAB_USER_NAME: The GitLab username who owns the personal access token.
  • GIT_PUSH_TOKEN: Our newly generated GitLab personal access token.

GitLab CI variables

2. A repository containing infrastructure as code for Yor to tag

While you’re more than welcome to use your own, we’ll demonstrate below how to run the pipeline in “don’t commit” mode first so you can see the changes Yor would make to a live codebase. For testing, you could also fork my sample repo, yor-gitlab, which contains three AWS S3 resources written in Terraform and already has the GitLab CI pipeline configuration for Yor.

Let’s go!

We want our Yor pipeline to do the following:

  • Download Yor to our GitLab CI runner
  • Check out the branch in question (not a detached head state as is the default)
  • Run Yor to generate, edit, or update tags.
  • If there are changes to the tags, commit the tags back to the branch or merge request.
  • Push the commit with “Skip CI” settings so that we don’t run our entire pipeline again just for adding tags.

Simply add the following into your GitLab CI configuration:

.git-script: &git-script |
 cd $CI_PROJECT_DIR
 git status
 lines=$(git status -s | wc -l)
 if [ $lines -gt 0 ];then
   echo "committing"
   git config --global user.name "$AUTO_COMMITTER_NAME"
   git config --global user.email "$AUTO_COMMITTER_EMAIL"
   echo ".yor_plugins" >> .gitignore
   git add .
   git commit -m "YOR: Auto add/update yor.io tags."
   git push -o ci.skip "https://${GITLAB_USER_NAME}:${GIT_PUSH_TOKEN}@${CI_REPOSITORY_URL#*@}"
 else
   echo "no updated resources, nothing to commit."
 fi

stages:
 - yor

run-yor:   
 stage: yor
 script:
   - git checkout ${CI_COMMIT_REF_NAME}
   - export YOR_VERSION=0.1.62
   - wget -q -O - https://github.com/bridgecrewio/yor/releases/download/${YOR_VERSION}/yor-${YOR_VERSION}-linux-amd64.tar.gz | tar -xvz -C /tmp
   - /tmp/yor tag -d .
   - *git-script

For a dry run (where no commits occur), you can remove the last line - *git-script to stop the pipeline from committing anything after running Yor. Alternatively, you can also unset the GIT_PUSH_TOKEN GitLab CI variable so that the pipeline doesn’t have permission to push to the repo.

Once saved, you will see Yor triggering output in the CI pipeline showing the tags added or updated, and then a new commit with our changes:

Automated commit with Tag changes

Yor output in CI Pipeline with new tags

That’s it! IaC resources will now automatically be tagged with useful information, ready for the next parts of your deployment pipeline!

You’ll notice the next time your pipeline runs, Yor will only make changes if there are new or updated resources, otherwise confirming zero resources changed and not making a new commit:

Yor findings summary

Learn more about Yor

Through automatic and consistent tagging, Yor traces changes down to each line of code, enabling GitOps by making it easier to triage misconfigurations in production and audit changes for compliance, finance, and ops use cases.

Yor’s docs are a great place for more information on the tool, including common tagging use cases like development cycle, development flow, or organizational structure. We also recently walked through an example of how to use Yor for ownership mapping using YAML tag groups.

As always, we’d love to hear from you. Drop us a line in our Codified Community channel in Slack, let us know how you’re leveraging Yor in your daily work, and be sure to stay up to date on new feature releases!