GitHub Actions: Building a Dynamic Matrix

Overview

Sometimes when you’re creating GitHub Actions, you have to get creative to get the job done. One such example is when you need to run several jobs at the same time, but the number of jobs and inputs are variable. If I were using Azure DevOps, I would use the ${{ each }} syntax in the YML to run the job multiple times with different inputs. GitHub Actions wasn’t implemented with this type YAML syntax (loops or anchors), but we can use a dynamic matrix to accomplish something similar. Using if: conditionals on jobs could work too, but then they show up as skipped in the Actions UI.

Instead, we can build a matrix dynamically. Let me show you how!

Full disclosure, I’m piggy-backing on this post from my co-worker @kenmuse, “Dynamic Build Matrices in GitHub Actions. If you’re interested in this topic, check out his post too!

Example

In my example, I was tying this to an IssueOps migration approach. I had an issue template that asked for a list of repositories. I wanted to dynamically run a job for each repository in the list.

To prepare the workflow, I parse the issue body with the stefanbuck/github-issue-parser action. This action will convert my input into a predefined variable as opposed to having to write something to parse the issue template myself.

Then, I write some JavaScript to convert the list of repositories into a valid JSON object. Finally, in another job, I can then use the JSON job output of the previous job to create the dynamic matrix.

Workflow

name: dynamic-matrix
on:
  issue_comment:
    types: [created]

jobs:
  prepare:
    runs-on: ubuntu-latest
    if: contains(github.event.comment.body, '/run-automation')
    outputs:
      repositories: ${{ steps.json.outputs.repositories }}
    steps:
      - name: Check out repository
        uses: actions/checkout@v4
      
      - name: Parse issue body
        id: parse-issue-body
        uses: stefanbuck/github-issue-parser@v3

      - run: echo $JSON_STRING
        env:
          JSON_STRING: ${{ steps.parse-issue-body.outputs.jsonString }}

      - name: Build matrix
        uses: actions/github-script@v7
        id: json
        with:
          script: |
            let repositories = process.env.REPOSITORIES.replace(/\r/g, '').split('\n');
            let json = JSON.stringify(repositories);
            console.log(json);
            core.setOutput('repositories', json);
        env:
          REPOSITORIES: ${{ steps.parse-issue-body.outputs.issueparser_repositories }}

  run-matrix:
    needs: prepare
    runs-on: ubuntu-latest
    strategy:
      matrix: 
        repository: ${{ fromJson(needs.prepare.outputs.repositories) }}
      fail-fast: false
      max-parallel: 15
    steps:
      - run: echo "${{ matrix.repository }}" # this will print the repo name

This will render the following workflow: Building a Dynamic Matrix in GitHub Actions

Another Workflow Example

Here’s another example that I’m sharing here for reference. I thought this one was interesting because:

1. I was building my own JSON object manually (see lines 14-15 below)
2. Note the difference with the matrix: line (line 22) compared to lines 40-41 in the example above. Since I am already defining repository in the JSON array I’m building manually, I don’t need to define my own object for the matrix.

name: workflow-dispatch-dynamic-matrix
on:
  workflow_dispatch:

jobs:
  prepare:
    runs-on: ubuntu-latest
    outputs:
      repository: ${{ steps.json.outputs.repository }}
    steps:
      - name: build matrix
        id: json
        run: |
          repository='{ "repository": ["repo1","repo2","repo3","repo4"] }'
          echo "repository=$repository" >> "$GITHUB_OUTPUT"

  run-matrix:
    needs: prepare
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix: ${{ fromJson(needs.prepare.outputs.repository) }}
    steps:
      - run: echo "${{ matrix.repository }}"

Summary

I have built this out for a few customers now, so I wanted to finally capture this in a blog post. This can be really useful for dynamically running jobs in GitHub Actions as opposed to using if: conditions to not run jobs. The problem with the if: condition is that it will still show the job as skipped in the Actions visualization.

Another example I have built recently was using a dynamic matrix to run a trigger N number of workflows in other repositories managed via a JSON input file stored centrally (this I will have to write as a separate blog post in itself!).

There are times where this is a no-brainer solution, but I have also seen others try to shoe-horn this into a solution where they are trying to run non-similar jobs in a matrix. This is not the intended use of a matrix I would say. Building out a dynamic matrix is a great solution for running similar jobs and the only thing that changes between the jobs is some input parameters.

I hope this helps you in your GitHub Actions journey! 🚀