GitHub Actions: Publish Code Coverage Summary to Pull Request and Job Summary

Overview

This is a follow-up to my previous post: The Easiest Way to Generate and Publish .NET Code Coverage in Azure DevOps

I was familiar with adding Code Coverage to my pipelines in Azure DevOps and having a Code Coverage tab appear on the pipeline summary page, but I wasn’t sure what was available for GitHub Actions. With GitHub Actions really starting to pick up steam, especially with recent additions such as Composite Actions, I thought now would be a great time to explore.

Adding Code Coverage to Pull Request

I found this GitHub Action in the marketplace - Code Coverage Summary. There might be others, but this one seemed simple and had the functionality I was looking for.

This post assumes you are using the coverlet.collector NuGet package. For a refresher, see the “the better way” section of my previous post.

Here’s the relevant part of my GitHub Actions workflow file:

    # Add coverlet.collector nuget package to test project - 'dotnet add <TestProject.cspoj> package coverlet
    - name: Test
      run: dotnet test --no-build --verbosity normal --collect:"XPlat Code Coverage" --logger trx --results-directory coverage

    - name: Code Coverage Summary Report
      uses: irongut/CodeCoverageSummary@v1.3.0
      with:
        filename: 'coverage/*/coverage.cobertura.xml'
        badge: true
        format: 'markdown'
        output: 'both'

    - name: Add Coverage PR Comment
      uses: marocchino/sticky-pull-request-comment@v2
      if: github.event_name == 'pull_request'
      with:
        recreate: true
        path: code-coverage-results.md

    - name: Write to Job Summary
      run: cat code-coverage-results.md >> $GITHUB_STEP_SUMMARY

Note the test command here that we are using to generate the Cobertura code coverage summary file:

dotnet test --no-build --verbosity normal --collect:"XPlat Code Coverage" --logger trx --results-directory coverage

The next action is the Code Coverage Summary Report action:

CodeCoverageSummary Inputs:

• filename: coverage/*/coverage.cobertura.xml
• badge: true | false
• format: markdown | text
• output: console | file | both

    - name: Code Coverage Summary Report
      uses: irongut/CodeCoverageSummary@v1.3.0
      with:
        filename: 'coverage/*/coverage.cobertura.xml'
        badge: true
        format: 'markdown'
        output: 'both'

The CodeCoverageSummary v1.3.0 action now supports glob pattern matching for multiple coverage files. Therefore, you don’t have to use the example below with reportgenerator to combine the code coverage report before processing!

This would be enough to show the code coverage in the action run: Code Coverage Summary Report in the Action run logs

However, the fun doesn’t stop there. How useful would it be to post this to the PR so it’s nice and easy for reviewers? Well, the next action shows a simple way we can add (and sticky) a PR comment with our code coverage report:

    - name: Add Coverage PR Comment
      uses: marocchino/sticky-pull-request-comment@v2
      if: github.event_name == 'pull_request'
      with:
        recreate: true
        path: code-coverage-results.md

Perfect - nothing for us to configure here, either. On the pull request, this comment is added:

Code Coverage Summary Report added as a pinned comment to the Pull Request

This is also demonstrated on my pull request here.

You’ll notice the badge along with the markdown table summarizing the code coverage report.

The nice thing with this action is that if a new commit is pushed to the PR triggering a new action run, the comment will be deleted/re-added with the updated code coverage summary.

Adding Code Coverage to Job Summary

I think this is looking great, but what if we don’t happen to create a pull request, how can we neatly see our code coverage report? Well, since May 9, 2022 (and GitHub Enterprise Server >= 3.6.0) we can use the Job Summary!

Since the CodeCoverageSummary action is already generating the markdown for us, all we have to do is append it to the $GITHUB_STEP_SUMMARY environment variable. Add in the following run command to the end of the job:

    - name: Write to Job Summary
      run: cat code-coverage-results.md >> $GITHUB_STEP_SUMMARY

Now, it publishes to the job summary page right next to the workflow run logs! See the screenshot below:

Code Coverage Summary Report added to the job summary

ReportGenerator?

If you are trying to run this on a Windows runner, you will quickly notice the Code Coverage Summary Report action is a Docker-based container action, meaning it only runs on Linux runners. Or perhaps you work in a GitHub instance that uses an Actions allow list to only allow approved actions to run. Don’t worry! You can still very easily generate a nice-looking code coverage report and upload to the job summary.

I have documented this for Azure DevOps in the past, but this would be the GitHub equivalent:

    - name: Create code coverage report
      run: |
        dotnet tool install -g dotnet-reportgenerator-globaltool
        reportgenerator -reports:coverage/*/coverage.cobertura.xml -targetdir:CodeCoverage -reporttypes:'MarkdownSummaryGithub,Cobertura'

    - name: Write to Job Summary
      run: cat CodeCoverage/SummaryGithub.md >> $GITHUB_STEP_SUMMARY

This is what it looks like in the job summary: Code Coverage Summary Report generated by reportgenerator added to the job summary

Conclusion

Maybe not as pretty as the Cobertura report shown in Azure DevOps, but just as effective! Certainly the addition of job summaries makes this a better experience.

And hey, now on the GitHub Pull Request, you get to actually see the code coverage report before the end of the entire pipeline run like in Azure DevOps 😀.