Post

Tips for Handling Dependabot, CodeQL, and Secret Scanning Alerts

My musings on handling security alerts in GitHub

Posted Updated
Security Overview for an Organization
Security Overview for an Organization
By Josh Johanning
7 min read

Overview

I recently had the opportunity to work with a large organization to help them manage their security alerts. Often, enabling security tooling is easy, it’s what comes next (how do we handle alerts, who fixes them, when are they fixed, etc.). For posterity, this post is a summary of my thoughts on how to handle security alerts in GitHub.

My Notes

  • It might seem obvious, but organizations should focus on the critical and high alerts first
    • Teams can get overwhelmed by the sheer number of alerts that Dependabot finds, but should instead be focusing on the critical/high alert count
  • Leverage Pull requests gates
    • For Dependencies: Introduce gates during pull requests to at least not allow people to introduce new vulnerable dependencies (ie: use the Dependency Review Action)
    • I would advise against blocking all PRs from being merged if ANY high/critical Dependabot alerts are present. This would affect PRs that didn’t touch dependencies as well
    • Certainly, in some high-regulated environments, this could have the desired effect of forcing teams to fix their security issues, but in nearly all cases, I would advise against this
    • While you can’t set up a Required Workflow for CodeQL, you can use this for the Dependency Review action/status check for Dependabot (sample here) for enforcing this across an organization
    • For Code Scanning: Same idea, introduce gates to prevent people from merging new code that contains a potential vulnerability.
    • This can be done by adding the CodeQL status check to the branch protection rule. For best coverage, add the CodeQL status check as well as the Analyze status check(s) (ie: Analyze (java))
  • Some organizations introduce an SLA for Dependabot Alerts that are found
    • Example: If a critical Dependabot alert is found, you have 10 days to fix it. High = 30, Medium = 60, Low = 90 (or similar)
    • Typically, a grace period is given when turning on security settings to allow teams to burn down the backlog of alerts
    • One could potentially use webhook event to trigger when alerts are found and post them to a Slack or Teams channel or even as Issues on their repository!
    • Ideally, you wouldn’t need to rely on the SLA for Code Scanning results after initial onboarding since vulnerabilities should be caught and fixed during pull requests with gates 😄. There are more likely to be Dependabot Alerts that are found out of band than CodeQL alerts
  • Some organizations introduce an “alert cap” similar to a “bug cap” - if we get more than 10 Dependabot alerts for example, we have to burn them down
  • Ultimately, organizations need to have procedural practices in place (culture) to make security a concern so that people don’t “ignore” alerts and instead work to fix them
    • If you’re asking “How can I snooze a Dependabot alert”, your team’s approach is wrong. It’s okay to get alerts, new vulnerabilities are discovered in packages every day, but you should be working to fix them
  • Great testing is effective at helping you resolve Dependabot alerts
    • If a Dependabot Security Alert PR is created, if your build job and unit tests pass, then you can be reasonably confident that the PR is safe to merge
    • There isn’t a great way to distinguish between a Dependabot Security Update PR vs. a Dependabot Version Update PR (non-security related), but you can use something like labels defined in your dependabot.yml file to help you distinguish between the two
  • Some teams ask if they can ignore development-scoped dependencies (such as devDependencies)
    • This was added as a feature to Dependabot in June 2022
    • While it is true your end users won’t be affected by a vulnerability in a development dependency, your developers very well may be, and your developers certainly have access to privileged information that make them a prime attack vector
  • Other Dependabot notes
    • Upon merging the PR with the package version fix, the Dependabot Alert will automatically be closed (same with Code Scanning alerts, once the code is fixed, the alert is automatically closed)
    • Not all Dependabot Alerts get created as Security Update pull requests (there’s a limit of 10), but can be slightly less than that due to various factors
    • Sometimes dependencies have a vulnerability disclosed with no new version to update to - in those situations you have to evaluate the risk and decide if you want to continue using that dependency or not
    • Some ecosystems, like Python’s pip, can show you if you are referencing a vulnerable code path in a dependency
  • Using tspascoal/dependabot-alerts-helper scripts to export Dependabot alerts to a CSV file
    • This can be used for further analysis / grouping / management of alerts, especially if you are responsible for a lot of repositories
    • The Security Overview is great, but if you only care about specific repositories but have access to a lot of repositories, it can be a little noisy
    • This tool also supports merging Dependabot Security Update PRs in bulk. The idea, in theory, is that if you used the same old version of a package throughout your organization, and are able to verify it’s a non-breaking change, then you could mass merge those open Dependabot pull requests to resolve those alerts
    • This analysis and testing could be useful if a single package was causing a high percentage of alerts across a team/organization
    • Maybe you can’t do this throughout the organization, but a single team could use some of the tools to at least make changes in their x number of repos that they own
    • Also, who doesn’t like pulling up Excel for some pretty tables/charts? 📊
  • Additional useful apps
  • Integrate GHAS with other Security Information and Events Management (SIEM) tools
  • What to do when you find Secret Scanning results
    • Turn on push protections! This won’t block all secrets, but it will block secret types with a high confidence score to minimize disruptive false positives
    • It is easier/more secure to rotate secrets than to clean the repo history with something like BFG to remove the commit
    • You still might want to clean the repo history for reasons, but regardless, you should still rotate the secret! And note in the README when the history was re-written for audit purposes.
    • For additional coverage, create custom secret scanning patterns based on your use cases. These can also be created with push protections enabled. Here’s a repo for some predefined patterns!

Further Reading

GitHub, Advanced Security
GitHub GitHub Advanced Security Dependabot CodeQL Secret Scanning
This post is licensed under CC BY 4.0 by the author.