Enhancing GitHub Migrations with Additional Tooling
A collection of additional tools to assist with your GitHub migration experience
Overview
When migrating to GitHub, there are a couple of options. You could do a git clone --mirror / git push, but if you’re migrating from another GitHub source, Azure DevOps, or Bitbucket Server, you should be using the official GitHub Enterprise Importer (GEI) tooling. This not only migrates the code with the history, but also additional metadata such as pull requests, issues (if GitHub), existing branch protection settings, and more.
GEI is great, but not everything is migrated. Don’t worry; there are some additional tools that can help enhance the migration experience and fill in these gaps. This post contains a collection of open-source tools that I’ve found useful when migrating to GitHub.
Supplementary Migration Tooling Collection
Based on my migration experience, here are additional tools I’ve found useful, organized by items not migrated by GEI and the corresponding supplementary tooling:
| Item | Tooling | Notes |
|---|---|---|
| Organization | ||
| - Metadata | N/a | Name of org, description, settings, OAuth app policy, scheduled reminders, org owners, etc. |
| - Custom repo roles | Analysis script | Any custom org roles will need to be migrated as well |
| - Org level webhooks | Analysis script (count), Analaysis script (detailed), gh-organization-webhooks | Need to know what webhook secrets are, can’t retrieve in UI/API |
| - IP allow list | Get org IP allow list, Get enterprise IP allow list, Set IP allow list rules for | The get scripts save rules to CSV and the set script sets them in target |
| Discussions | Analysis script (count) for each org, Analysis script (count) for each repo in an org | Discussions exist in repos, but may have to configure which repo will be used for org discussions |
| Projects | ||
| - Projects v2 | Analysis script, gh-migrate-project | CLI utility can help migrate org-level projects |
| - Org Projects (classic) | Analysis script | Deprecated |
| - Repo Projects (classic) | N/a | Deprecated |
| GitHub apps | Analysis script for org apps, Analysis script by org app count | The manifest flow help when recreating apps manually |
| Teams / membership | gh-migrate-teams, gh-migrate-team-permission, Recreate security in repos & teams, Create teams from list, gh-collaborators | Use the recreation script if wanting to mirror teams/membership |
| User settings | N/a | PATs, SSH keys, notification settings |
| Webhook secrets | Script to analyze webhooks, gh-migrate-webhook-secrets CLI | Repo-level webhooks migrate, but webhook secrets need to be recreated |
| Actions | Action runs don’t migrate, workflows will migrate with code, and everything below will need to be recreated | |
| - Repo/org secrets | gh-secrets-migrator, gh-seva | Actions secrets values can only be retrieved during Actions runtime |
| - Environments | gh-environments | Environments need to be recreated |
| - Variables | gh-seva | Variables need to be recreated |
| - Self-hosted runners | Analysis script for all org runners, Analysis script for all repo runners in org, Analysis script for repo+org runners in org, Analysis script for enterprise runners | Runners need to be re-created |
| - Larger runners | N/a | Larger GitHub-hosted runners need to be re-created; no API for large runners |
| Rulesets | gh-migrate-rulesets | Rulesets are not migrated |
| Packages | See package migration posts | Packages are not migrated |
| Code owners | Analysis script, Code owners mapping helper | Updating org/team names |
| LFS | Migrate LFS artifacts | LFS is not migrated |
| Username mapping | Getting SAML entities at enterprise, Getting SAML entities at org | Getting SAML identities can help map personal github.com accounts by tying their email to their identity provider credential |
| Repository visibility | gh-repo-visibility | Repos migrate as private by default |
| Deploy keys | gh-migrate-deploy-keys | Deploy keys are not migrated |
| Submodules | N/a | URLs (org/repo name) may need to be updated |
Migration Planning Tooling
These tools are more for helping you plan / track a migration. For example, you can use some of these tools to potentially identify problem repositories before you start the migration (e.g. with git-sizer, repositories that contain large files committed), or otherwise repositories with a lot of pull requests that will take longer to migrate.
| Tool | Description |
|---|---|
| gh-repo-stats | GitHub CLI extension to pull statistics on repository metadata used in GitHub migrations |
| gh-migration-audit | Audits GitHub repositories to highlight data that cannot be automatically migrated using GitHub’s migration tools |
| gh ado2gh inventory-report | Azure DevOps to GitHub inventory report using the GEI commands |
| git-sizer & gh-sizer | Compute various size metrics for a Git repository, flagging those that might cause problems |
| gh-bbs-analyzer | GitHub CLI extension for analyzing BitBucket Server to get migration statistics |
| gh-gitlab-stats | GitHub CLI extension to pull statistics on GitLab repository and server metadata |
| gh-pma | Post-Migration Audit (PMA) Extension For GitHub CLI |
| github-migration-monitor | Monitors GitHub Enterprise Importer (GEI) migrations for an organization through a simple command line tool |
Non-technical Migration Planning Tips
Here are list of miscellaneous non-technical considerations to keep in mind when planning a migration:
- You should have training material in a wiki or similar for developers to reference. This should include a list of “what’s changing” and what things developers need to do in the new environment. Things to consider:
- Either adding a new remote to a local repository or cloning a new repository from the EMU
- Recreating PATs and SSH keys (note that SSH keys have to be globally unique in github.com; if they want to re-use an existing key, they will have to remove it from the old account first)
- For PATs and SSH keys, note that users may have to authorize tokens for SSO after creating them, which may be a new step for them
- Creating a migration plan, documenting what repositories are moving and when, as well as who is responsible for each pre and post-migration step
- Running through a dry run first, which helps you identify if there will be any issues ahead of time, as well as getting a sense of the timing
- Dry run means running a migration, but don’t cut over or lock the source (you should be able to run the migration again and again with the same repositories and have the same results)
- If you are migrating from github.com to another enterprise in github.com, the new organization name must be globally unique - however, you can rename the original organization and then right after, rename the new organization to the original name
- Make sure you don’t delete any organization where you want to keep the name, as the name will be locked for 90 days after the deletion
- It is better to rename and then delete in this case
- Links
- If org name or repo name changes, existing links will be broken
- Private/non-public GitHub Pages will have a new URL, so any links to the old pages will be broken
Special Thanks
Special thanks to everyone who has contributed to these OSS tools!
- katiem0
- tspascoal
- timrogers
- mickeygousset
- dylan-smith
- amenocal
- andyfeller
- antgrutta
- bryantson
- pmartindev
- samueljmello
- And many more I am SURE I am missing
Summary
Hopefully, this collection of migration tooling and tips helps you out! I’ll use this post as a living document and update it as I find new tools or tips. If you have any suggestions or additions, please add a comment! 🙌 ✨