Post

So You Want to Migrate Trac Tickets to GitHub Issues

Posted Updated
By Josh Johanning 8 min read

Disclaimer

I should first off state that I wouldn’t entirely recommend this, if you are migrating to GitHub for the first time, you should try to start off with a blank slate. Keep the old tickets in Trac or the database around for a period in time in case you need to reference, but don’t migrate the entire Trac ticket repository. I could understand wanting to port in active tickets, though, which is a very valid use case for a tool like I am going to demonstrate.

We were working with a client who was migrating off of their old Trac server, and I wanted to document (mostly for myself, but for anyone else who finds this too) how exactly it worked.

Tools

There are plenty of tools out there on GitHub (svigerske/trac-to-github, robertoschwald/migrate-trac-issues-to-github, hershwg/github-migrate-trac-tickets… some of them require XML-RPC, which I had a heck of a time installing on my Apache Trac webserver, so I wasn’t able to test those.

The two I have tested are:

Both of these are more focused on Issues, and neither really do attachments (see trac-hub how to section below for a possible solution though). The svigerske/trac-to-github tool mentions that it uploads attachments as gists.

Note: These tested against GitHub’s Cloud instance (not server).

trac-hub

trac-hub Overview

I originally tested out trac2github and was going to write about that, but I think I might like this trac-hub tool a little better. When comparing the two, trac-hub might be better in that it uses the Import Issues API. The issues import API never left preview, so be aware it’s not officially supported from GitHub. The benefits of using this API are that it can create issues:

  • without hitting abuse detection warnings and getting blocked
  • without sending email notifications
  • without increasing your contribution count to ridiculous heights
  • much faster than with the normal issues API
  • with correct creation/closed date set
  • atomically without users being able to interfere in the creation of a single issue

Basically, using this tool allows the GitHub issue to look like it was created originally when it was created in Trac versus looking like a brand new issue that was created.

I created a fork of mavam/trac-hub (joshjohanning/trac-hub) (that has since been merged) that adds the ability to map Trac ticket owners to GitHub Issues assignees. Make sure not to typo the GitHub username as the issue will fail to create if a ticket’s owner (assignee) has a mapping in the configuration file. If a mapping doesn’t exist, the GitHub Issue will be created with no assignee, as expected.

The caveats of both of the tools is that the “Issue creator” will appear as the one who originally ran the tool - but at least with this tool, we can preserve create and comment dates.

trac-hub How To

I ran this on a debian 10.7 server, the same server that was running my Trac installation.

Pre-requisites to install:

  • sudo apt-get install bundler
  • sudo apt-get install libmariadb-dev
  • sudo apt-get install libsqlite3-dev

Instructions:

  1. Clone the repository - git clone https://github.com/joshjohanning/trac-hub
  2. Rename the example configuration file - mv config.yaml.example config.yaml
  3. Edit the configuration file vim config.yaml and modify the following sections:
    1. trac to provide the sqlite database path or mysql connection (untested)
    2. github to provide the target org/repo name and personal access token (make sure to grant the token enough access!)
    3. users to provide a list of mapping of Trac users –> GitHub. If a mapping doesn’t exist, it won’t use the GitHub handle and just refer to the user as the Trac user. In my version of the tool, if a mapping is found but the GitHub handle doesn’t exist, it won’t migrate tickets –> issues that are owned by/assigned to that non-existant user. Essentially, just make sure that the mapping uses a valid GitHub user or don’t include it :)
    4. labels to provide a mapping of Trac ticket metadata to labels in GitHub Issues
  4. Run the migration in a test repo! sudo bundle exec trac-hub -v -s 1 -F. Command line options:
    1. -v : verbose/debug logging
    2. -s 1 : start at the the first ticket in Trac
    3. -F : fast import, import without safety-checking the issue number. The way this tool runs is it expects Trac ticket #1 to be created as Issue #1. If you already have issues in the repository, or you want to do testing, add the -F import. For your real migration, you could probably drop this and trac ticket #1 will map to issue #1. Otherwise, with this -F argument, the issues will be created even though the ID’s won’t make 1:1.
    4. No example here, but -a for attachment-urls is interesting. The tool doesn’t migrate attachments, but if you used one of the download-trac-attachment-*.sh scripts in the repo, you could host the files somewhere, presumably with the same file names and it will hyperlink the attachments??

trac-hub Command Line Argument List

1
2
3
4
5
6
7
8
9

$ bundle exec trac-hub --help
    -c, --config config              set the configuration file
    -s, --start-at ID                start migration from ticket with number <ID>
    -r, --rev-map-file FILE          allows to specify a commit revision mapping FILE
    -a, --attachment-url URL         if attachment files are reachable via a URL we reference this here
    -S, --single-post                Put all issue comments in the first message.
    -F, --fast-import                Import without safety-checking issue numbers.
    -o, --opened-only                Skips the import of closed tickets
    -v, --verbose                    verbose mode

trac-hub Example Migration Screenshots

Migrating 5 tickets from Trac: trac-tickets Original ticket list in Trac

How they appear in GitHub: trac-hub-example-issues Issues in GitHub migrated using trac-hub

Trac ticket comments: trac-ticket-comments Original ticket comments in Trac

Issue comments: trac-hub-example-issue-comments Issue comments in GitHub migrated using trac-hub

trac2github

trac2github Overview

This was the first tool that I used, and it works! It uses the regular GitHub Issues API, which is subject to rate limits and abuse detection. The team that tried to use this import with me ran into the abuse detection a few times and tweaked lines 484 and 485 of the php script to play more friendly with GitHub (decreasing $request_count > 50) and increasing sleep(70). Note that these are only used if $ticket_limit is set in the configuration file.

The team did not like this tool because the original create date was not preserved - all of the issues look like they were created at the time the import tool ran.

Otherwise, it does the job at migrating tickets, comments, user mapping with assignee, and label mapping.

trac2github How To

I ran this on a debian 10.7 server, the same server that was running my Trac installation.

Pre-requisites to install:

  • PHP (e.g. on Ubuntu/Debian sudo apt-get install php) (I prefer sudo apt-get install php-fpm to NOT install a new version of apache over my existing web server)
  • Support for the trac database format, e.g. sudo apt-get install php-mysql, sudo apt-get install php-sqlite3, etc.
  • sudo apt-get install php-curl

Instructions:

  1. Clone the repository - git clone https://github.com/joshjohanning/trac-hub
  2. Edit the configuration file vim config.yaml and modify the following variables:
    1. $username : GitHub username b. $password GitHub personal access token (make sure to grant the token enough access!) c. $project : The GitHub organization you are migrating to. If migrating issues to your own GitHub repo under your own account, this would be the same as $username. Use the Organization name if using GitHub Enterprise
    2. $repo : The GitHub repo name
    3. $users_list : Provide a mapping of Trac user –> GitHub user
    4. The database driver settings - whether that’s $mysqlhost_*, $sqlite_trac_path, or $pgsql_ settings.
    5. Explore the other items in the config file to see if they are needed, such as $ticket_offset for resuming a migration or $remap_labels to modify the label mapping. Note that while you might not think you need to set $ticket_limit because you want to migrate the entire Trac ticket database, this setting needs to be set in order to trigger the aforementioned rate limiting sleep control. Therefore, I advise giving $ticket_limit an arbitrary value for this purpose.
  3. Run the import: php trac2github.php

You’ll notice that this import is a lot slower than trac-hub :).

trac2github Example Migration Screenshots

Migrating 5 tickets from Trac: trac-tickets Original ticket list in Trac

How they appear in GitHub: trac-hub-example-issues Issues in GitHub migrated using trac2github

Trac ticket comments: trac-ticket-comments Original ticket comments in Trac

Issue comments: trac-hub-example-issue-comments Issue comments in GitHub migrated using trac2github

Takeaways

Run a few of these migrations, and run them in repositories that you don’t mind deleting afterwards as deleting issues can be…challenging.

Run the migration using someone’s PAT that you don’t mind being the creator for the Issues (throwaway / service account possibly??).

Make sure you have the users created in GitHub and mapped in the appropriate configuration file ahead of time.

Be patience, have reasonable expectations, and good luck!

GitHub, Migrations
GitHub GitHub Issues Migrations
This post is licensed under CC BY 4.0 by the author.