github down take control of your git repository

in Development

How To Migrate From Github to a Self Hosted Repository

Last week’s Github outage caused quite a stir. Due to the distributed nature of Git, there was a lot of discussion about whether or not this was even a problem. After all, we can all still work just fine from our cloned repository. This is of course why we all love Git, but if you were one of the thousands of developers or companies who deploy their code from Github, you were likely affected by this outage. Your deployment process didn’t have to just include your own code repository either. If your deployment process pulled NPM packages, or any kind of packages that rely on Github, your launch was dead in the water. Did this incident cause you to re-think your deployment process? Or even your code management strategy in general? If your launch was delayed by this outage, it might be a good idea to at least consider alternative Git hosting options. In this post I’d like to show you how you can turn your Github repository into a self-hosted repository under your own control, and cover ways you can avoid the dreaded package install timeout during a Github outage.

Github != Git

This may not need mentioning, because we all know this by now right? In my early Git learning days, many people would describe Git using the word Github. I always felt obligated to correct them, by politely telling them that Github is a service that serves Git repositories. It’s easy to understand why this correlation was made though. Github catapulted Git into the tech mainstream, and many a developer is thankful for this.

Self-Hosting a Git Repository

There are two methods of self hosting a Git repository that I’d like to cover. Hosting your repository on a server with SSH, and hosting your repository with Gitlab. For this tutorial you’ll need an Ubuntu 14.04 server or desktop. Let’s begin.

Setup Your SSH Keys

Before we can move on, we’ll need to make sure we have SSH keys for the user we’ll be testing with on our Ubuntu machine. This is a quick process.

ssh-keygen -t dsa

You should see:

Generating public/private dsa key pair.
Enter file in which to save the key (/home/me/.ssh/id_dsa): (HIT ENTER)
Created directory '/home/me/.ssh'.
Enter passphrase (empty for no passphrase): (HIT ENTER)
Enter same passphrase again: (HIT ENTER)
Your identification has been saved in /home/me/.ssh/id_dsa.
Your public key has been saved in /home/me/.ssh/

SSH Git Hosting

This is the easiest remote Git repo you can setup. On your Ubuntu 14.04 machine do the following.

First clone your repo:

cd /home/me
git clone --mirror gitrepo.git

**You should do the following steps as root, it makes the following easier:

Become root user:

sudo -s

Run these:

apt-get install git
useradd -d /git -s /usr/bin/git-shell git
mkdir -p /git/git-shell-commands
mkdir -p /git/.ssh/

Now let’s add your key to the git user’s authorized_keys file:

cat ~me/.ssh/ > /git/.ssh/authorized_keys
chmod 600 /git/.ssh/authorized_keys
chown git.git /git/.ssh/authorized_keys

Now let’s move your cloned repository into the git user’s home:

mv ~me/gitrepo.git /git/

Now that you have your mirror clone of your repo in /git/, let’s set the perms:

chown -R git.git /git
chmod -R g=,o= /git

Now let’s exit sudo and go back to our user and test out our new repo’s location:

cd /home/me
git clone git@localhost:/git/gitrepo.git

You should see a successful clone of your Git repository that you can now work out of. Congrats, you migrated your repo the easy way! This is obviously an incredibly basic form of a remote Git repository, but there are ways you can flesh this out a little more. If you have a small team and all members are trusted, simply adding their SSH keys to the /git/.ssh/authorized_keys will allow everyone to pull and commit to it. If you want a little more control, adding users to the system as well as to the git user’s group, and setting read-write group permissions will accomplish this.


The above method is really to illustrate just how easy it is to setup a remote Git repo. While this is great for small teams, most developers or companies will want features much like Github. For that, we’ll use Gitlab(you can find most of the following here).

***If you are going to run this part of the tutorial on the same machine you did the above tutorial on, you will need to remove the git user:

userdel -r /git

And just to be sure the home is gone:

rm -fr /git

Ok, now that the machine is ready, let’s install the dependencies:

sudo apt-get install curl openssh-server ca-certificates postfix

This will popup a Postfix configuration screen. For the purposes of this tutorial, choose “Local only” and set your System mail name:


Gitlab Postfix System Mail Name

Now let’s install Gitlab:

curl -sS | sudo bash
sudo apt-get update
sudo apt-get install gitlab-ce

(this step takes a bit)

Alright, once it’s installed, let’s configure it:

sudo gitlab-ctl reconfigure

(This also may take some time. Make sure nothing else is running on port 80)

At this point if all of the steps above worked successfully, you should be able to browse to the hostname from a different machine, or if you are using the desktop version of ubuntu 14.04, to localhost.

From your Ubuntu Desktop:


or From a different machine:


You should get a login screen that looks like this:

gitlab login screen

You can use these default credentials:

user: root
pass: 5iveL!fe

Follow instructions to change password:

gitlab change password screen

Log back in and create a new repository…we’ll call it “gitrepo”:

gitlab create new project screen

If successful, you should see this:

gitlab create project success screen

From that page click the “Add an SSH key to your profile” button. Add the SSH key we created above:

gitlab add ssh key screen


gitlab add ssh key success

Now clone your Github repo:

cd /home/me
git clone --mirror gitrepo.git

Now let’s import our Github repo into the Gitlab installation we just created:

cd gitrepo.git
git remote add gitlab ssh://git@localhost:22/root/gitrepo.git
git push -f --tags gitlab refs/heads/*:refs/heads/*

Now you can delete the mirrored clone:

cd /home/me
rm -fr gitrepo.git

And then clone your new Gitlab repo for normal use:

git clone git@localhost:root/gitrepo.git

And that’s it! You’ve just setup your own private Git repository with functionality similar to Github. An important point I didn’t cover in this install of Gitlab is that for production deployment, you will need to make sure you serve your Gitlab over https. A good doc to help you accomplish this can be found here. In addition, you can find a lot of useful information about how to manage your Gitlab installation, here.

NPM or Other Package Managers That Rely on Github

So now that we’re hosting our own Git repository, what do we do about our package deployments that rely on Github. There are varying ways to deal with this issue, and none of them are perfect in my opinion. Using NPM as an example, we can:

  • Commit the installed packages to our repository. This allows us the ability to deploy our applications without Github, but the downside is that our commit logs can get noisy, and our repo will grow at a faster rate as we continue to update our modules.
  • Along those same lines, we can create tarballs with our installed modules, and also commit those to our repo. This would solve the noisy commit log problem, but our repo will grow even faster.
  • The solution I like to use, is to create tarballs of our installed modules, but commit them to an entirely different repository. This way, changes in packages are managed separately, but still managed in a repository. This is by no means a perfect solution, but it has worked for us.
  • I would love to hear how others solve this problem, please leave comments.
  • Is Self Hosting Our Git Repository Worth All The Trouble?

    As we’ve seen, it’s not a major undertaking to host our own Git repositories, but is it really worth the trouble? I would argue that it is. I like to be the one responsible and accountable for the management of my Git repositories. While it is extra work, it is worth it to be in control of how my repositories are managed.

    • BoTuLoX

      There’s also Gogs. Just run the docker image and it’s more or less good to go.

      • Eric

        You know, just before I published this article, I ran into Gogs and it looks very interesting. I will check it out for sure.

    • Gautham Thyagarajan

      I’d just mitigate a potential GitHub outage by keeping my opensource projects synchronized between a GitHub, GitLab hosted and BitBucket. Since Git lets you add multiple remotes and push changes to them this is pretty easy.

      Honestly speaking, setting up my own Git server for production is a bit of a pain in the posterior because a vanilla single-server GitLab instance is as likely to go down (or probably more) as GitHub, especially if you are hosted in a budget SSD cloud like DigitalOcean, Linode, Vultr, RamNode, ArubaCloud, etc which are very much DIY.

      I’ll need atleast two servers for redundancy. The servers will need some load balancing and synchronization setup between them. I’ll need monitoring to see if one went down. Then I’ll need to manually write code that does autoscaling to handle sudden spikes in traffic.

      All of this sounds a lot for an opensource project.

      Things would be a lot different for a commercial project though – a single tiny server is good enough for most small teams so a couple of GitLab instances kept synchronized would cost as little as 2€

      • Eric

        These are fair points, and I agree, this is a lot for an opensource project. Why do all this if someone else is doing it for you, and for FREE?? It’s definitely not for everyone. What I hoped to accomplish with this was to present the options we all have available to us. I like your idea of pushing to multiple Git hosting services for opensource projects. That is another great option. Thank you for the feedback!