Hell Oh World!

We have been talking about having a patch review system in place for glibc for some time now since the volume of patches has been steadily increasing and we don’t have enough reviewers to go through them in time, leading to missed patches and general contributor unhappiness. Due to the fact that we’re a primarily email driven project, we needed something that fits into our current workflow and patchwork was the obvious choice to start with. I decided to do a setup on my domain first to get a feel of things before I made a request to set up a patchwork instance on sourceware. The instance is live now on patchwork.siddhesh.in for glibc contributors to get a feel of it.

The hard part about patchwork is that the documentation is a myth. There is an INSTALL file that sort of works, except that it doesn’t the moment you decide to use slightly different settings. Additionally, the instructions are targeted at dedicated hosting providers, so they almost completely don’t apply to someone trying to set up patchwork on Dreamhost on their shared hosting account. Of course, figuring out what to do with your patchwork installation once it is done is an adventure as well, since there seems to be no user documentation at all. Anyway, here’s how I did it:

Setting up the server and getting sources

I assume you have a shell account for the user that would be administering the subdomain, since you’d be doing a fair bit of sysadminy stuff on it. Also, it’s assumed that you’re hosted on a Linux based server; I don’t really care about how it works for Windows based hosting.

Create your Dreamhost subdomain using the control panel and make sure you have Passenger support enabled. The Passenger support creates a directory called public in your subdomain directory. Don’t bother setting up django at this stage.

Now get patchwork from their git repo:

$ git clone git://ozlabs.org/home/jk/git/patchwork

and copy the contents (i.e. whatever is inside patchwork, not the directory itself) into your subdomain directory. The contents of your subdomain directory would then be something like this:

$ ls
apps  docs  htdocs  lib  public  templates  tools

Now remove the public directory and create a symlink from htdocs.

$ ln -s htdocs public

We could technically just copy things over from htdocs to public, but it’s easier to update this way.

Next, we need django. patchwork needs django 1.5.x, so if your server doesn’t have it, you’ll need to download the sources yourself. To check the installed django version:

$ python
Python 2.6.6 (r266:84292, Dec 26 2010, 22:31:48) 
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
>>> django.VERSION
(1, 2, 3, 'final', 0)
>>> 

Since we don’t have 1.5.x, we install django from sources in the lib/packages in your subdomain directory:

git clone https://github.com/django/django.git -b stable/1.5.x

Create a directory lib/python in your subdomain directory and symlink the django installation in it:

$ ln -s ../packages/django/django ./django

Configuring django and patchwork sources

The first thing to configure is the database. From your dreamhost control panel, create a mysql database and user. Have that information handy to put in your django/patchwork configuration.

The default settings for patchwork (and django) are in apps/settings.py. We need to override those by creating our own file called apps/local_settings.py. The first thing to go in our local_settings.py is our database configuration:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'db_name',
        'USER': 'db_username',
        'PASSWORD': 'super_secret_password',
        'HOST': 'mysql.myhost.name',
        'PORT': ''
    },
}

The instructions in the patchwork documentation mention using DATABASE_* style variables, but they didn’t work for me, probably because I figured out that I had an older version of django after I was done with the initial configuration.

Next you need set the following variables in local_settings.py:

SECRET_KEY = 'a random generated long string'

ADMINS = (
     ('Super Admin', 'super@foo.com'),
)

TIME_ZONE = 'Asia/Kolkata'
LANGUAGE_CODE = 'en-us'
DEFAULT_FROM_EMAIL = 'Patchwork (foo.com) '
NOTIFICATION_FROM_EMAIL = DEFAULT_FROM_EMAIL

# If you change the ROOT_DIR setting in your local_settings.py, you'll need to
# re-define the variables that use this (MEDIA_ROOT and TEMPLATE_DIRS) too.
ROOT_DIR = '/path/to/patchwork.foo.com'
TEMPLATE_DIRS = (
    # Put strings here, like "/home/html/django_templates" or "C:/www/django/templates".
    # Always use forward slashes, even on Windows.
    # Don't forget to use absolute paths, not relative paths.
    os.path.join(ROOT_DIR, 'templates')
)
# Absolute path to the directory that holds media.
# Example: "/home/media/media.lawrence.com/"
MEDIA_ROOT = os.path.join(
    ROOT_DIR, 'lib', 'python', 'django', 'contrib', 'admin', 'media')

The SECRET_KEY can be generated using the following python snippet:

import string, random
chars = string.letters + string.digits + string.punctuation
print repr("".join([random.choice(chars) for i in range(0,50)]))

Other options are obvious from their names and values, so adjust them to your taste. ROOT_DIR is set to the directory where patchwork is, i.e. your subdomain directory. TEMPLATE_DIRS and MEDIA_ROOT are derived from ROOT_DIR, so I’ve just copied those over from settings.py.

Next up, we need to get static files for admin sessions into a place where django can find and serve it. They’re present in contrib/admin/static in your django installation and we need to copy them over to htdocs/static. Once this is done, we need to tell django that it can find the static files by adding the following configuration snippet to local_settings.py:

PROJECT_ROOT = os.path.normpath(os.path.dirname(__file__))
STATIC_ROOT = os.path.join(PROJECT_ROOT, 'static')
STATIC_URL='/static/'
ADMIN_MEDIA_PREFIX='/static/admin/'

INSTALLED_APPS = (
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.admin',
    'django.contrib.staticfiles',
    'patchwork',
)

Finally, we don’t want to spew out debugging messages to the server and we want to be able to debug problems at the same time, so we need logging support to be enabled. Add the following snippet to local_settings.py:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'file': {
            'level': 'DEBUG',
            'class': 'logging.FileHandler',
            'filename': '/path/to/patchwork.foo.com/django-debug.log',
        },
    },
    'loggers': {
        'django.request': {
            'handlers': ['file'],
            'level': 'DEBUG',
            'propagate': True,
        },
    },
}

making sure that the path in the ‘filename’ is writable by django. We can now disable debugging, so add this:

DEBUG=False

Now here’s the fun part, disabling debugging changes the behaviour of django, in that it suddenly starts doing extra checks, due to which you’ll start seeing failures like below:

SuspiciousOperation: Invalid HTTP_HOST header (you may need to set ALLOWED_HOSTS)

so fix this by adding the following:

ALLOWED_HOSTS = ['your.subdomain']

These are all the settings you would normally need to get patchwork up. Now we run manage.py from the apps directory (note that the instructions in INSTALL are slightly wrong here):

$ PYTHONPATH=../lib/python ./manage.py syncdb

This should initialize the database for patchwork and django. Follow whatever prompts that come up till you come back to the shell. If the command throws errors, read up and fix your configuration. The django documentation is surprisingly good once you get used to the layout, so don’t despair if patchwork documentation doesn’t help (it won’t).

Getting patchwork up

With the database set up and the sources in place, one needs to tell apache how to serve content through django. We had set up passenger for precisely this, so we just need to add a python script in our subdomain directory to tell passenger what to do. The script should be named passenger_wsgi.py and it’s contents should be:

import sys, os
basedir = os.getcwd()
sys.path.insert(0, os.path.join(basedir, 'lib/python'))
sys.path.append(basedir)
sys.path.append(os.path.join(basedir, 'apps'))
os.environ['DJANGO_SETTINGS_MODULE'] = "apps.settings"
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

This tells passenger where to find the app and what paths to use to find our custom django and any additional libraries and also our settings. At this point, browsing to your site should get you to a working patchwork installation. Login to the admin page (which should be http://patchwork.foo.com/admin/) with the superuser name and password you had created while running manage.py.

The first thing you’ll need to do is create a Site. Make sure you delete the example.com there since it will otherwise be used first and you won’t like it. After the site change, if you start getting a server error, then look at your django log and look for this error:

DoesNotExist: Site matching query does not exist.

This likely means that there was some data that referred to example.com and broke. You’ll have to edit the row for your subdomain in the django_sites table and change the id to 1.

Next, you add a project of your choice. It took me a while to understand what the names of the fields in the new project form meant, but maybe that was because it was getting very late. Anyway, Linkname is any name you wish to give to the project, which appears in the URL – I used glibc here. Name is a free form name for the project – I used GNU C Library. Listid is the List-ID header in the mailing list email that it should look for. Listemail is the email address for the mailing list. I just ignored the rest of the fields.

Getting emails from the project

Patchwork is useless without emails, so subscribe to the mailing list whose patches you want to monitor. You obviously need an email address for it, so create an email address first. There is usually a confirmation step involved in mailing list subscription, which should also be completed.

Next step is to forward emails from this account to the shell account where the site is running. To do this, dreamhost has a wiki document that describes how to link an email address to a shell account email. Follow that and send test emails to make sure that emails are reaching the shell user.

Now we need a mechanism to forward the mails received by the shell user to the parsing script in patchwork. This is easy to set with a forwarder file for postfix, called .forward.postfix in your home directory with the following contents:

"|/path/to/subdomain/apps/patchwork/bin/parsemail.sh"

Now you should start seeing patches in your project view on the website.

Sending emails from patchwork

This bit is fairly straightforward – patchwork needs to be able to send emails via SMTP for various events or for registration confirmation. Django needs the following information in local_settings.py to be able to send these emails, or else it uses the local SMTP, which won’t work on Dreamhost:

EMAIL_HOST = 'mail.patchwork.foo.com'
EMAIL_HOST_USER = 'from@patchwork.foo.com'
EMAIL_HOST_PASSWORD = 'supersecret'
EMAIL_USE_TLS = True

Getting started with patch reviews

When browsing, you’ll quickly figure out that you can’t really do anything with the patches you see there, unless you have sent a patch, i.e. your email address configured in patchwork is the same as the email address of the patch sender. This is not really very useful as far as peer reviews are concerned and working through the UI doesn’t tell you anything. One would obviously gravitate towards the admin UI to see if there are any settings there in the Users or Groups or Projects or People sections, but there are none.

The option is effectively hidden in plain sight in the User Profiles section. Click through to your user (or any user you're interested in updating) and come to the page that shows the projects that you're part of (Primary Projects) and projects that you're maintainer of (Maintainer Projects). The list of projects that show up next to Maintainer Projects aren't projects you're maintainer of, unless they're selected! So select the project(s) you are maintainer of and save your changes. Now when you login as that user, you'll see options to change patch review state and even delegate to other reviewers.

Final thoughts

It was quite an exhausting experience to get patchwork working and it wasn't just because it was on dreamhost. The project is fairly poorly documented and the usual clumsiness associated with web apps didn't help things. My relative inexperience with django may have compounded the problems I had getting this up, but I would again blame that on clumsy webapp syndrome.

I have written this down with the hope that someone else looking to do this would be able to get patchwork up in a bit less time than I took, and also because we may have to do it again on sourceware when there is consensus on using it. Given my wonderful memory, I'll probably end up making all the mistakes once again when I try it out the next time.