Publ: Continuous deployment with git

Last updated:

This is the approach I use for managing my site content on my main website. It requires that you can run shell scripts from your git repositories, and ideally your git repository lives on the same server as your actual Publ installation.

Regardless of how you manage your deployment trigger, you will need a deployment script, which does a git pull and updates package versions if the Pipfile.lock has changed. Save this file as deploy.sh in your website repository, and make sure it’s set executable:

#!/bin/sh
# deploy.sh -- wrapper script to pull the latest site content and redeploy

cd  $(dirname $0)
git pull --ff-only || exit 1

if git diff --name-only HEAD@{1} | grep -q Pipfile.lock ; then
    echo "Pipfile.lock changed; redeploying"
    pipenv install || exit 1
fi

if [ "$1" != "nokill" ]; then
    echo "Restarting web services"
    killall -HUP gunicorn
fi

The remainder of the deployment process depends on how you’re actually hosting your git repository.

Self-hosted git repository

Repository on same server as the website

On your webserver, create a private bare git repository wherever you want it; for example, if you have your deployed website in $HOME/example.com, this will create a bare repository in $HOME/sitefiles/example.com.git:

mkdir -p $HOME/sitefiles
cd $HOME/sitefiles
git clone --bare $HOME/example.com example.com.git

Now you’ll have a bare repository in sitefiles/example.com.git and an application directory in example.com.

Back wherever you’re actually developing your website, add the new bare repository as a remote, for example:

git remote add publish username@servername:sitefiles/example.com.git

Finally, add a post-update hook to the bare repository, e.g. $HOME/sitefiles/example.com.git/hooks/post-update:

#!/bin/sh

echo "Deploying new site content..."

cd $HOME/example.com
unset GIT_DIR
./deploy.sh

Separate servers using an ssh key

If you keep your git repository on a separate server from where it’s deployed to, set up an ssh key or other authentication mechanism other than password so that you can do passwordless ssh from the repository server to the deployment server, and then add this as a post-update hook on the repository:

#!/bin/sh

echo "Deploying new site content..."

ssh DEPLOYMENT_SERVER 'cd example.com && ./deploy.sh'

replacing DEPLOYMENT_SERVER with the actual server name, and example.com with the directory that contains the site deployment.

Simple webhook deployment

If you don’t have the ability to run arbitrary post-update hooks but do have some sort of webhook functionality, you can add a webhook to your Publ site to run deploy.sh; for example, you can add this to your app.py:

@app.route('/_deploy', methods=['POST'])
def deploy():
    import threading
    import flask
    import subprocess

    if flask.request.form.get('secret') != os.environ.get('REDEPLOY_SECRET'):
        return flask.abort(403)

    try:
        result = subprocess.check_output(
            ['./deploy.sh', 'nokill'],
            stderr=subprocess.STDOUT)
    except subprocess.CalledProcessError as err:
        logging.error("Deployment failed: %s", err.output)
        return flask.Response(err.output, status_code=500, mimetype='text/plain')

    def restart_server(pid):
        logging.info("Restarting")
        os.kill(pid, signal.SIGHUP)

    logging.info("Restarting server in 3 seconds...")
    threading.Timer(3, restart_server, args=[os.getpid()]).start()

    return flask.Response(result, mimetype='text/plain')

Then, in whatever mechanism you use to run the website, set the environment variable REDEPLOY_SECRET to some secret string. For example, if you’re using a systemd service, add a line like:

Environment="REDEPLOY_SECRET=the secret password"

Deploy these changes to your website and restart it. Now you can configure a webhook on your git repository that sends a POST request to the /_deploy route with the secret parameter set to your REDEPLOY_SECRET key.

GitHub-style web hooks

If you’re using GitHub (or something GitHub-compatible) to host your site files, there is a more secure way to run a webhook.

First, install the flask-hookserver package with pipenv install flask-hookserver.

Next, add the following to your app.py somewhere after the app object gets created:

from flask_hookserver import Hooks

app.config['GITHUB_WEBHOOKS_KEY'] = os.environ.get('GITHUB_SECRET')
app.config['VALIDATE_IP'] = False

hooks=Hooks(app, url='/_gh')

@hooks.hook('push')
def deploy(data, delivery):
    import threading
    import subprocess
    import flask

    try:
        result = subprocess.check_output(
            ['./deploy.sh', 'nokill'],
            stderr=subprocess.STDOUT)
    except subprocess.CalledProcessError as err:
        logging.error("Deployment failed: %s", err.output)
        return flask.Response(err.output, status_code=500, mimetype='text/plain')

    def restart_server(pid):
        logging.info("Restarting")
        os.kill(pid, signal.SIGHUP)

    logging.info("Restarting server in 3 seconds...")
    threading.Timer(3, restart_server, args=[os.getpid()]).start()

    return flask.Response(result, mimetype='text/plain')

Now, set up your deployment to have an environment variable called GITHUB_SECRET set to some random, unguessable string. Do a manual redeployment.

Finally, go to your GitHub repository settings, then “Webhooks,” then “Add webhook.” On the new webhook, set your payload URL to your deployment hook (e.g. http://example.com/_gh), the content type to application/x-www-form-urlencoded, and the secret to the value of your GITHUB_SECRET string.