Because of the rise in badly-behaved website scraping bots, it can be very helpful for site performance to have a fronting CDN, such as Cloudflare, in front of your origin server. But there are a few things you need to keep in mind.

Caching

When configuring caching, it is very important that you exclude the following paths from any aggressive caching behavior:

• /_cb/
• /_login and /_login/
• /_logout and /_logout/

If these paths get cached, it may lead to unpredictable behavior for user login and logout.

Image CDN

There should be no need for any additional fronting image CDN on Publ, although having it enabled shouldn’t hurt. However, because Publ manages its own cache lifetime and the CDN is not aware of this, some strange behaviors may occur in extreme edge cases.

Setting the CDN to aggressively prefetch images or to use status 103 “Early Hints,” if possible, is probably a good idea.

For smaller Publ sites, the suggested configuration is to use SQLite for the index database, as it is low-maintenance and performs very well due to being in-process and taking advantage of the operating system’s disk cache.

However, sometimes it’s helpful to use a larger-scale database system for the deployment, primarily to gain access to finer-grained locking. This is especially useful in situations where there are many thousands of entries and a desire to keep the site running at full capacity during a reindex.

Here is a configuration snippet that allows you to use MySQL/MariaDB on your Publ site:

# ...

if 'DATABASE_URL' in os.environ:
    import urllib.parse
    parsed = urllib.parse.urlparse(os.environ['DATABASE_URL'])
    user = re.match(r'(.*):(.*)@(.*)', parsed.netloc)
    db_config = {
            'provider': parsed.scheme,
            'user': user.group(1),
            'password': user.group(2),
            'host': user.group(3),
            'database': parsed.path[1:],
            # charset and collation must be specified, as the MySQL defaults do
            # not properly support emoji and other 4-byte characters
            'charset': 'utf8mb4',
            'collation': 'utf8mb4_bin',
        }
else:
    db_config = {'provider': 'sqlite', 'filename': os.path.join(APP_PATH, 'index.db')}

config = {
    'database_config': db_config,
    # ...

Then when running the site, set an environment variable such as:

DATABASE_URL='mysql://username:password@server/dbname'

If migrating from SQLite to MySQL, it is a good idea to create the index first before flipping the configuration:

DATABASE_URL='mysql://username:password@server/dbname' poetry run flask publ reindex

An alternate way to store the database configuration is to put it into a local db_config.py file, like so:

db_config = {
    'provider': 'mysql',
    'user': 'db username',
    'password': 'db password',
    'host': 'localhost',
    'database': 'my_site',
    'charset': 'utf8mb4',
    'collation': 'utf8mb4_bin'
}

# ...

try:
    from .db_config import db_config
except ImportError:
    db_config = {'provider': 'sqlite', 'filename': os.path.join(APP_PATH, 'index.db')}

config = {
    'databse_config': db_config,
    # ...
}

but as always it is important to ensure the security of this file; environment-based configuration is traditionally considered to be much easier to secure and prevent mishaps such as accidentally checking it into source control.

When templates start to get particularly complex, Publ can start to slow down; this is one of the tradeoffs when using a dynamic site generator, rather than a static one. Fortunately, it is fairly straightforward to configure Publ with caching; here are some pointers on how.

Basics

Entire textbooks can (and have been) written on caching and their optimum usage. This is not intended to be an entire textbook, but rather a set of brief-ish guidelines for how to configure Publ’s caching mechanism.

What is a cache?

At its most basic and general, a cache is a temporary storage area where intermediate results get stored for a while.

Fundamentally, there are two algorithms involved in a cache: key generation, and result generation.

The key is basically a filename that tells the cache where to look for a result, and this is computed based on the unique aspects that are involved in generating the result. In the case of Publ’s page cache, the key is based on what’s being looked at (the selected category, template, entry, and URL parameters), how it’s being looked at (the URL), and who’s looking at it (the current user, if any).

The result is the result of performing the operation that’s being cached. In this case, this is the rendered content of the template.

The goal in any cache is to correctly produce the right result while reusing those results as much as possible, while taking less time on average than what it would take to simply compute the result each time.

Performance tradeoffs

When setting up a cache, you need to consider what the right balance is between memory usage, performance, and immediacy. These three things are at odds with one another and there is fundamentally no perfect solution that maximizes all three.

Generally-speaking there are two statistics we are looking for in trying to optimize a cache:

• Hit rate: How often a page render can be reused; we want this to be as high as possible
• Staleness: How out-of-date a page render can be; we want this to be as low as tolerable

There is a third statistic which also matters, namely the active cache size (also called the “hot size,” among other things) – this is simply the total amount of storage used by page renders.

To achieve these goals, there are two “knobs” of particular interest: maximum cache size, and expiration time.

Increasing the expiration time will increase the hit rate, while also increasing the staleness. To understand why, if a page is only accessed once every 15 minutes, and the expiration time is only 5 minutes, then the cached result will be expired before the page is accessed again, making the cache useless. But, if the expiration time is one hour, then if a page changes at all, the update won’t appear until an hour after it was rendered.

The other knob, maximum cache size, is simply a measure of how much can be kept in the cache. As long as the expiration time is finite, the cache will never grow to be infinitely large, but you might still want to put a limit on how large it can grow. Raising the cache size won’t necessarily increase the hit rate, but decreasing the cache size to be less than the active cache size will decrease it.¹

A note on staleness

To mitigate the staleness problem, Publ uses aspects of when content was last edited as part of its caching mechanism, so if content changes that might affect a page, all previous versions of that page will be discarded.² So, while many caching systems recommend only setting the expiration time to the maximum amount of time you can tolerate a change not being available, Publ does not have this issue.

However, some things that Publ can do will still become stale. For example, things that are essentially non-deterministic (such as things based on random numbers) or based on the current time (such as entry.date.humanize()) will still be subject to becoming stale. So if you use any of these things in your templates, you’ll want to set your expiration time accordingly. Some guidelines:

• If you use entry.date.humanize(), an expiration time of around 1 hour is probably fine (although this will make a recent entry’s relative time appear to be wrong for its first hour of existence or so)
• If you use random number generation for whatever reason (such as displaying a random image on a particular page), the expiration time should be for however long you are willing to tolerate the number staying the same
• If you want to display the current time on the page for some reason³, the expiration time should be for the most “slow” you’re willing to have the clock be.

Also, if you plan on using any sort of monitoring service (munin, Pingdom, etc.) to measure page load time over time (to get an ongoing performance indicator), it’s a good idea to set your expiration time to a nearby prime number; for example, if you want your expiration time to be around an hour, try values of 3593 or 3659 or the like. This helps to prevent sampling bias and gives a much better overall average.

Configuration

Publ uses flask-caching for its caching layer. The short version of its documentation is it provides caching backends for a number of common caching servers (Redis and a few variants of MemcacheD), as well as file-based and in-process caching.

If you have Redis or MemcacheD available, definitely use those; they have the advantage of persisting beyond your application’s run time, and going into a common shared memory pool for all sites that use them, which can be way more memory-efficient than the in-process cache. They also both support distributed caches.

If you are self-hosting and trying to decide which cache server to go with, I suggest MemcacheD, as it’s by far the easiest to install and configure, and most distributions pre-configure it with sensible defaults.

If you don’t have any of those (for example, you’re on shared hosting that doesn’t allow you to run your own services), your best choices⁴ are FileSystemCache (file-based) and SimpleCache (in-process), which each have strengths and weaknesses. Both of them suffer from not being configurable based on actual storage size; they can only track the number of items in the hot cache. SimpleCache is faster, but also makes your process take more memory – a big concern if you’re on a shared system where memory usage is limited. FileSystemCache is generally friendlier and fast enough (and also will take advantage of the operating system’s file cache which makes it kinda-sorta similar to using a caching server), but its performance disadvantage can end up slowing the site down in many cases. But if you can look at the state of the filesystem, FileSystemCache gives you better visibility into what’s happening with the cache, which can give you better ideas for how to tune it.

In general, I’d recommend this priority order:

1. MemcacheD or Redis
2. FileSystemCache (if you have lots of storage available)
3. SimpleCache (if you have lots of RAM available)

Regardless of caching backend, in your application’s cache configuration you should configure the following:

• CACHE_KEY_PREFIX: Set this to some random but fixed string (this helps to prevent certain kinds of attacks that are possible if you share the cache with someone else)
• CACHE_DEFAULT_TIMEOUT: Set this to your expiration time, as discussed above

Some backend-specific notes are below.

MemcacheD

In order to use MemcacheD you’ll also need to add a MemcacheD client to your Python environment. I generally use python-memcached as it’s easier to install (especially in cross-platform scenarios), but libmc and pylibmc both have some advantages in performance-critical and large-scale scenarios, being slightly faster as well as supporting consistent hashing.

If you’re on Google AppEngine, a suitable client is already provided to you and there is no reason to add your own.

SimpleCache

Set the value of CACHE_THRESHOLD to limit the number of items that can be kept in the cache. As noted above, there is no way to limit the size directly, so if you have a specific allocation limit in mind, try to figure out what the average size of a page is (in bytes), and divide your allocation limit by that. That is the value you should use.

FileSystemCache

Similarly to SimpleCache, this uses CACHE_THRESHOLD to limit the number of items in the cache, rather than the total storage size. Since filesystem space is usually much more plentiful than RAM, there’s no reason to be particularly miserly with this. Unless you start to run out of storage space, anyway

You should also make sure that the CACHE_DIR is set to a directory that you have write access to, and you should set CACHE_OPTIONS to {'mode': 0o600} (the default).

If you know of something that isn’t listed, please open an issue. Note that a listing for a specific provider is not necessarily an endorsement.

Shared hosting

If you can find a compatible shared hosting setup, this is by far the easiest and least-costly way to get running.

Shared hosting providers typically offer very inexpensive bandwidth and storage, with a tradeoff of not having as much flexibility with your environment and service stack. In addition, you’ll be sharing resources with other users, which can have both performance and security implications.

Additionally, shared hosting providers typically limit you to PHP and/or CGI. However, there are a few known shared hosting providers which do support Flask (and therefore Publ) to some extent:

• Nearly Free Speech claims direct support for Flask
• Dreamhost ostensibly supports Flask via their Passenger WSGI wrapper (although I’ve had issues with them in the past) This is no longer the case.

When considering a shared hosting provider, look specifically to see if they support Flask, or a bit more broadly, WSGI. If they support Django (which is another common WSGI-based framework) it is very likely to support Flask.

However, “supporting Python” doesn’t necessarily mean supporting Python WSGI, as sometimes it just means being able to run Python CGI (which is a different, older mechanism which is unsuitable for a modern web stack). If in doubt, ask their support folks if they support Flask.

Once you do find a host, refer to their guides for setting up Flask or WSGI applications to get started with running Publ.

Self-hosting

If you’re savvy with running a Linux server, this is a pretty good choice. The advantages are having complete control over your hosting setup and generally having the cheapest storage available, and a lower overall cost per site if you run multiple sites. The main disadvantage is having to be your own sysadmin, which can be a pretty big burden.

On all of these choices, you’d be running a fronting web server (usually nginx or Apache) and doing a reverse proxy to your Flask application; see the self-hosting guide for more detailed information.

VPS providers

A virtual private server gives you complete control over a virtual slice of hardware. Typically this provides better fault tolerance, easier migration, and simpler server management, with the downside of being slightly lower in performance (in ways which don’t matter for hosting websites most of the time).

Some known providers:

• Linode
• Digital Ocean
• Amazon EC2

Physical hosting

Colocation, either with managed hardware or with bring-your-own device. Not for the faint of heart. Generally gives you more performance but worse bang for your buck, worse fault-tolerance, and much more of a management headache when things go wrong.

For some people this can even mean running a server on your home Internet connection. Many ISPs do not take kindly to this; check your terms of service before doing this.

Cloud app hosting

Cloud app hosting means you provide an image for an application which gets run only when it’s actively being used.

The advantages are that there’s a lot less administrative overhead on your end, and it’s usually easier to deploy your site and also handle scaling and so on.

The disadvantages are numerous, however; usually there’s a much lower limit on how much content you can host within the container (and Publ is not designed to use content kept in an external object store), deployments can be significantly slower, and when the site “sleeps” it can take quite some time for it to serve the first page while it wakes up. Also, storage, CPU, and bandwidth tend to be much more expensive than dedicated hosting.

Some known providers:

• Heroku
• Google App Engine

It’s pretty common to use git to host your website files. There are a few different ways that you can use git itself to automate the deployment of site updates, depending on the specifics of your git hosting situation. The below methods should cover the vast majority of git-based workflows, at least if you’re self-hosting Publ.

Deployment script

Regardless of your actual git hosting situation, you will need a deployment script, which does a git pull and performs different tasks depending on what’s been updated. Save this file as deploy.sh in your website repository, and make sure it’s set executable:

#!/bin/sh
# wrapper script to pull the latest site content and redeploy

cd  $(dirname $0)

# see where in the history we are now
PREV=$(git rev-parse --short HEAD)

git pull --ff-only || exit 1

if git diff --name-only $PREV | grep -qE '^(templates/|app\.py|users\.cfg)' ; then
    echo "Configuration or template change detected"
    disposition=reload-or-restart
fi

if git diff --name-only $PREV | grep -q poetry.lock ; then
    echo "poetry.lock changed"
    poetry install || exit 1
    disposition=restart
fi

if [ "$1" != "nokill" ] && [ ! -z "$disposition" ] ; then
    # insert server restart command here (see note), e.g.
    # systemctl --user $disposition SERVICE-NAME.service
fi

The remainder of the deployment process depends on how you’re actually hosting your git repository.

Self-hosted git repository

Repository on same server as the website

For this example, we will keep the main git repository in $HOME/sitefiles/example.com.git and the deployment directory in $HOME/example.com.

On your webserver, create a private bare git repository wherever you want it; for example, on the server, run:

# change this!
SITENAME=example.com

# create the bare repository
mkdir -p $HOME/sitefiles
git init --bare $HOME/sitefiles/$SITENAME.git

# add the post-update hook
cat > $HOME/sitefiles/$SITENAME.git/hooks/post-update << EOF
#!/bin/sh

echo "Deploying new site content..."

cd \$HOME/$SITENAME
unset GIT_DIR
if [ ! -x ./deploy.sh ] ; then
    echo "Deployment script not available or not executable" 1>&2
    exit 1
fi
./deploy.sh || echo "Deployment script failed"
EOF
chmod u+x $HOME/sitefiles/$SITENAME.git/hooks/post-update

# set up the live/deployment workspace
cd $HOME
git clone $HOME/sitefiles/$SITENAME.git
# optional: make this repository share the raw objects (to save some disk space)
# see https://git.wiki.kernel.org/index.php/Git_FAQ#How_to_share_objects_between_existing_repositories.3F
cd $SITENAME
echo "$HOME/sitefiles/$SITENAME.git/objects/" > .git/objects/info/alternates
git gc

Now you’ll have a bare repository in $HOME/sitefiles/example.com.git and a live deployment workspace in $HOME/example.com, and pushing to the bare repository will run the deployment script in the live workspace, if it exists.

On your local repository (i.e. where you’re actually working on your site) add the bare repository as your remote:

git remote add origin user@server.example.com:sitefiles/example.com.git
git push -u origin master

This should push all of your content into the bare repository; however, since the live workspace won’t yet have deploy.sh you’ll need to do one git pull manually. After this, every push to the bare repository should automatically run your deployment script.

Separate servers using an ssh key

If you keep your git repository on a separate server from where it’s deployed to, set up an ssh key or other authentication mechanism other than password so that you can do passwordless ssh from the repository server to the deployment server, and then add this as a post-update hook on the repository:

#!/bin/sh

echo "Deploying new site content..."

ssh DEPLOYMENT_SERVER 'cd example.com && ./deploy.sh'

replacing DEPLOYMENT_SERVER with the actual server name, and example.com with the directory that contains the site deployment.

Simple webhook deployment

If you don’t have the ability to run arbitrary post-update hooks but do have some sort of webhook functionality, you can add a webhook to your Publ site to run deploy.sh; for example, you can add this to your app.py:

@app.route('/_deploy', methods=['POST'])
def deploy():
    import threading
    import signal
    import subprocess
    import flask

    if flask.request.form.get('secret') != os.environ.get('REDEPLOY_SECRET'):
        return flask.abort(403)

    try:
        result = subprocess.check_output(
            ['./deploy.sh', 'nokill'],
            stderr=subprocess.STDOUT)
    except subprocess.CalledProcessError as err:
        logging.error("Deployment failed: %s", err.output)
        return flask.Response(err.output, status_code=500, mimetype='text/plain')

    def restart_server(pid):
        logging.info("Restarting")
        os.kill(pid, signal.SIGHUP)

    logging.info("Restarting server in 3 seconds...")
    threading.Timer(3, restart_server, args=[os.getpid()]).start()

    return flask.Response(result, mimetype='text/plain')

Then, in whatever mechanism you use to run the website, set the environment variable REDEPLOY_SECRET to some secret string. For example, if you’re using a systemd service, add a line like:

Environment="REDEPLOY_SECRET=the secret password"

Deploy these changes to your website and restart it. Now you can configure a webhook on your git repository that sends a POST request to the /_deploy route with the secret parameter set to your REDEPLOY_SECRET key.

GitHub-style web hooks

If you’re using GitHub (or something GitHub-compatible) to host your site files, there is a more secure way to run a webhook.

First, install the flask-github-webhook package into your environment (with e.g. poetry add flask-github-webhook).

Next, add the following to your app.py somewhere after the app object gets created:

from flask_github_webhook import GithubWebhook

# Configure the GitHub publishing webhook
app.config['GITHUB_WEBHOOK_ENDPOINT'] = '/_gh'
app.config['GITHUB_WEBHOOK_SECRET'] = os.environ.get('GITHUB_SECRET')

# Deployment hook for self-hosted instance
hooks = GithubWebhook(app)

@hooks.hook()
def deploy(data):
    import subprocess
    import threading

    import flask

    LOGGER.info("Got github hook with data: %s", data)

    try:
        result = subprocess.check_output(
            ['./deploy.sh', 'nokill'],
            stderr=subprocess.STDOUT)
    except subprocess.CalledProcessError as err:
        LOGGER.error("Deployment failed: %s", err.output)
        return flask.Response(err.output, status_code=500, mimetype='text/plain')

    def restart_server(pid):
        LOGGER.info("Restarting")
        os.kill(pid, signal.SIGHUP)

    LOGGER.info("Restarting server in 3 seconds...")
    threading.Timer(3, restart_server, args=[os.getpid()]).start()

    return flask.Response(result, mimetype='text/plain')

Now, set up your deployment to have an environment variable called GITHUB_SECRET set to some random, unguessable string. Do a manual redeployment.

Finally, go to your GitHub repository settings, then “Webhooks,” then “Add webhook.” On the new webhook, set your payload URL to your deployment hook (e.g. http://example.com/_gh), the content type to application/x-www-form-urlencoded, and the secret to the value of your GITHUB_SECRET string. It should look something like this:

Anyway, once you have it set up, every time you commit to GitHub, your site should automatically pull and redeploy the latest changes.

An example of this in action can be seen at the files for this site; in particular, see the app.py and deploy.sh files.

Troubleshooting

Site doesn’t update, or “updates rejected”

This is usually a sign that something changed on your deployment repository that’s causing a git conflict. ssh into your live workspace and do a git pull to see what’s going on. Chances are there was a change on your live repository that’s causing an update conflict, for example you checked in an entry which didn’t yet have an assigned Entry-ID or the like.

Generally you can fix this by going into your live repository and doing git status to see what’s changed, and a command like git checkout . && git pull or git reset --hard origin/master will sort everything out (but be sure not to lose any changes you meant to make directly on the server).

Webhook is timing out

If your packages change, the webhook will likely time out while poetry install runs. This should be okay, but try sshing into your live workspace and running ./deploy.sh manually.

This assumes that you have your own webserver where you can run your site. There are two parts to this: running the Publ service, and routing public traffic from the fronting webserver (Apache, nginx, fhttpd, etc.) to it.

This list is not exhaustive; if there is a mechanism that you’d like to see supported, feel free to open an issue or, better yet, pull request with details!

Running the Publ service

Prerequisites

You will need Python 3. The most reliable way to install this is via pyenv, but any system-provided Python 3 will do, as long as it meets the minimum system requirements.

The easiest way to manage your package dependencies is with poetry; you can see their installation instructions for more information. pipenv and virtualenv also work but these instructions will focus on the use of poetry.

Your Publ environment needs to have a WSGI server available. gunicorn is a good choice for this. The ideal means of installing gunicorn is directly into your site’s environment. If you’re using poetry then that’s done with:

poetry add gunicorn

It will also be helpful to know the full path to the poetry command. You can usually find this with

which poetry

It will typically be $HOME/.poetry/bin/poetry.

Basic approach

Anything that runs the application server should change into the site’s working directory, and then start the gunicorn process with

/path/to/poetry run gunicorn -b unix:gunicorn.sock app:app

which will use the app object declared in app.py (per the getting started guide) and listen on a socket file named gunicorn.sock. The presence of that socket file will also indicate whether the site is up and running.

Note that you don’t have to put gunicorn.sock in the same directory as the application – it can go anywhere that the web server has access. For example, it’s nice to make a $HOME/.vhosts directory to keep your socket files, named with the site name:

/path/to/poetry run gunicorn -b unix:/home/USERNAME/.vhosts/example.com app:app

This way you can also set the directory permissions on your site files with chmod 700 which prevents other users of the server from peeking into them.

systemd

If you’re on a UNIX that uses systemd (for example, recent versions of Ubuntu or CentOS), the preferred approach is to run the site as a user service. Here is an example service file:

[Unit]
Description=Publ instance for example.com

[Service]
Restart=always
WorkingDirectory=/home/USERNAME/example.com
ExecStart=/path/to/poetry run gunicorn -b unix:/path/to/socket app:app
ExecReload=/bin/kill -HUP $MAINPID

[Install]
WantedBy=default.target

Install this file as e.g. $HOME/.config/systemd/user/example.com.service and then you should be able to start the server with:

systemctl --user enable example.com
systemctl --user start example.com

and the site should now be up and running on the local port; as long as it’s up there should be a socket file named gunicorn.sock in the site’s directory.

However, this service will only run while the user is logged in; in order to make it run persistently, have an admin set your user to “linger” with e.g.:

sudo loginctl enable-linger USERNAME

Anyway, once you have this service set up, you can use systemctl to do a number of useful things; some example commands:

# Start the website if it isn't running
systemctl --user start example.com

# Shut off the website temporarily
systemctl --user stop example.com

# Reload the website (if e.g. templates changed)
systemctl --user reload-or-restart example.com

# Restart the website (if e.g. Publ or gunicorn gets updated)
systemctl --user restart example.com

# Disable the website from starting up automatically
systemctl --user disable example.com

# Enable the website to start up automatically
systemctl --user enable example.com

# Look at the status of the website (and see the logs)
systemctl --user status example.com

See the systemctl manual page for more infromation.

Other watchdog services

Some Linux distributions provide other mechanisms for running persistent services under a watchdog, such as daemontools or upstart. These mechanisms are not highly recommended.

cron

In a pinch, you can use a cron job as a makeshift supervisor; create a file named cron-launcher.sh in your site directory:

#!/bin/sh

cd $(dirname "$0")
flock -n .lockfile /path/to/poetry run gunicorn -b unix:gunicorn.sock app:app

and then run crontab -e and add a line like:

* * * * * /home/USERNAME/example.com/cron-launcher.sh

Next steps

You can verify that the site is working with curl; for example:

curl --unix-socket gunicorn.sock https://example.com/

By default, gunicorn only runs with a small number of render threads. You might want to increase this with the --threads parameter, e.g.:

/path/to/poetry run gunicorn --threads 8 -b unix:/path/to/socket app:app

The number of threads to use varies greatly. You almost certainly want more than 1 thread (especially if you’re planning on using authentication), but overallocating on threads can cause other problems to happen in some circumstances. I find that even for larger sites, a thread count of 8-12 is more than sufficient.

Routing traffic

Configuring Publ to work with your web server is a matter of configuring the fronting httpd as a proxy to your Publ instance’s socket. The two most common are Apache and nginx, but there are others as well.

Apache

To use this with Apache, you’ll need mod_proxy installed and enabled.

Next, you’ll need to configure a vhost to forward your domain’s traffic to the gunicorn socket. Here is a basic Apache configuration (e.g. /etc/apache2/sites-enabled/100-example.com.conf), showing how to configure both http and https:

# http
<VirtualHost *:80>
    ServerName example.com

    AcceptPathInfo On
    ErrorLog /path/to/error.log
    CustomLog /path/to/access.log combined

    <Proxy *>
        Order deny,allow
        Allow from all
    </Proxy>

    ProxyPreserveHost On
    ProxyPass / unix:/user/USERNAME/example.com/gunicorn.sock|http://localhost/
</VirtualHost>

# https
<IfModule mod_ssl.c>
<VirtualHost *:443>
    ServerName example.com

    AcceptPathInfo On
    ErrorLog /path/to/ssl-error.log
    CustomLog /path/to/ssl-access.log combined

    <Proxy *>
        Order deny,allow
        Allow from all
    </Proxy>

    ProxyPreserveHost On
    ProxyPass / unix:/user/USERNAME/example.com/gunicorn.sock|http://localhost/

    # tell gunicorn that this is https instead of http
    RequestHeader set X-Forwarded-Proto https
    SSLCertificateFile /path/to/fullchain.pem
    SSLCertificateKeyFile /path/to/privkey.pem
</VirtualHost>
</IfModule>

Note that both connections go over the same socket; you can have arbitrarily many fronting configurations to the same Publ instance, even with different domain names! Publ doesn’t mind at all.

If you use Let’s Encrypt to manage your SSL certificate, the configuration change made by certbot --apache should “just work” although you will want to make sure that it sets X-Forwarded-Protocol or else Publ will still generate http:// URLs for things, which is probably not what you want.

nginx

Here is an example nginx configuration, e.g. /etc/nginx/sites-available/example.com:

# http
server {
    listen 80;
    server_name example.com;
    access_log  /var/log/nginx/example.log;

    location / {
        proxy_pass http://unix:/path/to/gunicorn.sock:/;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

#https
server {
    listen 443;
    server_name example.com;
    access_log  /var/log/nginx/example.log;
    ssl_certificate     /path/to/fullchain.pem;
    ssl_certificate_key /path/to/privkey.pem;

    location / {
        proxy_pass http://unix:/path/to/gunicorn.sock:/;
        proxy_set_header Host $host;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}

Others

The basic premise to configuring any arbitrary httpd to work with Publ:

• Reverse proxy from your vhost to the UNIX socket (or localhost port, if UNIX sockets aren’t supported)
• On SSL, add X-Forwarded-Proto $scheme
• Try to preserve the incoming Host and set X-Forwarded-For if at all possible

If these are not options, you could see if there is direct support for WSGI from the server instead. However, this usually has security implications, especially with regards to how the image rendition cache works; running as a reverse proxy is almost always the preferred approach.

Alternatives to gunicorn

hypercorn

In some situations, hypercorn has better scaling characteristics than gunicorn. However, it isn’t quite as turnkey as gunicorn, as it defaults to tighter security regarding proxies. So, in order to use it with hypercorn, the following changes must be made to your deployment:

1. In order to set the correct socket permissions, either launch it with -m 0 (giving it the same wide permissions as gunicorn), or ensure that your user shares a group with the webserver process and set that as the parameter to -g. My recommendation is to just use -m 0, e.g.

   poetry run gunicorn -b unix:/path/to/socket -m 0 app:app
   

2. In order to correctly handle the X-Forwarded-* headers, add these lines to the bottom of your app.py:

   from werkzeug.middleware.proxy_fix import ProxyFix
   
   app.wsgi_app = ProxyFix(app.wsgi_app, x_for=1, x_proto=1, x_host=1, x_prefix=1)
   

   This should only be done if Publ is running behind a fronting proxy; while this is the most common deployment scenario, it is not universal, so Publ does not ship with this enabled by default.

   Additionally, the numbers may need to be adjusted if Publ is running behind multiple forwarding proxies; see the Flask documentation for more information.

Heroku is probably the easiest environment to configure for Publ, especially for smaller websites. However, it is primarily intended for experimenting with Publ. Heroku comes with a number of limitations:

• Your git deployment size must be under 1 GB
• Your slug size must be under the 500MB Heroku limit
• SQLite databases will not persist across site deployments, requiring a full reindex every time your site changes
• You can use a Postgres database instead but this causes a (very) slight performance hit on page loads

That said, Heroku is a great platform for running a smaller Publ site. Additionally, on the higher tiers you get some nice features like automatic load-balancing and staged deployments. Which is probably overkill for the sites you’re running Publ for, but it’s still nice to have.

Prerequisites

You’ll need a Heroku account, of course, and you’ll want to go through their Python introduction to get your local environment configured.

This also assumes you are using a git repository to manage your website files.

Setting up your site files

The easiest way to configure Heroku is using pipenv and gunicorn. Assuming you have a local test version of your site on your computer, do the following:

cd WEBSITE_DIRECTORY
pipenv --three install Publ gunicorn

This will install all of the deployment requirements for Heroku, and configure them in your Pipfile and Pipfile.lock. Check these files into your git repository.

At this point you should be able to run it locally with:

pipenv run gunicorn app:app

and connecting to the URL gunicorn tells you (likely http://127.0.0.1:8000/).

Next, you’ll need a Procfile which tells Heroku how to launch your site:

web: gunicorn app:app

Check this file in as well.

Setting up Heroku

Now you should be ready to deploy! You’ll need to create your Heroku app and add the git remote to your site’s git repository with e.g.:

heroku create
heroku git:remote -a NAME_OF_APP

replacing NAME_OF_APP with whatever name heroku create gave you.

Deploying to Heroku is now as simple as:

git push heroku

You may also want to run heroku logs --tail to watch its progress.

Database persistence

As mentioned above, most of the website startup time is taken up by the initial content index. Since Heroku does not persist your filesystem, you might want to consider using a Postgres database to store the index persistently. Note that this will slow your site down a little bit (since accessing an in-process SQLite database is faster than going over the network to talk to Postgres), but it reduces the amount of site downtime during a content update so that might be a worthwhile tradeoff depending on your needs.

This slowdown can also be mitigated by increasing your cache timeout; since the site will be redeployed whenever content updates, the cache timeout really only affects how soon scheduled posts will appear after they are set to go live.

To provision a Postgres database at the free tier, from your local git repository, type the following:

heroku addons:create heroku-postgresql:hobby-dev

This creates a database and then exports it to your environment as DATABASE_URL. Unfortunately, PonyORM does not quite directly support database connection URLs, so we need to do a bit more work on our end. Namely, in app.py, change the database_config part to the following:

from urllib.parse import urlparse

config = {
    'database_config': {
        'provider': urlparse(os.environ['DATABASE_URL']).scheme,
        'dsn': os.environ['DATABASE_URL']
    } if 'DATABASE_URL' in os.environ else {
        'provider': 'sqlite',
        'filename': os.path.join(APP_PATH, 'index.db')
    },
}

A full example can be seen in the app.py for this site’s repository.

You will also need to install the psycopg2 library; this is simply:

pipenv install psycopg2

If you get an error about it failing to install (accompanied with a gigantic log) you might try pinning the version:

pipenv install 'psycopg2<2.8'

For local testing (to make sure everything is wired up correctly) you can do:

DATABASE_URL=`heroku config:get DATABASE_URL` pipenv run gunicorn app:app

although be advised that the database scan will be much slower than in production.

Deployment using Phusion Passenger (also known as Passenger WSGI) is fairly straightforward, once you have a python3 environment working. However, on some web hosts, setting up python3 isn’t quite obvious.

Building Python 3

First you need a Python 3 environment. If your hosting provider doesn’t provide one (you can check by running python3 from the command line), there are a few ways to install it yourself.

The easiest way is to use pyenv:

curl https://pyenv.run | bash
exec $SHELL
pyenv update
pyenv install 3.7.6

After a while you should be able to verify that you have a working Python 3 installation with e.g.

python3 --version
pip3 --version

If all worked well, you can now install pipenv:

pip3 install --user pipenv

and, optionally, add this line to your ~/.bash_profile to get better shell tab completion:

eval "$(pipenv --completion)"

Set up the virtual environment

Now it’s time to set up your virtual environment for Publ. Again, ssh to your webhost and do the following:

cd (website_directory)
pipenv --three
pipenv install Publ

This sets up the virtual environment and installs Publ and its dependencies. Now you need to write a passenger_wsgi.py script that tells Passenger how to run it. A fuller example is in the files for this site but a minimal version is below:

import sys
import os
import subprocess

INTERP = subprocess.check_output(
    ['pipenv', 'run', 'which', 'python3']).strip().decode('utf-8')
if sys.executable != INTERP:
    os.execl(INTERP, INTERP, *sys.argv)

sys.path.append(os.getcwd())

from main import app as application

Configure the website

Once you get your files on your server, it’s generally a matter of telling Passenger where to find the files. Generally, this involves going to your web host’s configuration panel and configuring your website to use Passenger. Different hosts may have different requirements. Known configurations are below:

Dreamhost

Configure your web domain as follows:

• Web directory: /home/(username)/(site file directory)/public
• HTTPS (via LetsEncrypt): Yes
• Passenger (Ruby/NodeJS/Python apps only): Yes

You will also need to have a public directory, ideally with a symbolic link to your static directory inside of it; you can set this up by logging in and doing something like:

cd (website_directory)
mkdir -p public
cd public
ln -s ../static .

You will also probably want to create a .htaccess file under public/ with the following contents:

RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*\.php)$ /$1.PUBL_PATHALIAS [L]

Care and feeding

Upgrading Publ should just be a matter of sshing to your host, cding into the site directory, and running:

pipenv update

After doing this, or after changing any site templates, you’ll want to restart your website. On Dreamhost you do this with:

cd (website_directory)
mkdir -p tmp && touch tmp/restart.txt