Publ: Development Blog

Some thoughts on WebMention

Posted (2 months ago)

So, for the last couple of days I’ve been playing with some of the IndieWeb concepts, in particular Webmention. Spurred on by a helpful thread with Kevin Marks, I took some time to actually do a rough implementation of outgoing Webmentions, and also did some of the work to set up the h-card and h-entry microformats on my main site.

As far as I can tell, it works great, but I’m also not going to actually merge this to master or push it to production. Read on to see why!

When I started Publ, it was with the very specific goal of providing the publishing mechanism for a content store. Given content files and templates, format them into reasonable web-friendly HTML and Atom and whatever else the templates do, in a basically-stateless manner, and do a great job of providing high-quality image renditions and generated CSS and so on, while also supporting things like legacy URL mapping and automatic redirection and so on.

Basically, be like a static publishing system, but dynamic.

I want Publ to be the publishing and formatting part of an open, dynamic, independent web. I do truly believe in the IndieWeb mission, and absolutely want to see it become successful! However, I feel like Publ’s role is best served as one of many interlocking parts to make this work.

Supporting Webmention correctly means having the following:

  • Automatically send out Webmention pings based on content changing
  • Automatically receive and handle Webmention pings as they come in
  • Provide structured metadata about the page content so that pings get credibly formatted on the receipient’s end

It also has a few other fun implications, like:

  • Have a canonical URL for the content, generated by the publishing system itself, regardless of how the incoming traffic is routed to it
  • Send updates as often as things change, but no more often than that
  • Keep track of updates being sent (rather than just spamming everyone else with spurious updates)
  • Generally, be stateful

And being stateful is a thing that flies in the face of the overall Publ design.

(So is the canonical URL thing, for reasons not worth getting into here, but briefly, things like staging/local/debug instances vs. the final production endpoint with any number of domain name normalizations and selections of protocol and so on. It’s messy and not something that’s easy to configure, and is impossible to detect in a way that’s consistent with what the outside world sees.)

The way I see it, this stuff is better left to external tools. Publ is for managing and formatting content. There are plenty of other tools that can be integrated into Publ templates to provide IndieWeb functionality in the way that seems most fitting to the person running the site.

For receiving WebMentions you can use webmention.io or brid.gy or this Disqus-like webmention endpoint thingamajig that’s currently in early alpha. For sending WebMentions you can use Telegraph or webmention-tools or ronkyuu or any number of other tools for sending mentions out. All of the configuration for the inbound webmentions, and for your semantic markup on your templates and so on, belongs in your templates, for what you think makes the most sense for your setup. There is no reason for it to actually be integrated into Publ.

A similar story exists for WebSub. Publ doesn’t know – nor should it have to know – what templates correspond to RSS or Atom feeds. It doesn’t know (or want to know) which of those feeds are affected by which category posts. It shouldn’t have to be configured with any mappings for which categories trigger which templates' updates, or which hub(s) those updates should be published to.

These are all better served by external tools that you, the publisher, configures to work the way you want them to.

So my proposal for supporting this stuff:

  • Have a cron job that checks your feeds to see if they’ve updated
  • When they do, run the appropriate tools to do the job of sending out notifications

To this end, I might start a separate project that lets you set up these mappings yourself, to be run as a sort of controller thing. It wouldn’t be at all integrated into Publ; indeed, it would work with any CMS that exports a feed or whatever. It would be configured with the following:

  • One or more feeds to check
  • Various plugins/actions to perform on those feeds

For example, a site’s master feed would have a WebMention task which would go through every entry in the feed, and does the following:

  1. Fetch every referenced item
  2. If the item has changed, parse it for all <a href> elements and send WebMentions

And they could also have a WebSub task which would simply:

  1. See if the feed has changed
  2. If so, send a WebSub notification to the hub of your choice

These tools can be simple and elegant and work universally across all of your sites, not just Publ ones! Got a Wordpress site? This will work with it! Got a Tumblr site? This will work with it! Got a YouTube channel? This will work with it.

And in the Publ case this also settles the canonical URL issue because it’s based on what the external tool sees, which is what the rest of the Internet should see – not based on whatever whimsical, fragile, momentary state might be the case due to it running on localhost:5000 or your private firewalled staging/preflight instance or whatever.

And this also gets along perfectly with another notion I had going into this – having services to syndicate content into a Publ site by reformatting an external Atom feed into an entry, for example. Want to automatically post all your Flickr items to a gallery on your Publ site? Use their Atom API. And the very same kind of simple tool can, for example, re-post your itch.io devlogs or your Tumblr reblogs or your YouTube videos or whatever.

I am entirely a fan of small tools doing simple things, and composing tools together. The world of CMSes already has too many Swiss army knives. Let’s keep our tools simple, doing a few things well, and have them work together with other tools.

I will absolutely have Publ support Webmention and WebSub and so on, in the best way possible – by doing nothing, and not getting in the way of other tools that can do it better.

Publ is a puzzle piece, not a complete solution.