Are you using webmention.io as your webmention endpoint? Want to get your incoming webmentions displayed on your website?
Well you’re in luck, I wrote a simple-ish script for that. (You’ll probably also want to see the accompanying stylesheet too.) And it doesn’t even require that you use Publ – it should work with any CMS, static or dynamic. The only requirement is that you use either webmention.io or something that has a similar enough retrieval API.
I wrote more about it on my blog, where you can also see it in use. For now, I’m just going to use the sample site repository to manage it (and issues against it).
It’s MIT-licensed, so feel free to use it wherever and however you want and to modify it for your needs. I might improve it down the road but for now it’s mostly just a quick itch-scratching hack that does things the way I want it to.
Posted Sunday, December 16 at 11:36 PM (5 years ago)
Big news: I finally fixed a long-standing issue, where Publ wouldn’t work on Windows due to how Windows uses a different path separator from every other OS on the planet.
(Okay, to be fair, macOS technically does too; internally it has a path separator of : instead of /. But the POSIX API transparently hides that.)
Anyway, I’ve updated the getting started guide accordingly, and now it shouldn’t be any more inconvenient to build Publ sites from a Windows machine as from Linux or macOS.
Also a few other smol bug fixes, like max_width now works correctly on remote images where width isn’t specified.
Just some bug fixes with view caching and image handling; in particular, remote and static images will now respect max_width and max_height for the sizing, and I fixed the way that inline images work (insofar as now inline images can work).
This entry marks the release of Publ v0.3.9. It has the following changes:
Added more_text and related functionality to image sets (an example being visible over here)
Improved and simplified the caching behavior (fixing some fiddly cases around how ETags and last-modified worked, or rather didn’t)
I also made, and then soon reverted, a change around how entry IDs and publish dates were automatically assigned to non-published entries. I thought it was going to simplify some workflow things but it only complicated the code and added more corner cases to deal with, all for something that doesn’t actually address the use case I was worried about. So never mind on that.
(What happened to v0.3.8? I goofed and forgot to merge the completed more_text et al changes into my build system first. Oops.)
I just released Pushl v0.1.3, which adds some minor performance optimizations and a bug fix.
Originally I was hoping to have a major performance optimization, in the form of having rewritten Pushl from thread-per-connection to async operation, but unfortunately I ran into a bunch of problems with it. Mostly that I was running into a “too many open files” error and I couldn’t figure out what was causing a descriptor leak. I have the work-in-progress branch online if anyone wants to take a look at it.
Anyway, the reason I went down this route is because I added WebSub subscriber support to my fork of Feed-On-Feeds, which makes it so that WebSub-enabled RSS and Atom feeds will push their updates to your reader instead of having to wait for a polling interval.
I just released v0.3.6 of Publ, which just allows it to work with databases other than SQLite. In particular this is part of testing more advanced heroku deployment options.
Right now I’m primarily focusing on improving the documentation, especially the quickstart guide, since people are finally showing interest in Publ but aren’t quite sure where to begin!
I’ve also updated the sample site templates with all of the changes that have happened since, uh, June, and also included some sample content so it’s easier to get started with it.
I finally got around to releasing a very rough prototype of Pushl to pypi. It only sends out WebSub notifications for now (does anyone even use those?), but I’ll work on actually implementing WebMention soon.
Also, recently someone pointed out to me fed.brid.gy which makes it easy to turn a static site into an ActivityPub source. At some point I’ll experiment with setting up Publ for this; it looks like it’s just a matter of adding a couple of additional route rules to Publ, so that will probably go into an advanced configuration guide if I ever get around to making such a thing. (Or it could actually be added to Publ directly but there isn’t much of a reason for that, IMO.)
I’ve started working on Pushl in earnest now, and one thing that was really bugging me about this is that anything which polls feeds and entries would really benefit from having client-side cache control working. Which was a big missing feature in Publ.
The short version: for any given view it figures out (pessimistically) what’s the most recent file that would have affected the view (well, within reason; it only looks at the current template rather than any included templates, which is pretty difficult to do correctly) and uses that to generate an ETag (via metadata fingerprint) and a Last-Modified time (based either on the file modification time or the time the entry was actually published).
There’s probably a few corner cases this misses but in general this makes client-side caching of feeds and such work nicely.
I found a few more annoying bugs that were shaken out from the whole PonyORM transition, as well as a couple of bugs in the new shape functionality. There’s probably a few more of these bugs lurking in the codebase (I mean, in addition to the existing bugs I know about), but here’s what’s changed:
Image shape bugs:
Fix some FileNotFound handling on images (so shape errors propagate correctly)
Make img_class and class work correctly per the documentation
Did you know that CSS3 has a style called shape-outline? It’s pretty neat, it makes it so that a floated object gets a shape based on the alpha channel of its specified image. But it’s kind of a pain to set up; in plain HTML it looks something like this:
and if you want a different shape mask for your image than its own alpha channel, you have to do a bunch of stuff like making sure that the image sizes are the same and whatever.
Version 0.3.0 is now released, with the change from peewee to PonyORM.
As a result of this change you’ll have to do two things to your config file:
The database configuration format has changed slightly
Any existing databases have to be manually deleted/dropped/etc.; unfortunately PonyORM doesn’t provide a mechanism for deleting tables not under its control
Everything else should work identically as before.
For a number of reasons, I have replaced the backing ORM. Previously I was using peewee, but now I’m using PonyORM. The primary reason for this is purely ideological; I do not want to use software which is maintained by someone with a track record of toxic behavior. peewee’s maintainer responds to issues and feature requests with shouting and dismissive snark; PonyORM’s maintainer responds with helpfulness and grace. I am a strong proponent of the latter.
PonyORM’s API is also significantly more Pythonic, and rather than abusing operator overloads for clever query building purposes, it abuses Python’s AST functionality to parse actual Python expressions into SQL queries. Seriously, look at this explanation of it and tell me that isn’t just amazing.
Earlier today I pushed v0.2.2, which was a minor configuration fix for the markdown library to support table syntax.
Then I pushed v0.2.2.1 which was another configuration fix to fix that fix (oops).
Then in deploying updates I upgraded to Python 3.7, and promptly discovered that Publ doesn’t actually work on Python 3.7, so just now I fixed that, and released the fixes as v0.2.3.
Updated code to use the current Flask cache-control API
Only set cache-control for responses that don’t have a natural cache response
Entry IDs and UUIDs are now semi-stably generated, in order to prevent (or at least reduce) problems like the last time
Publ itself is stable enough (and enough has changed since v0.1.0) that I felt that a minor version bump was a reasonable thing to do.
Anyway! While Publ has been running quite nicely on my website, I’d love to see more people actively using and developing it. This site in particular needs a lot of attention and probably reworking; my other top priorities are:
A better installation/deployment guide
Proper test coverage (rather than manual smoke tests)
So, sorry to anyone who was subscribed to the RSS feed for this and got spammed with v0.1.24 release announcements. I made a mistake and pushed a version of the entry that didn’t have a canonical ID assigned yet, and as a result, every time Heroku spun up, it assigned a new ID. This is something that’s happened before and I really ought to do something about it.
Three things come to mind:
Figuring out how to always make IDs get assigned in an idempotent manner (hard to do correctly)
Don’t run on Heroku so the assignments persist between executions (easy)
Add a pre-push hook to the repo that verifies that all entries alread have an assigned ID (???)
2 seems like the easiest approach for now, so that’s what I’ll probably do.
The image rendition cache now gets periodically purged; the default is to delete renditions which haven’t been used in the last week (this can be disabled)
Bug fixes:
entry.title can now accept the no_smartquotes parameter, which is necessary in Atom feeds
entry.card now uses the same Markdown extensions as entry.body
I neglected to mention that I set Publ to beta status in v0.1.22, which was a minor bugfix release, rather than moving to 0.2 like I previously stated. The changes for 0.1.22 were:
Fixes to category Sort-Name
Added support for regex path-alias hooks (this is configured on the Python/WSGI side, and has been working quite nicely over on beesbuzz.biz)
Fixed a dumb bug in the cache-control headers
And then the changes for 0.1.23:
Enable automatic smart-quote substitutions (this is the default setting, and can be overridden by passing no_smartquotes=True to entry.body/entry.more/entry.title)
Improve the handling of last-modified times on entries (now there’s a Last-Modified header which only gets set when you want it to be)
This server is now running Publ v0.1.21, which I believe will be the final beta version; all of the release blockers have been taken care of. And as a bonus I also implemented the ability to include template fragments, which allows templates' content to change formatting based on the category of the formatted entries. This is particularly useful on Atom feeds, where you might want to have different entry formats for each category. I’m already using it on busybee, to provide full-content feeds in ways that make sense for the different site sections.
I will play with this for a few days and if I’m happy with it I’ll be releasing v0.2.0, which will be the first beta release. In the meantime I’ll also look into packaging up busybee’s templates for other people to use as a starting point, because I know that setting up a new site right now can be incredibly overwhelming!
The amount of stuff I’m having to fix in Publ to support beesbuzz.biz is diminishing rapidly! Here’s what’s happened since 0.1.18:
Improved the Path-Alias redirection logic; now it will do a 301 Permanently Moved for inbound Path-Aliased requests, and if a Path-Alias points to an entry with a Redirect-To it will redirect directly to that URL instead (and it will be a 302, same as the old Redirect-To behavior)
Pagination can now be weekly; you can use entry.archive(paging='week'), and a ?date= view parameter ending in _w will provide a weekly view instead.
Better default formatting for view.range, and an addition of a week format parameter there
So aside from the Dreamhost issues, I would like to share what’s new in the latest version as part of my big “get my website online” push:
PERFORMANCE: Improved the threading mechanisms around image renditions for better stability and performance
BUGFIX: Made markup tagging work consistently between image types
FEATURE: view.link() now allows overriding category
FEATURE: You can now mark an entry with an Entry-Status of DELETED or GONE, which results in a 410 error instead of a 404 (be the envy of your web-developer nerd friends!)
FEATURE: Error templates will automatically use the x00 error code as a potential fallback (e.g. error code 503 will also fall back to a template for 500)
FEATURE: Entry titles can now have Markdown in them, and it usually works most of the time!
BUGFIX: Now when running in debug mode you don’t end up with two watchdog threads
FEATURE/QUALITY: Refactored the way error pages are handled, and now if you get a 404-type error on a page while the index is being asynchronously scanned, it’ll turn into a 503 with a Retry-After disposition in case it’s just something that hasn’t been indexed yet
UX: View pagination URLs now all use id as the query parameter rather than a miasma of contextually-dependent start, last, or first which made no sense anyway
One of the overarching reasons I decided to build Publ the way I did was in order to take advantage of Dreamhost’s support for Passenger WSGI. I was expecting that to be the primary means of hosting my main site (which is way too big for a Heroku instance) and given how smoothly things were working with this site on Dreamhost I figured it wouldn’t be a big deal.
However, there was a huge monkey wrench thrown into things when I switched my site’s configuration over to Passenger; despite all of my configuration being exactly the same between publ.beesbuzz.biz and beesbuzz.biz, the rendition cache on beesbuzz.biz was getting its permissions set wrong, and there was some rather weird behavior with how it was making the temporary files to begin with.
In investigating this I attempted to upgrade my packages on publ.beesbuzz.biz, and all h*ck broke loose.
Oh, it’s been a little while since I’ve posted an update, hasn’t it? That’s
because I’ve been very busy building the templates for my personal site! In
doing so I’ve greatly improved the way that view refinements worked, fixed a few
silly bugs with image search paths on templates, and also added an easier way to
specify background images in CSS, via the image().get_css_background() method.
I’m really excited to be able to bring my first fully-realized Publ site to the
public; I hope it gets other people interested in what a flexible publishing
system allows them to do!
These things are already making my new personal site look way better and easier
to use! I feel like I’m almost ready to flip the switch.
Oh, and I also improved the getting started guide, including
adding basic setup instructions for Linux and Windows. Not that I’ve gotten Publ
to run on Windows, yet, but documenting how to get the environment set up is
the first step, right?
There’s an old joke in programming, that the two hardest things to do are naming things, cache invalidation, and off-by-one errors. But this doesn’t pay sufficient respect to one of the other hardest things, namely handling date and time.
Today I got two major bits of functionality in: Publ will now asynchronously scan the content index (which speeds up startup and fixes some annoying race conditions with entry creation), and it also asynchronously generates image renditions (which makes pages not take forever to load on first render, and will also use multiple CPU cores if available). Seems to work well so far.
I was running into scaling problems with beesbuzz.biz (what with there being a couple thousand entries and some pages with hundreds of images on it) and this keeps it feeling pretty good.
Rewrote the pagination logic to actually work across category-recursive views, and to support pagination where the
sort order isn’t necessarily the same between renders
Refactored images so that the template image() function has access to the tag writer
These changes were made as part of migrating beesbuzz.biz over to Publ, which I’ve made a lot of progress on now.
It’s pretty nice getting my overall design proven out, and to shake out all the little bugs and misfeatures.
Hopefully soon I’ll have Publ in a state where I feel comfortable releasing it as beta.
I just released v0.1.12 which adds a couple of quick, minor-ish fixes to Unicode handling; in particular:
UTF-8-containing headers no longer get MIME-mangled on first import
I finally made the automatic slug text way better by switching to awesome-slugify
So, now it’s no longer US-English-centric, at least. Right now it just uses the defaults; at some point I’ll want to make it so that you can configure a site’s or entry’s language locale so that things work better on that end too.
I have now implemented the basic OpenGraph API to Publ, so now a template can generate an OpenGraph card with entry.card.
So in theory when this entry gets autoposted to Twitter, this first paragraph should appear, as should the below image:
Anyway that’s what’s new in v0.1.11 (as well as a bunch of internal refactoring to support this addition).
If you are reading this, it means that Publ v0.1.10 is out. This release is mostly about a few cleanups, such as:
No longer nests a <div> for an image gallery inside of a containing <p> (which both fixes an HTML validation error and makes styling more controllable)
Cleans up error handling somewhat
Also cleans up a bunch of code for property caching
But there’s also a new feature, namely view.range, which you can read about over in the API docs.
I am also making significant progress in porting my main website over to Publ and hopefully I’ll
have something to show for it soon. (And I promise it looks way nicer than this site!)
I’ve had people ask me why I’m not building Publ using PHP. While much
has been written on this subject from
a standpoint of what’s wrong with the language (and with which I agree quite a lot!), that isn’t, to me, the core of the problem with PHP on the web.
So, I want to talk a bit about some of the more fundamental issues with PHP, which actually goes back well before PHP even existed and is intractibly linked with
the way PHP applications themselves are installed and run.
Posted Wednesday, April 25 at 4:12 PM (6 years ago)
I’ve released version 0.1.4 of Publ, which makes the following major changes:
limit has been changed to count in the API (for more consistency and clarity)
Similarly, on image sets, limit is now count and limit_offset is now count_offset
More template functions now support positional (rather than keyword) parameters; this has yet to be documented however
And on this site I’ve split out the manual to put the various template object
APIs into their own category, although the organization of the
documentation is still h*ckin' messy.
Posted Thursday, April 19 at 5:45 PM (6 years ago)
Converting Publ into a pip-installable library ended up making the Heroku deployment
really easy. While the main site is running on Dreamhost,
there is an instance on Heroku as well, which seems to
be running reliably (although if it gets hammered with activity I assume Heroku will
shut it down until I start actually paying them money).
There’s a few pluses and minuses to Heroku compared to traditional
hosting on a persistent server (such as Dreamhost).
Posted Wednesday, April 18 at 10:42 PM (6 years ago)
I finally separated out Publ-the-library from the main Publ website files, and
released v0.1.0 on PyPI, making this the first
time I’ve released anything there. Woo!
Instead of working on image renditions I decided to try adding in some caching
functionality, and experimented with both functools.lru_cache and
Flask-Cache. Neither is a particularly
great solution to caching but they get the job done.
I was just on a few days' business trip for my augmented reality startup
(we just released a new app, by the way! check it out, it’s
free and fun) and so I haven’t had time to work on Publ for a bit. But I’ve been getting a bunch
of questions about Publ, so I figured I should answer some of them!
Posted Wednesday, April 11 at 9:22 PM (6 years ago)
So I was trying to figure out why new entries weren’t appearing in indexes until
things restarted. I thought it might be a caching thing, but editing entries caused
them to update instantly. So then I thought it might be an index thing, like maybe
something silly with how SQLite handles timezone-aware DateTimes, so I overhauled
the date handling so that it separated out display_date from entry_date, but
that didn’t help either. Then I looked closely at the database before and after
restarting the process, and that didn’t help. And then I made sure my queries
were using the right timezone, and that didn’t help.
Then I had one of those moments where I realized what I’d done wrong with such
sudden clarity that I couldn’t help but yell at myself a little.
Posted Wednesday, April 11 at 5:00 PM (6 years ago)
So, previously I was just letting the database generate entry IDs on its own.
In SQLite this means it would always generate the next entry ID as the highest
one plus one. This is fine in a circumstance where entries always get assigned
their IDs in the same place, but in Publ’s model that might not be the case.
One particularly fun circumstance: A site’s maximum ID is 406.
You’ve written an entry on your own machine,
and someone else has written an entry on their own machine (or you wrote two
entries, each on a separate branch), and when testing the rendering they both
end up getting entry ID 407. Which one’s right?
Posted Wednesday, April 11 at 12:06 AM (6 years ago)
There was a silly bug with how I was generating paginated views which I noticed
as soon as I did my commit/merge last night but I figured I’d wait until today
to fix it (because the fix was obvious and I was already up way too late).
And I fixed it.
So.
There is only one feature left before I can start making a real site with this
thing: Image renditions.
This is the single most important feature (at least for the initial phase of
Publ), and in fact about 95% of the reason
why I decided to write a new CMS to handle my site content; this is the one thing
that every CMS I’ve seen does a poor job of handling, and which is very difficult
to hack around. (I hacked around it in Movable Type, as I mentioned previously,
and it is one of the most irritating things about running my site as it is.)
Something you might notice on this very blog is that now there are only
5 entries shown at a time by default, and there are handy links for the previous
and next page.
There’s still a bit of stuff that should be added but this was the second-to-last
major thing to do before considering Publ useful enough for me to start migrating
my own site over.
They also point to Dave Winer’s feedbase.io, where you can submit
your feed reader’s OPML export and see what stuff other folks are subscribing to as well. In case you want
to start discovering some more feeds!
Posted Saturday, April 7 at 11:32 PM (6 years ago)
I am trying to make my weekends actual weekends, so the only thing I did
on Publ today was to refactor the way that properties with optional arguments
work. Previously I was using a hack that only worked for string-type things
(such as links), but now I’m using a better hack that works for every sort of
object you might throw at it.
I ended up refactoring the query generator and a lot of template stuff. It feels good to have my codebase approaching sensibility on a lot of stuff. Even if my wrists don’t feel so good.
Anyway I’m posting this from my iPad in bed just to prove that
it’s possible. Although not recommended; vi is kind of hard to use this way.
(this is why i really hope someone steps up and makes some sort of web-based site editing tool for Publ.
i just might do it myself after i get Publ itself in a state where it feels done-ish.)
Oh, and some of my fixes for the visibility bug also led to some code refactoring
that will make view pagination a bit easier to implement, so that’s a nice bonus.
I’ve also put some more thoughts into how view pagination will work. In particular
I think to keep things simple I’ll only allow views to sort by oldest or newest;
The only other useful sorting option I can think of is by title and that’s not
really all that stable.
So, I have a problem where when I really get into a project I start to work
myself to death on it. Perhaps not literally, but enough so that my limbic
system thinks I’m dying. Last night I had a panic attack — my first in quite
some time — due to me having pushed myself too hard.
Fortunately this project is getting to a point where I can slow down my
development, which is pretty necessary for my sanity and long-term survival!
Posted Wednesday, April 4 at 8:20 PM (6 years ago)
Well, I’ve been pretty busy working away on Publ. Right now I’m just focusing
on the stuff necessary to make for an okay blog, with more content types to come
later. Fortunately everything stacks on top of everything else!
This is the first blog entry on Publ! It’s pretty exciting to finally have this
posted.
Publ is a system I’ve been thinking about building for quite a long time, and over
the last few years it’s been occupying an especially high-priority slot in the back of my mind.
Let’s talk
a bit about the history and why I decided to build Yet Another CMS.
