v0.5.12 released, and lots of documentation fixes

Release notes

Today I got a fire lit under me and decided to do a bunch of bug fixing and general performance improvements.

Changes since v0.5.11:

• Fixed a micro-optimization which was causing some pretty bad cache problems (I really should write a blog entry about this but tl;dr micro-optimizations are usually bugs in disguise)
• Fixed an issue which was causing the page render cache to not actually activate most of the time (you know there’s going to be a ramble about this below…)
• Fixed a bunch of spurious log meessages about nested transactions
• Refactored the way that markup=False works, making it available from all Markdown/HTML contexts
• Changed no_smartquotes=True to smartquotes=False (no_smartquotes is retained for template compatibility) (although I missed this on entry.title; I’ve already committed a fix for the next version)
• Improve the way that the page render cache interacts with templates
• Fixed an issue where changing a template might cause issues to occur until the cache expires

Documentation improvements

• The Apache/nginx deployment guide is vastly improved:
  • Now it uses UNIX domain sockets instead of localhost ports, making service provisioning a bit easier
  • The systemd unit is now a user unit instead of a system unit, which improves security and also allows for gentler service restarts
• The git deployment guide has been updated per the above, and also some of the code snippets are cleaned up
• The information about HTML processing and image renditions has been consolidated and cleaned up
• Information about private posts and user configuration has also been cleaned up somewhat
• Also lots of updates to the beesbuzz.biz Publ templates

Publ v0.5.11, Authl v0.3.5

Some fresh new versions of things.

Publ

Changes to Publ:

• Massive improvements to how footnotes are handled; now they get their own virtual section (so if you’re currently using footnotes you’ll need to update your templates!)¹
• Various performance improvements:
  • Some internal caching on image rendition stuff
  • Reduce contention in the content indexer (to hopefully make large sites more responsive on restart)
  • Minor optimizations to category.subcats
• Removed config.secret_key; now this should be configured on the application object per Flask standards
• Allow HTML attributes other than href and src to contain image renditions

Authl

• Improve WebFinger support
• Improve the JavaScript in the default Flask template

Publ v0.5.10 released

Publ v0.5.10 is now available. The following has changed since v0.5.9:

• Image sets will generate fullsize renditions (and their links) for images which were skipped, so they will still appear in the lightbox
• Footnotes now get rolled up into the entry.more (with some caveats), and also get stable permalinks for their references¹
• You can now annotate an HTML attribute with $ to force it to resolve as an asset (useful for certain JavaScript libraries); see the relevant documentation

I’ve also made a bunch of changes to the sample templates.

Publ v0.5.9 released

Publ v0.5.9 is now out. Just a couple of bugfixes in this one:

• Login pages now properly redirect to https again (after that was broken due to some of the recent auth-related changes)
• view.range works correctly again, as does everything else that relied on len() on parameter-optional properties (e.g. view.count)

Publ 0.5.8, Authl 0.3.1, and IndieAuth security

So, both Publ and Authl had a pretty naïve issue with the identity verification step of the IndieAuth flow; it simply accepted whatever the authorization endpoint said the user’s identity was. This made it very simple to spoof one’s identity and log in as anyone on any Publ or Authl site.

Authl 0.3.1 fixes the problem with the IndieAuth login flow, and Publ 0.5.8 fixes the problem with the Bearer token flow.

Publ v0.5.7, now with theoretical AutoAuth support!

There is only one feature for this new release of Publ, but it’s a big one – there is (theoretical) support for AutoAuth! That’s right, deploy this version and people should be able to magically log on to your website using unattended IndieAuth providers.

Unfortunately, there aren’t any tools that I know of which actually support this mode of operation; all testing has been manual and In Theory.

Fortunately, if someone does want to test AutoAuth (or IndieAuth Bearer authentication in general), you can test it out on this site! You can use this entry as an individual entry, and this category or this feed to see how well it works with the “partial public” path.

Also, this page will tell you all sorts of useful information about the current user (if any).

And I’d might as well use this opportunity to show off the admin dashboard – just sign in as the user test:admin to see how it looks.

EDIT: It looks like there’s a problem with third-party auth due to the way that Heroku works. I should have anticipated this. Third-party auth is temporarily disabled for now. (But this doesn’t affect AutoAuth at least!)

Publ v0.5.6 released

Oops, I’d been sitting on a bunch of bugfixes for a month, which I didn’t notice until I put in another bugfix tonight.

Changes since v0.5.5:

• Fix title sanitization
• Handle category.name with the same formatting options as entry.title
• Replace hand-rolled atomic file operations with atomicwrites
• Add link_class to image renditions
• Fix automatic alt generation for external images
• Simplify the way entry URLs are canonicized
• Fix some bitrot in older tests

Publ v0.5.5 released

Howdy y'all! Here’s a new release of Publ for you.

What’s new in this version:

• Add the ability to filter by multiple categories, and also to filter out categories as well
• Various code cleanups, especially around the query generator

Also the unannounced v0.5.4 release was to fix some stuff that broke due to an upstream Arrow change (specifically dealing with them removing an API that I was using to suppress warnings for a different upstream change that I’d already handled).

I should also mention that I’ve updated the beesbuzz.biz template samples to improve IndieWeb and ActivityPub compatibility. (Publ still doesn’t support ActivityPub itself but these changes make it interoperate with Bridgy Fed a bit better.)

On a meta note, I’ve left the microbiology lab I was at; I hope they continue to use Publ, of course! Over the next little while I’m going to spend some more time working on my own things again (including Publ et al), but I’ve also had some interesting job interviews with one of them seeming very likely to turn into an offer. Wish me luck, if you’re into that sort of thing! (And of course, follow my blog for the primary source of this stuff.)

Publ v0.5.3, Authl v0.2.2

I’ve released updates to both Publ and Authl.

On the Authl side:

• Code quality and documentation improvements
• Add an asynchronous client-side lookup thing that tells users how their login will proceed
• Add the redirection target to disposition.Error so that can be preserved correctly
• Update the Flask wrapper to use disposition.Error.redir
• Let the application know the redirection target in render_login_func

On the Publ side:

• If the site is configured to force HTTPS in authentication, force the cookie to be HTTPS-only
• If a user is already logged in, make the login handler redirect them to their destination

For both:

• Improved build scripts to make it less convenient to accidentally push a build from the wrong branch or version

These changes help to keep sites more secure from eavesdroppers, while also hopefully improving the user experience!

v0.5.1 released (also Authl v0.1.6)

Oh gosh I seem to be on a roll with these updates again. Here’s what changed in Publ:

• Fixed a silly bug in the admin dashboard renderer which made it not work in production mode
• Make the admin log only record the most recent access per user per entry, making it way more useful
• Make the logout operation happen via POST method rather than GET, fixing a problem with browser prefetching; added a logout.html template to support that. (Also made the default unauthorized.html use Authl’s default CSS.)
• Actually make entry.authorized available, rather than just documented. Also gave it a better name while I was at it.
• view.entries can now take an optional argument for inlining unauthorized entries, improving its usage within feeds.
• view.unauthorized can now take an optional argument for limiting the unauthorized view count, which helps performance and makes it a bit more predictable
• Images now provide their filename as the default alt text, which is arguably better for accessibility than just leaving it a blank string. I am willing to change my mind on this, however.
• Cleaned up the code around category.subcats(recurse=True) and also added some actual tests for the sort ordering. They pass.

And the Authl changes (which were actually released before Publ 0.5.0 but I didn’t bother announcing them until I had them tested “in the wild”):

• Changed to using packaged data for templates
• Made the login page CSS available through url_for
• Removed the spurious precision from the email message template

Anyway, I of course updated the sample beesbuzz.biz templates to reflect the new functionality.

Wow, Publ’s feeling like it’s actually kinda pretty good at stuff now. I hope someone else ever wants to actually, like, use it or something.

v0.5.0 released

I figured there wasn’t really any reason to keep waiting. So here we are.

Changes since v0.4.6:

• Improve the performance and stability of the admin dashboard
• Correctly fall back to the internal Authl templates
• Remove some spurious/empty attributes from image tags
• Don’t cache template mappings forever
• Don’t mark an entry title as being markup if its markup is disabled
• Correctly set the default entry recursion for entry.previous/next
• Disable an arrow warning for a future change

In other news, over on my main website I have successfully migrated my comments over to Isso, which is a nice self-hosted alternative to Disqus that does a much better job of handling privacy in particular, as well as providing a simpler UX that doesn’t try to get in your face about everything. If you want to read more about how I made that change, read the several blog entries starting with “Moving away from Disqus,” and also look at the sample templates to see the actual implementation.

May your private entries remain exclusive, and your public entries be brilliant.

UPDATE: Someday I’ll learn to use and test rc builds before making an actual public release. Oops.

Publ 0.4.6, Authl 0.1.5

Updated some packages.

Main things with Publ since the last release:

• Internal cleanups to how caching happens
• Stop spuriously-caching a bunch of stuff; in particular login/logout endpoint URLs no longer get cached
• Various cleanups
• Improve the way that built-in templates are managed
• Initial cruddy implementation of an admin authentication dashboard (although this isn’t quite ready for prime time)

The only Authl change is that email identities are now given as a full mailto: URL; going forward all identity strings will be full URLs. This simplifies the UX for admin dashboards, in particular, and removes some ambiguity.

Publ 0.4.5.1, Authl 0.1.4

I’ve released a mini-update of Publ to fix an authentication problem (the config parser was “helpfully” sanitizing things that didn’t want to be sanitized), and also some refactoring/improvements/bugfixes to Authl.

The big changes to Authl are that the email handler generates shorter/nicer links, and it also puts an anti-abuse timeout into email login attempts to prevent people from spamming themselves or others with spurious email notifications. There’s also a bunch of small bugfixes to Authl’s login flow, and Flask apps can specify that sessions should not be made permanent.

v0.4.5

Normally I wouldn’t release a new version just for a single minor bugfix, but this was causing bigger problems. Oops.

Anyway, there was one other minor fix, which allows “cb” to be a valid category name again. It’s minor and fiddly but hey, consistency, right? (And anyway you never know, someone might use Publ for a site that has a CB enthusiasm blog!)

v0.4.4, and private entries in the wild

I’ve added private entry stuff to my website (here’s an example post) and in doing so I shook out a few loose ends:

• Improved the login flow for when someone is logged in but goes to an entry they don’t have access to
• Simplified generating login and logout links from templates
• Added Status: UNLISTED as a synonym for Status: HIDDEN

All the auth-related things are now documented here and also demonstrated in the sample templates.

There is not much left for v0.5, incidentally!

v0.4.3! Authentication!

Wow, this is a pretty major update: authentication is now a thing!

It isn’t quite complete yet – I still have a few more things I want to add before I consider it done (and therefore release v0.5.0) – but this is at least in a state where it’s ready to be experimented with. Probably. I need to sleep first, before I start adding authentication to my website.

v0.4.2 released

Just another small-ish release to address some bugs and minor interop concerns:

• HIDDEN entries now come with headers that opt out of caching and archival
• absolute=True now applies to all links, not just rewritten ones
• Templates now correctly cache when using arbitrary GET parameters (using request.args per Flask convention)
• Improved the error messaging for a malformed pagination query

This weekend I’ll be at IndieWeb Summit in Portland, Oregon! I hope to see some of you there.

v0.4.1 – it’s released

Not a whole lot new in this one, because I haven’t had time to work on Publ lately.

This release is mostly to fix a silly bug that was causing apostrophes and quotes to get dropped from plaintext renditions of titles and whatnot, which was caused by some of the refactoring I did in the run up to the 0.4.0 release and didn’t notice until just now.

I also fixed a MathJax test, and did a bunch of refactoring/cleanup of one of the internal API shims. Nothing user-facing.

Oh! And I’ll be attending IndieWeb Summit 2019 in Portland at the end of the month. I look forward to meeting other IndieWeb wonks! I love that RSVPing to it got me to finally fix my site template’s RSVP function.

And in other news I’ve finally been cleaning up the quickstart guide thanks to some feedback from a potential user. If you’ve been wanting to try Publ but were entirely confused or lost or whatever, please give it another look! And of course I am happy to get documentation feedback (or pull requests!) on the site repository.

v0.4.0 released! Let’s talk future!

Oh wow, I finally closed out the Publ 0.4.0 milestone. So, wow, this is a pretty big deal for me.

Changes since 0.3.23:

• Fixed a bug with absolute paths in asset links
• Added a default error handler if there’s no global error template configured
• Finally added the ability to configure the Markdown processor, both at the global and per-template level

So that closes out the last three issues before I was ready to consider this usable by a wider audience. Neat.

There’s a lot of stuff I want to do for 1.0 of course, but also more importantly I need to vastly improve the documentation on this site, as well as collect things to make it easier for people to get started with Publ. I would absolutely invite people to make suggestions; have you looked into Publ, and been confused about what exactly to do? Please let me know!

The best ways to do so are to either hop onto my Discord channel or to open issues against the documentation site.

v0.3.23 Released

I got an itch today to do a bunch of feature work, and also found and fixed a bunch of bugs that I didn’t catch before.

Changes for this release:

• Fixed a bunch of tests which broke when I reorganized the test repository
• prefix_ attributes now apply correctly to image and imageset style and class attributes
• Entries with empty slug-text now omit the slug-text separator from their permalink URL
• Markdown entries now support link rewriting for href and src attributes (so for example <audio src="local_file.mp3"> will now work)
• Major bug fixes to the HTML renderer, which was dropping all entities, character references, and comments (oops)
• Disabled automatic smartquoting in HTML entries

v0.3.22, now with tag browsing functionality

I’ve now released Publ v0.3.22, which is mostly a bugfix release but also adds a couple of useful things. Major changes:

• Entries which are set to status DRAFT or GONE no longer function as path-alias redirections
• category now has tags, allowing templates to build in tag browsing
• Similarly, view now has tag refinements, making it easier to modify views' tag restrictions
• Fixed a silly bug which was causing top-level categories to not redirect without a trailing /
• Also did a bunch of refactoring on image handling, with no functionality change but it makes the code easier to navigate (I hope)

Oh and I guess I forgot to post a release announcement of v0.3.21 back on April 2, oops! There were some pretty major fixes to that:

• Somehow the <img alt> attribute got turned to <img alt_text>, which is not valid. That has been fixed. (I suspect I was really tired when I was reorganizing some code.)
• JPEGs now correctly support EXIF orientations/rotations
• Added support for entry.summary, as well as overriding the summary text on an entry..

Also, how did I miss that v0.3.21’s release was on the first anniversary of my first functioning blog post?! Wow, time flies.

Pushl v0.2.5 (not a joke!)

I have released v0.2.5 of Pushl. The changes since v0.2.4:

• Improved the logging levels a bunch, making them more useful
• Added the ability to only send WebSub for feeds

The latter improvement makes it so that if your site is accessible from multiple URLs (e.g. http and https, or multiple domain names), it won’t send multiple Webmentions to everyone with each possible URL. This helps to cut down on spamminess to sites which don’t detect multi-origin pings (such as most IndieWeb blogs or fed.brid.gy).

I also finally added a tools page to this website, to collect useful things that make Publ work better with other things. I also intend to add various useful quality-of-life things like an image cropping tool, and whatever else might occur to me down the road.

Someday I need to get around to making a proper “Features” page for this site, too. Someday…

Publ v0.3.20 released

Yep, it’s another release. New stuff in this one:

• Changed cropping to be (x,y,w,h) (as previously mentioned) and also changed scale-cropping to be a single-pass operation
• Added Hidden-Tag
• Added a tags property to view objects

Also, on an administrative note, if you would like to help fund the development of Publ, I have switched from Patreon to Liberapay, which provides a bunch of advantages over Patreon and is much better-suited to open-source development.

Pushl v0.2.4, now with a proper user-agent

While trying to figure out some weird access patterns on the day-job site I had the realization Pushl wasn’t actually specifying a user-agent, so it was just coming through as the generic aiohttp one, which isn’t very friendly.

Now it sends a reasonable user-agent by default, and this can be overridden by the --user-agent flag if you want to for your own analytics or whatever.

Oh, and I had quietly released 0.2.3 a few days ago; there were just some minor internal changes to logging and also declaring Pushl as beta, rather than alpha, software.

Pushl 0.2.2

I’ve done a bunch more work on Pushl to try to get it more stable. In particular, I’ve made it so that it will only recurse into feeds that are on domains that were declared in the initial requests, and I seem to have cleared up some cases which were causing it to hang and also added a global timeout which will, hopefully, prevent it from hanging indefinitely.

I do wish I could figure out what is causing the hangs when they do happen though. Oh well. Some discussion of the issue below the cut.

Pushl v0.2.1 released

I’ve been working on getting Pushl much more stable and reliable, particularly around a persistent “too many open files” error I was having, which turned out to be primarily due to a fd leak in the caching routines. Oops.

Anyway, there’s also seemingly a problem with how aiohttp manages its connection pool, at least on macOS, so I’ve disabled connection keep-alive by default. However, if you still want to use keep-alive, there’s now a --keepalive option to allow you to do that. I’m finding that it doesn’t really improve performance all that much anyway.

This is feeling beta-ready but I’ll give it a few days for other issues to shake out first.

Pushl v0.2.0 released

So, I just released v0.2.0 of Pushl. It was a pretty big change, in that I pretty much rewrote all the networking stuff, and fixed some pretty ridiculous bugs with the caching implementation as well.

The main thing is now it’s using async I/O instead of thread-per-connection, so it’s way more efficient and also times out correctly.

And oh gosh, I had so many tiny but critical errors in the way caching was implemented – no wonder it kept on acting as if there was no cached state. Yeesh.

Anyway, I’ll let this run on my site for a few days and if I like what I see I’ll upgrade it to beta status on PyPI.

v0.3.19, now with extra tagging goodness!

I’ve released Publ v0.3.19, which now finally has a tagging system, which is only one of the oldest issues that was still open.

Here’s a list of what’s been added or changed since 0.3.18:

• The test site is now part of the Publ repository itself (as mentioned previously)
• Added crop and fullsize_crop to image renditions (although I’ve already decided the format should be different after using this for a day)
• Made Path-Alias less annoying
• Fixed an issue with unpaged archive links getting filtered
• Prune missing files from the index

Credits

I want to thank Karina Antonio for implementing image cropping.

v0.3.18, now with better asset management!

I’ve just released v0.3.18, with the following changes:

• Add date grouping properties to entry
• Add a pages property to view
• Provide the current category object to the error handler
• Support linking to non-image/non-entry local files
• Added, then removed, some performance micro-optimizations that only caused problems

More details about the major changes below!

Update: I released a hotfix as 0.3.18.1 because there was a last-minute bug that snuck in while I was trying to silence a new pylint error. Oops.

v0.3.17

Just some little fixes that I put in for the sake of the day job:

• Fixed the way that get_view() et al parse the parameters, so now you can actually search for entry_type='' or entry_type_not=''
• Fixed the sort order of recursive category.subcats

Publ v0.3.16

Just another minor release of Publ! Mostly bugfixes. Here’s what’s new:

• Fixed an issue with HTML <img> rewriting with non-lowercase attribute names
• Fixed a bug with missing entries causing Publ to die when it tried to remove them from the index
• Added category.root
• Some code cleanups, especially removing some unneeded backwards compatibility shims for unsupported versions of Python
• Redirect-To will now resolve link targets

Relatedly to category.root, I’d neglected to add category.breadcrumb to the manual when that functionality went in, and I’ve also added some usage examples, including something useful for fancy navigation bars.

v0.3.15 Released (finally!)

It’s been a while since I’ve had a chance to work on Publ, but the great thing is that I actually had a reason to work on it for my day job. Which is to say I’m finally being paid to work on Publ. ;)

Changes since 0.3.14:

• Add requirement for Arrow 0.13.0 (issue 41)
• Fix a dumb tpyo that was the cause of issue 158
• Don’t rewrite DRAFT files; fixes 137
• Move sample-site files back to the library repo rather than in the doc repo
• Fix the way we map malformed category URLs (issue 156)
• Update upstream library versions
• Move version number to publ module
• Allow empty slug-text in entry route (fixes 161)
• Process HTML entries, to finally handle issues 136 and 154.

Some more information about that last one under the cut!

v0.3.14 (happy π release)

Version 0.3.14 of Publ is now released!

This update is mostly about image stuff:

• Fixed a subtle caching bug that affects sites accessible from multiple URLs
• Fixed the way that size-clamping (max_width et al) work on remote and static images
• Enable JPEG optimization in the image renderer
• Add the ability to link to local rendered images in a Markdown link

Also, if you’re using Publ and hosting your repository on GitHub you may have gotten a security warning regarding the version of pyyaml that Publ depends on. Don’t worry, Publ doesn’t actually use the vulnerable code (it’s actually pulled in by one of the utility scripts from the watchdog library, and not used by watchdog itself). Watchdog has an open issue about this and they’re on track to fix it Real Soon Now.

In the future Publ may actually pull in pyyaml itself for the friends-only functionality, but when it does you can be sure it’ll be a current version. :)