As part of the migration detailed in the previous post, I took the opportunity to write a small 'product' that makes it easy to integrate a StatCounter page counter into Plone sites. I had finally gotten sick of having to hack at main_template.pt in order to jam in the StatCounter code fragment. The result is Products.PloneStatCounter.

It provides an easy-to-use configlet for setting up the StatCounter project id and security code - as well as a couple of other slightly mysterious variables that appear to be required.

I made extensive use of Malthe Borch's tutorial on 'Adding configuration settings using Zope 3 schemas and formlib' and David Convent's tutorial on 'Customizing the viewlets in main_template'. As such, there is little of interest in a coding sense.

... Except for the fact that I found it necessary to avoid registering my viewlet in ZCML as I only wanted it to be used once the product itself had been installed in a portal_quickinstaller/GenericSetup sense. As such, I had to write the local viewlet registration code manually and have it run in my final GS import step.  philiKON was very helpful in pointing me in the right direction for how to do this, so thanks to him for saving me several hours of pain.

I spent this weekend upgrading the underlying Plone instance used to host my blog.  (This was largely because I wanted to get my personal site onto Plone 3 so I could use the NuPlone theme.)

In the process, I took the opportunity to migrate the blog to Quills 1.7.0b2.  Doing so required me first to release 1.7.0b2.  The only change included from beta 1 is that the migration now ensures that weblog entry workflow states are correctly handled.  My hope is that this will address issue 129 in the Quills tracker.

Eating my own dog food has revealed a couple of bugs.  First, portlets don't appear to work in the archive and topics areas of blogs.  From what I recall, this works in QuillsEnabled, so the fix shouldn't be difficult.  Second, when I tried to use ScribeFire (for remote blogging), adding a new post as a draft meant it was lost to ScribeFire (although there on the server).  Not sure whether this is a Quills or ScribeFire bug, though.

Looks like there'll need to be a 1.7.0b3 release...

QuillsEnabled

The 1.7.0b1 code is the first actual release of the QuillsEnabled codebase.  The version number reflects the progress of development in the rest of the Quills project.

It removes the need for custom content types and instead makes it possible to mark standard Plone folders as blogs and then treats standard Plone 'Page' objects as entries.  Those objects are adapted to specific blog-ish interfaces under the hood.  Advantages of this over just using those standard objects on their own are:

• Blog-ish URLs of the [weblog]/2008/06/31/entry-id form.
• The ability to use a MetaWeblog remote blogging client to post new entries.
• A selection of blog-ish portlets - including by-month archives, a tag cloud, author listing, recent comments, recent posts, etc.
• Powerful syndication features - including syndication feeds on a per-topic basis.

Quills

The 1.7.0b1 release provides incremental changes over the existing 1.6 series.  A detailed change log will probably be forthcoming when we get to 1.7.0 (final)... or when someone writes it :).

Giving Them a Try

Both products are released as eggs and are available on PyPI.  Giving the code a try should be as easy as add "Products.Quills" (or "Products.QuillsEnabled") to the "eggs" listing in your buildout.cfg.  (Svn tags of all packages have also been made, but there seems little need to get involved in that.)

Using the MetaWeblog remote blogging facility is optional.  Add the "Products.QuillsRemoteBlogging" (or "Products.QuillsEnabledRemoteBlogging") egg to enable this feature.

These releases should work on both Plone 3.0.x and 3.1.x.

Bug reports and feedback most welcome via the issue tracker.

Following up on the previous post about, the Quills trunk now makes it possible to use the newMediaObject feature of the MetaWeblog remote blogging API. This makes it possible to use your remote blogging client to upload images. Hopefully Jon Stahl will be pleased.

That new feature is also available on the new QuillsEnabled trunk. For the uninitiated, QuillsEnabled provides a new way for you to make use of Quills-ish blogging technology. It "blog-ifies" standard plone content types - Pages and Folders, currently implemented - with just easily-written adapters required for other types you may wish to support (like, say, SmartFolders). It gives you blog-ish URLs (e.g. blog/2008/01/13/entry-id), blog views, tag cloud and other relevant portlets, remote blogging (MetaWeblog API), etc. I plan to make a first beta release of this code in the next few days. If you can't wait that long, then please do give the buildout a go.

Jon Stahl just posted about how good he thinks Windows Live Writer is. In doing so, he notes that,

it would be cool if we could support uploading of images via FTP or via a blogging API from an offline client.

quills.remoteblogging's implementation of MetaWeblog API does not allow this at the moment, but it shouldn't be too difficult to add. From what I can tell, we just need to implement the newMediaObject method. The only complication is that doing so would require extending the IWeblog interface defined in quills.core.

To cut a long story short: I'll try to do this shortly.

"Getting the Blogging story right in Plone" is a good post, getting to lots of the problems that we see and have seen in blog products in Plone. I just want to respond to a few of the specific points as Tom Lazar and I have been working on many of them for the upcoming Quills 1.6 and "2.0" (that version number is not set in stone yet, I'd say) releases.

So what’s wrong with blogging in Plone right now? Mainly I think it’s commenting, at least this is the reason why I switched my COM.lounge TV blog to Wordpress. With Quills I had several problems back then. Some might be due to the version I used (it simply seemed to be bugs) but the most important was trackback spam actually.

Yup. Commenting sucks in Plone. This is something that Plone needs to fix, not Quills. Happily, people do seem to be working on this area. See the new release of easycommenting (which I haven't had a chance to try out yet).

Trackback is something that the Quills project should handle. For the 1.6 release, we have recognised that our implementation is not nearly as good as it should be, and so ship without it. That's better than shipping with something unusable (due to spam) as was the case in Quills 1.5, but obviously it's not optimal to ship without this important functionality. My view is that trackback should share lots of code with commenting, so careful work on the latter has the potential to push forward the former as well.

enable every content or folder to be turned into a blog. This was an idea I had a while back and it seemed that Tom had it, too :-) It probably is simply about adding some marker interface to a folder to enable various blogging features.

This is what is slated to be be Quills 2.0. There is code that does this in the collective under the QuillsEnabled project. It worked a couple of months ago, but is currently broken (I expect) as the code it shares with Quills 1.6 was refactored somewhat, without these changes being reflected in QuillsEnabled... yet. I believe Tom plans to sprint on this after the Plone Conference!

If a folder is marked as a blog it might enable a blogging API for using a blog editor

See QuillsEnabled :).

If a folder is marked as a blog it might have technorati tags as a viewlet available

This may be something that PloneBookmarklets does or could address.

enable blogging APIs (the more we can provide the better) to also edit non-blog content, like normal pages or news items (what’s the difference anyway?)

See QuillsEnabled :). We currently have a MetaWeblogAPI implementation that will work with standard plone content as soon as QuillsEnabled is updated. I know Tom is keen on AtomPub. Ian Bicking appears to be working in this area as well, albeit not explicitly for Plone.

Enable simple statistics on your blog. It would be nice to directly be able to see a rough click count per entry a la Wordpress, the same would be nice for your RSS subscribers.

I agree this would be nice, but has to be something the Plone and/or its documentation handles, I think.

Enable a "more" section on posts which doesn’t show up in the overview listing or the RSS feed.

Quills basically has this already. The weblog view can be told to only display entry summaries (i.e. the standard AT description field), instead of the full text. This is not wired up to the syndication feeds yet, I believe. But that should be easy enough for an only slightly enterprising hacker :).

Enable a feed per category (should be easily doable with collections but making it even easier for end-users or even automatic might be nice)

Quills has had this since 1.5. Append /@@atom.xml to any of the 'topics' URLs, and you get an atom feed for that topic. Likewise for the other feed types we offer. e.g. http://quills.sitefusion.co.uk/blog/topics/Core/@@atom.xml.

So, Quills already offers lots. And where it doesn't quite meet expectations, we have code in the works to address them. Things are getting better :).

Thanks to Tres Seaver (who applied my fix to the various Zope and Five branches), the barrier to merging the maurits-traversal branch to trunk is now gone.

Also, as Jon Stahl pointed out on the Quills-Dev mailing list, it would be a good thing if quills.remoteauthoring was made to work with standard Plone content types, not just the specific Quills Weblog type.

I'll be working on the merge-to-trunk and extending the application of quills.remoteauthoring soon.

I made some good progress today.  With a bunch of checkins (still on the maurits-traversal branch), I removed the old MetaWeblogAPI code that worked by adding persistent attributes called 'metaWeblog' and 'blogger' to Weblog instances.

This was achieved by implementing 'metaWeblog' and 'blogger' as a (i.e. the same) view, instead.  The view is currently implemented in quills.remoteblogging, together with a PAS plugin to extract user credentials from a MetaWeblog request (MetaWeblogPASPlugin).

I'm not completely comfortable with the code layout at the moment.  I'm wondering if there should be another layer of abstraction such that the default metaWeblog implementation adapts to, say, IEditableWeblog which would provide addEntry, deleteEntry, etc.  That might make it easier to implement other remote blogging APIs (like Atom), which presumably call equivalent methods (but with different names).  It might also make it easier for others to reuse the remote blogging functionality for other content types as well.

Something to think about...

Traversal Security Bug

First, I think I have partly tracked down what the security problem is that requires the use of __allow_acess_to_unprotected_subobjects__ on the archive and topic classes. After another pdb session, it seems that those classes do not have a __roles__ attribute, which the Zope security machinery wants, apparently. Actually, I think it's more than that. Even when I add a workaround of __roles__ = Acquisition.Acquired (so that the attribute is just picked up from the next persistent object it finds), things don't work properly as the security machinery chokes a little further down the line.

My best guess at the moment is that InitializeClass isn't getting called (properly?) by the Five ZCML handler for this. That's my next avenue of investigation.

Remote Authoring (aka MetaWeblogAPI)

The other news is that I've started refactoring the MetaWeblogAPI support into separate packages that implement the API with a view rather than a persistent subobject of Weblogs. This should make it easier to disable the functionality, and easier to apply it to alternative content types.

As things stand, I have a PAS IExtractionPlugin that gets user credentials from a MetaWeblog request, and a view that is largely a copy-n-paste job from the existing MetaWeblog implementation.

Some issues remain, though. While I can add the blog successfully with ScribeFire (nee Performancing) - which means at least one MetaWeblogAPI method is getting called without problem - attempts then to get category listings or otherwise use the API fail with a security error.

None of this is checked in to svn, yet. I will do so as and when it gets a little more polished.

I haven't made any Quills checkins for the past couple of days, so I thought I'd explain the current status of my main development branch (maurits-traversal).

As I mentioned before, there is now a hard dependency on Plone 3.  After chatting to Derek Richardson about his work on Vice, this seems even more inevitable as that is likely to have a Plone 3 dependency as well.  (Vice is the new, improved, outbound syndication code that will replace base/fatsyndication as soon as it's ready.)

So, onto the current specifics of the maurits-traversal branch.  WeblogArchive has been removed and posts are now stored directly within the Weblog object.  This means that the "evil" workflow that would cut-and-paste weblog entries into a hierarchy of archive folders is now gone too.  Yay!  The previous folder hierarchy of the form "[weblog]/archive/2007/06/13/entry_id" is now `faked' with an IPublishTraverse adapter.

Not only is the banishment of "evil" inherently good, but removal of the custom workflow makes it possible to be more Plone-ish in the operation of a Quills blog.  The use of standard Plone author/editor roles should now be possible as weblog entries don't need to be (implicitly) deleted during the publish step.  So, the WeblogAuthorRole should now be superfluous.

The main thing stopping me from merging this branch to the trunk is a security issue that means it is necessary to declare __allow_access_to_unprotected_subobjects__ = True on the (non-persistent) topic and archive classes.  This is obviously a bad thing as it competely circumvents security.  For some reason, security and/or the acquisition chain seems to get a bit screwy when the new plone.portlets code gets invoked.  At first, I thought this was something to do with my traversal of non-persistent, non-standard-plone, objects, but Martin Aspeli recognised my traceback as something he's trying to chase down in a different context (pardon the pun).  Hopefully, he can figure out what's going wrong as I've been all through things in a pdb session, and am still none-the-wise.

The other notable change on the branch is that QuillsTool has now been removed.  In its place is a view that has all the same methods, plus some extra ones factored out from over-complicated skin templates.  I'm not completely convinced that the view is the right place for all these methods, though, so a utility may grow back at some point.

The upshot of all this is that the maurits-traversal branch should be in a state for people to check out and give a test run to (as long as you understand it is pre-alpha).  Migrations are not there (yet), so you'll have to use a fresh instance.  Unit tests almost certainly don't pass, either.  They probably need quite a lot of work after the major surgery that has happened...


That's it for now.

Ooh, I forgot to add this in my last post.  I'm delighted that my Google SoC mentor is Raphael Ritz.

Known to just about everybody in the Plone community, no doubt, Raphael is a current member of the Plone `framework team'.  Raphael and I have actually worked together a bit before on his rather fantastic CMFBibliographyAT project.

That means he's kept my code in line a few times before ;-).

Weblog

First, following the pattern used by plone4artistscalendar, provide a TTW/TTP facility to mark individual content objects - like "Folders" and "Smart Folders" - with an interface that causes the default view for the object to switch to a weblog-ish view based on the "IWeblog" interface. Further, such a marker interface could/should cause extra options to become available for the purpose of providing more weblog-specific configuration options. I propose to store this configuration data in "IAnnotations".

Second, provide adapters for those types that are markable as a Weblog to an IWeblog interface. The major part of the implementation would be fetching those (sub)objects that should form part of the weblog. Each of those (sub)objects would, in turn, be adapted to an "IWeblogEntry" interface. Thus, the use of arbitrary (and standard) Plone content types as weblog entries would be made possible simply by providing adapters for each to the "IWeblogEntry" interface. The implementation would provide such adapters for the "Page", "Image", "Link", "Event", "News Item", and "File" standard Plone content types.

Third, provide a traversal adapter that allows such a repurposed container-ish content object to use archive-by-date URLs of the form "[containerish]/2006/05/06/[weblog-entry-id]". This is a standard weblog feature. As an intermediate step, the implementation would also require the provision of a non-persistent WeblogArchive class. Instances of this class would be injected into the traversal path by the Smart
Folder traversal adapter, and would then allow for archive listing views to be used on URLs of the form "[containerish]/2006/05/". I have worked on a partially complete branch of Quills implementing this traversal feature already exists on the maurits-traversal branch. It has issues at the moment, but provides a good base to build from.

Trackback

Completion of the plonetrackback product that allows for trackback pings to be sent to and from arbitrary Plone content objects. The core implementation is largely complete, with just unit tests and more complete integration with the CMF/Plone catalog required. However, the major impediment to the usage of trackback is ping spam. In order to make the product truly usable, some kind of moderation queue system is a necessity. (This is true to a large extent for commenting in plone, IMHO.) Optional integration with the Akismet service would be desirable - and very possible given the two python Akismet wrappers and the Akizmet Zope product - but is probably reliant on implementation of moderation to be effective. Simplistic moderation for trackbacks should be fairly trivial to implement, but I wonder if there is a more generic approach that could/should be taken. The existing workflow machinery is an obvious candidate for reuse here, but the current plonetrackback implementation probably doesn't play nicely with that. I'd be grateful for guidance from a mentor in this area.

Update Pings

Provide a component-enabled tool/utility that allows for update pings to be sent to services such as Technorati. This would be equivalent to the functionality offered by qPingTool, but implemented in a way that is agnostic to the object types that update pings should be made for (by using adaptation). With the introduction of workflow transition events in CMF 2.1, I propose to listen for "publish" transitions and send pings as appropriate. This will require some TTW/TTP configuration for the object types and locations to listen for.

Remote Blogging

Provide remote authoring functionality with Zope3 views. The current Quills implementation of "MetaWeblogAPI" works as a persistent attribute on weblog objects, which makes it more complicated than it should be to disable the feature. Migration code is also potentially more complicated, and a hard dependency on the "RPCAuth" product (to provide authentication) is introduced to the Quills product. The latter relies on something of a hack to achieve its goal. None of this is desirable, and none of it is necessary. Non-persistent views that authenticate via a custom PAS plugin offer a way around all of this, with the added bonus of considerably enhancing the potential for reuse by other products.

p.s.  On the issue of motivation for getting Plone blogging software up to date, see Jon Stahl on how blogs can capture users for plone.

For the past couple of days, I've been working on updating my main development branch (maurits-traversal) to be compatible with Plone 3.  This means being compatible with Zope 2.10, which in turn means that I have had to change the traversal mechanism applied to URLs like "[weblog]/topics/..." over to use the new IPublishTraverse interface and design.  As IPublishTraverse is not available in Zope 2.9.x, this amounts to a new hard dependency for Quills (1.6+) on Zope 2.10 - which presumably means a hard dependency on Plone 3+.

A few notes on the new IPublishTraverse mechanism are in order.

First, it took quite a long time to swap things over, although this was largely because it took me a while to figure out what I was supposed to do.  Maybe there are docs, but I coudn't really find them.

Second, the IPublishTraverse is, for my purposes, not as nice as what went before.  My goal was to be able to intercept all URL segments below "topics" in "[weblog]/topics/Football/Tennis/Zope", and thus get the kind of traverse_subpath behaviour that "Script (Python)" objects have.  In the old setup, your traversal adapter was passed "furtherPath" as a parameter, and this would look like "['Football', 'Tennis', 'Zope']".  Easy enough to intercept the rest of the path, then.  In the new setup, the IPublishTraverse.publishTraverse method only gets the individual URL segments one at a time.  To get around this, you have to grab hold of "request['TraversalRequestNameStack']" directly - and that feels a little bit wrong.  It appears I'm not the only one who thinks this.

Ho hum, traversal (at least) seems largely to be working now.

My first step today was to fix the plonetrackback bug for deleting received trackback pings (issue 90).

After deleting a trackback ping, it gets uncataloged.  However, the way that the pings are stored in the catalog is rather convoluted and hackish.  As a result, the standard unindexObject call didn't work as expected, and instead also uncataloged the actual WeblogEntry instance to which the ping was attached.  I resolved this with a change to uncatalog_object(trackback.getPath()).

Applying through the Plone Foundation, I have been awarded funding by Google through their Summer of Code programme to improve the Quills weblog solution for Plone.  This is just a brief post to say that I'll be posting updates on my progress to this blog.

In the interests of eating my own dogfood this is, of course, a Quills blog.  More specifically, it is a fresh instance based on Quills 1.5RC3.  In the interests of continuing to eat my own dogfood, I hope that it won't remain as such while I develop...