Volume 13, Issue 33; 12 Oct 2010

Hello again. It's mostly the same here, except it's all different.

Try as hard as we may for perfection, the net result of our labors is an amazing variety of imperfectness. We are surprised at our own versatility in being able to fail in so many different ways.

Samuel McChord Crothers

With this essay, and some DNS changes, I've finally pulled the trigger. I've replaced the hairy ball of Perl+XML+XSLT+cwm+RDF+HTML that was my weblog with an entirely new weblog built on top of MarkLogic Server. My weblog is dead, long live my weblog.

There are still a bunch of loose ends, improvements that I can make now that I've got a rational foundation to build on, but I believe that “everything works”. I'm bound to be wrong, but I've found all the bugs I think I'm likely to find on my own.

I've been wanting to do this for ages, but it's the support for XSLT in V4.2 of the serverOr, more accurately, a daily build of some code that is likely to bear a striking resemblence to V4.2 when V4.2 is finally released. About which I have no comment. that finally made it practical. My old system was interesting to build, and I learned a lot doing it, but it wasn't fun to use anymore. Every change to any essay required a build process that took several minutes of wall clock time. This had mostly to do with the shaking and stirring of RDF and the baking.

Given that the system is entirely different, it hasn't changed all that much. Essays, authored in a customization of DocBook, are transformed into HTML with XSLT. The rest of the system manages relationships between various resources and builds links between pages. All those links used to be computed at build time, they're now constructed dynamically by the server (which has indexes to make such computations nearly instantaneous).

What this means is that everything just works as soon as I POST a new essay to the server. (Of course, that's the way these things are supposed to work, it just didn't used to be that way.) Essays that I post go into a “staging” collection that you can't see, but as soon as I'm happy and promote them to the “published” collection, they become visible to you.

I went to some trouble to make sure that the vast majority of the URIs will continue to work: you can still get both XML and HTML versions of each essay, as well as RDF metadata about them. You can still navigate (most of) the directory structure. The feeds still work. I've even preserved some of the purely RDF artifacts, even though the fact that they're in RDF is no longer of particular interest to me. If something seems to be missing, please let me know.

You get one new feature out of the box. Over there on the sidebar, in addition to displaying essays that are “temporally” nearby, I now display essays that are spatially nearby, if there's a geographic location associated with the essay. I'm writing this on the train to Philadelphia for XML 2010, so I've tagged it with “Philadelphia”.

All told, the site is now about 4,000 lines of XQuery instead of about 6,000 lines of Perl (and the same underlying 100,000 or so lines of XSLT, of course). I don't know how long it would take me to reimplement the system from scratch in Perl. I did it in XQuery in a couple of weeks of concentrated, spare-time effort.

Unfortunately, none of this writing is helping to get my presentation finished for the conference. I do hope that I'll write here more now that it's easier to use. I will write more about how this system works, if anyone expresses an interest.


This is a test. This is only a test. Had this been a real emergency, we would have fled in terror and you would not have been informed.

If you can see this, comments work. :-)

—Posted by Norman Walsh on 14 Oct 2010 @ 03:46 UTC #

Congratulations on the successful relaunch! (...and also on locking yourself in to a proprietary vendor :-)

—Posted by Vojtěch Toman on 14 Oct 2010 @ 05:38 UTC #

Congrats on your new weblog ;-) So how does the Nearby (spatially) feature work, could you expand on that? I don't see a spatial element in your RDF. Also, how do you search for articles that are 'nearby' each other?

—Posted by Linda van den Brink on 14 Oct 2010 @ 08:51 UTC #

That fact did not escape my attention, Vojtěch. But I have a lot of respect for and trust in the vendor in question :-)

—Posted by Norman Walsh on 14 Oct 2010 @ 01:42 UTC #