DITA 2006

Reflections on DocBook and DITA This is a transcript of my presentation to the DITA 2006 conference in Raleigh-Durham, NC on 24 Mar 2006. The slides I presented are interjected as sidebars. Thank you, Ken. (I was ably introduced by KenBerger of SiberLogic, one of the conference's platinum sponsors.) Good morning. I'd like to start by thanking Kay and the rest of the program committee for inviting me to speak to you today. I have to admit that given my longstanding association with DocBook, it feels a little strange to be presenting at the DITA conference. I work for Sun Microsystems and I'm the chair of the DocBook Technical Committee. I'm also active in about a half-a-dozen other working groups spread across the W3C, OASIS, and the Java Community Process. I have to start with a disclaimer however, and that is that I'm not representing any of those organizations today. Disclaimer

JimMelton “Facts are facts. But any opinions expressed are the opinions only of myself and may or may not reflect the opinions of anybody else with whom I may or may not have discussed the issues at hand.”

The ideas and opinions in this presentation aren't necessarily shared by my colleagues on the DocBook Technical Committee, my employer, or anybody else. No cutlery please Jeeves Saves the Cow-Creamer Disclaimer aside, Given that this is a lunchtime presentation, I would like to request that you limit your throwing of any projectiles to dinner rolls. Actually, the subject of fights and throwing things segues directly into the topic that I'd like to start with. Opposition Emacs vs. vi Linux vs. Windows Python vs. Perl REST vs. WS-* People generally have a habit of placing things in opposition, especially when the things in question serve a common purpose but have been adopted by different communities. On my more cynical days, I attribute this to consultants and marketing departments trying to stir up excitement about their latest products or services. But it's also simply harder to cooperate than to fight. Cooperation requires developing a shared understanding of problems, and mutual respect in the face of different choices. Whatever the reason for the tendency towards competition, I want very much to avoid having “DocBook vs. DITA” make it onto that list. At the very least, I don't want to play that game. Healthy Competition DocBook and DITA can solve similar problems Users are asking: DITA? DocBook? Both? Competition challenges us Challenges inspire us to improve At the same time, we can all see that DocBook and DITA are able to solve similar problems. Their approaches differ somewhat, and there are a lot of differences down in the details, but I doubt that there's any significant documentation problem that you could solve with DITA that you couldn't also solve with DocBook, and vice-versa. So the question of which schema to use isn't a question about what one can do and the other can't. It's about design patterns, the richness of the vocabulary, the maturity and capability of tools, familiarity, comfort, and other tangible and intangible things. For that reason I think it's much more useful and interesting to look at the things DocBook and DITA offer, what they have in common, where they really differ, and what they can learn from each other. That's mostly what I want to talk about in this presentation, though I promise to end with just a little bit of controversy so it isn't just the coffee keeping you awake. DocBook DocBook is more than 10 years old and is still actively maintained Historically, DocBook models traditional print publishing Tends not to be prescriptive Deeply committed to extension and customization Completely suitable for modular documentation One of the ways that DocBook and DITA differ is age. DocBook started in roughly 1991. My personal involvement extends back to about 1994 when the Davenport Group, the first official DocBook maintenance organization, was formed to guide its development. Back then it was, of course, an SGML DTD. Today it's an XML (or SGML) DTD and in the near future it will also be a RELAX NG Grammar. But back in the early nineties, SGML is what we had and development continued for several years until around 1996 or 97 when almost everyone involved turned their attention to developing something else. Can anyone guess what? The answer is XML. DocBook was developed by some of most respected markup specialists in the world. The overlap between Davenport Group members and the folks driving the development of “SGML on the web”, which would eventually become XML, is pretty remarkable. DocBook development stalled for a bit in the dot-com boom. I think we were all busy with other things. In order to make sure that it would continue to have a maintenance organization, and that it could continue to evolve to support the needs of it's ever growing user base, DocBook became one of the first OASIS Technical Committees in July, 1998. Our long history has made us think about maintenance issues that weren't always readily apparent when we first started. We have policies for, and experience with, managing change and maintaining stability and usability for a large and diverse user community. I imagine that DITA will find it has similar maintenance issues as time goes by. DocBook's history is firmly rooted in traditional techdoc publishing. Heck, we even managed to get “book” in its name. New techniques, like hypertext and topic-oriented authoring, are an evolutionary improvement over traditional words-on-papyrus and their descendants. They may even be revolutionary. But traditional, words-on-papyrus and their descendants have a successful history of knowledge and information transfer spanning back thousands of years. I don't think books are going away any time soon and I think DocBook has a lot of strength in that domain. That said, DocBook has supported lots of topic-oriented media for a long time including web pages, slides (like the ones you're seeing now), HTML Help, Java Help, and other formats. The ability to adapt to new authoring and publishing techniques, of course, requires a mechanism to support extension and customization. DocBook is perhaps the largest, most successfully parameterized public DTD ever created. There's almost nothing that you can't change in DocBook through the parameter entity mechanism. In fact the only thing I think you can't change are the actual names of the elements. I wouldn't go so far as to say that all of the changes you might like to make are equally easy with parameter entities. (As an aside, I'm happy to report that many of them are much easier with RELAX NG.) DocBook's commitment to customizability continues in the DocBook V5.0 efforts which are based on RELAX NG instead of DTDs. Once again, with the exception of the element names, we're working hard to make sure that the patterns defined in the schema provide a complete framework of extensibility points. Another aspect of extensibility is supporting authors who have different mental models of the artifacts defined in the schema. Sometimes these are cultural differences, sometimes they're a consequence of an author pushing the boundaries of the domain you had in mind when you wrote the schema, sometimes authors are just…well, let's just say “unique”. It isn't always a case of needing new or even different markup, sometimes it's a question of allowing elements where you didn't expect them. For example, DocBook used to have a fairly complex content model for the content of the “book” element. We tried to make sure that some elements came only at the beginning or the end, or in a particular relative order, or what-have-you. To be honest, I don't even really remember what we were trying to accomplish exactly. But at some point, it just became clear that it wasn't worth the effort. There was a lot of unexpected variety out there and we weren't really helping our users by forcing some of them to write customization layers just so they could put an appendix between two chapters or a colophon at the beginning of a book, if that's what they really thought they needed. I'm not saying there are no rules, of course, but the rules in DocBook tend to divide the elements up at a fairly course level of granularity. I think there's a natural tension between the desire to guide authors with fairly prescriptive markup and the need to support authors who have broader or different approaches to documentation. The bottom line is, if you start with a group of people as talented as I've had the pleasure to work with on DocBook, and you spend a decade iterating over a problem as interesting and challenging as documenting what computer hardware and software actually is and does, you develop a quite rich vocabulary. We have markup for concepts, for tasks, for reference materials, for publication details, for technical details, for programming language concepts and user interfaces, for metadata, bibliographies, glossaries, indexes, and many other things. If it's about computer hardware or software, we've probably thought about it. To some extent, whether you build books or topic sets or articles or reference pages or something else from this vocabulary is a separate question. Turning my attention to DITA, about which I do not claim to really be an expert, it seems to stack up this way. DITA DITA is younger (though it was certainly inspired by earlier work) Designed to support a topic-oriented authoring paradigm Tends to be prescriptive Deeply committed to extension and customization Topics can be presented linearly DITA is a lot younger than DocBook. I don't have an opinion about whether younger is better than older or vice-versa. But youth doesn't last, and we'll both be old eventually. I mention this difference only to point out that I think some of the “us versus them” tension I've seen may be motivated by the fact that DITA is shiny and new and exciting in ways that DocBook isn't anymore. And to point out that I don't think that fact has any interesting technical bearing on the problems you might be trying to solve. I think the topic-oriented authoring paradigm is DITA's outstanding feature. It's great because it attacks one of the hardest problems of writing reusable documentation: one of the non-technical problems. Let me explain what I mean. I used to do consulting. It was my job to go out to companies that were considering XML and explain how it was going to help them achieve the same goals many of you are probably here to learn about: how could XML help them deliver the same content in multiple ways (print, web, help, etc.) and how could it help them author content that could be reused in different ways (product catalogs, reference guides, tutorials, what-have-you). No two consulting gigs are exactly the same, but they often have strong similarities. The meetings I'm talking about now generally went like this: first, I'd try to figure out what the company actually did, then we'd talk about how they were currently doing documentation, then I'd try to get them to explain what they wanted to be able to do and, if XML was a good fit, I'd tell them a little bit about XML and how it had features that would help them achieve some of their goals. Then I told them the bad news. There are two kinds of problems, I would explain, that had to be overcome: there were technical problems and there were non-technical problems. The technical problems, I'd say, developing markup to identify the significant facets of their information, managing their content, combining it in different ways, and publishing it to different media were hard problems. But we could help them with those problems. We could sell them products and professional services, and we had teams of engineers trained to tackle those problems. Simple Matter of Information Architecture and Programming. The non-technical problems, I'd explain, were the fact that writing reusable document is different, often very different, from the way the authors had been writing. And authoring within a strict structure is different, often very different, from the way author's had been writing. And different means change and change is hard. In other words, training authors to write reusable documentation and training them to write correctly structured documentation, usually in a totally different kind of editing tool, is really, really hard. And there was basically nothing we could do to help them with that problem. I'm not sure the sales guys always appreciated my determination to tell the whole story. But I don't think we lost any sales because of it either. I think most folks appreciated the fact that we were being honest. But I think DITA really attacks the non-technical, writing reusable documentation problem head on. I don't know if it can do anything to soften the human propensity for discomfort when faced with change, but it's definitely a framework that encourages authors to write documentation in reusable modules. And that's really cool. At the same time, I'm not sure it's the right strategy for every kind of document.Topics give the author the flexibility to pull loosely coupled components together for different purposes. Writing in the traditional, linear fashion gives the author the ability to provide continuity for the reader. Pulling a set of very loosely coupled topics together into something that has a fluid, coherent end-to-end narrative that satisfies the readers expectations of continuity strikes me as seriously, really, very hard. Impossible, I think, except in the most unique of circumstances. Of course, I'm not saying that it's impossible to write excellent documentation using the topic paradigm, but I'm sure we've all experienced the pain of using a badly built system. Writing is hard work. On the one hand, you're attempting to satisfy the needs of the author, and the author's employer: to manage a large collection of information that changes over time and needs to be tailored quickly for different presentation formats, audiences, and purposes. And on the other hand, you're attempting to satisfy the needs of the reader: to find the information they need when they need it in a form that is physically and intellectually accessible to them. Those are sometimes conflicting requirements. Looking at DITA markup, DITA seems inclined to somewhat more prescriptive content models than DocBook. As I said before, I think there's a tension between prescriptiveness and flexibility. DocBook provides enormous flexibility. But it's possible that the prescriptiveness is comforting to new users who just want to know what to do. Anyway, exactly how to manage that tension is something you probably just have to figure out over time. It's also possible that the DITA specialization techniques will somehow mitigate that tension. I'll come back to specialization in a few minutes. Being a markup geek, I can't write a presentation about an XML-related topic without sticking some angle brackets in somewhere. Let's take a look at a little bit of markup. DITA Tasks Changing the oil in your car

Once every 6000 kilometers…

To change the oil:

Remove the old oil filter. Drain the old oil. … ]]> Here we see a DITA task, slightly abbreviated to fit on the slide. If you've downloaded the DITA toolkit, this will be a familiar example. It's a task for changing your oil. So let's look at the same task in DocBook. DocBook Tasks Changing the oil in your car Once every 6000 kilometers… To change the oil: Remove the old oil filter. Drain the old oil. … ]]> Remarkably similar, isn't it? What's different? The document element is article instead of task. I have a customization layer that has topic and task elements, for this presentation I chose to use plain, uncustomized DocBook. You can see from the namespace and the version attribute that this is DocBook V5.0. Other than that, the markup is pretty much what you'd expect. What about a concept? In DITA: DITA Concepts Oil

Motor oil keeps your car's engine running smoothly…

Changing the oil ]]> And in DocBook: DocBook Concepts Oil Motor oil keeps your car's engine running smoothly…

Related links Changing the oil

]]> The big difference here is the related links section. Where DITA has a specialized element for that, I've just used a normal section for this example. One question we could ask is, do I need a specialized element there. Would there be advantages or disadvantages? More broadly, what about this whole idea of specialization? Specialization Is it a good idea? Does it work? Are there better ways to do it? Remember when I promised to keep you awake with some controversy. We've reached that point now, so if you still have any dinner rolls left, now's the time to start warming up your throwing arm. I think specialization is really clever. But I don't think it's the panacea that some of the reports I've read suggest. And, with respect, I don't think it's instantiation in the DTD is very attractive. The marketing folks behind DITA products and consultancies have latched onto specialization as if it were some sort of silver bullet to slay the document interchange dragon. Interchange is a challenge, there's no question about it. The DocBook reference documentation has a chapter on interchange that includes a list of almost fifty questions that you need to work out with your interchange partners. And any particular interchange exercise will certainly generate it's own unique questions. They're detailed questions and answering them requires careful review of your documentation, your processing expectations, your formatting expectations, and your processing tools. Unfortunately, it also requires careful review of your interchange partner's documentation, processing and formatting expectations, and tools. A little web searching for “DITA specialization” will turn up plenty of hype that suggests it can replace that long list. I don't believe it. I'm not saying specialization isn't useful. The idea behind specialization, as I understand it, is that when I invent a new element, I declare what existing element it “specializes”. In theory, this declaration allows a tool processing my document to fall back to some default processing if it doesn't understand my specialization. The thing to remember is, the extent to which this fall back processing is useful or correct depends largely on the importance of the semantics of my new element. To the extent that you're creating a new kind of inline as a specialization of another kind of inline, or a new div-like structure as a specialization of another div, and I'm not saying these aren't common cases, it probably works pretty well. It's certainly easy to work up some use cases where it does what you want, but let's consider a more interesting case. Suppose, as is also often the case, that you want to add something with quite specific semantics. The example that I used when I wrote about this once before was the EBNF diagram. EBNF is short for Extended Backus-Naur Form, it's a way of representing grammars. EBNF [See EBNF diagram in DocBook 5.0: The Definitive Guide. There's currently an unresolved issue with formatting EBNF diagrams in this blog—ed.] If you're familiar with EBNF diagrams, that probably looks pretty familiar. If you aren't, well, don't worry about it too much. Exactly what it means isn't really important here. Without going into a lot of detail, although it's formatted something like a table here, the markup doesn't really have that structure. The markup is a set of productions. Each production has a “left hand side”, which appears before the “::=” and a “right hand side” that appears after. Right hand sides include both literals and non-terminals, that is, references to other left hand sides. There are also optional annotations and constraints. So here we see that an expression is either an arithmetic expression or a multiplicative expression, with an annotation about precedence, an arithmetic expression is a kind of expression, and a multiplicative expression is another kind of expression with a constraint regarding division by zero. The point is that there are a lot of semantics in there and it's non-trivial to devise a presentation that can convey those semantics. It strikes me as very unlikely that you'd be able to specialize those elements from any other existing elements. And if you did, suppose, for example, you made the left hand sides, non-terminals, and the like specializations of some inline and the set of productions (or maybe each production) a specialization of some division. Then you could send it to someone else, and they would produce some sort of rendering for it. But you'd be completely mistaken if you believed that they'd “successfully” handled it in any meaningful way. Now, it may simply be the case that most specializations are of topics, blocks, inlines and other sorts of elements where the fallback is appropriate. But that's not obviously the case to me considering the kinds of customizations people have used with DocBook over the years. Even in the case were this strategy is the right answer, I have reservations about the particular technical mechanisms used by DITA to achieve it. I think the long list of tokens in class attribute values in the instance is going to lead to some painful compromises later on. Concerns about specialization Long lists of class attribute values Constrained selection of validation technologies Well-formedness isn't enough Stylesheets and other XPath or XQuery-based technologies that operate on documents that rely on this technique are…atypical at best First off, using the class attribute to enumerate the specializations looks problematic. No one can actually be expected to put all those values literally in the instance and that means you're forced to rely on defaulted attribute values. Relying on defaults puts some constraints on schema validation, and schema validation tools, that I would find uncomfortable. You're basically stuck with either DTD validation or W3C XML Schema validation. (Although there's a RELAX NG annotations framework to support attribute value defaults, it's not a standard part of validation and it's not clear to me that it's widely supported.) So in the short term, you're stuck with DTDs or XSD. DTDs are going to hamper namespace support in serious ways, and you're eventually going to need namespaces, and simple XSD validation won't be sufficient, you'll actually need the PSVI for the defaulted attribute values. (XSD also isn't my first choice for defining grammars for documentation, but I guess that's a topic for another presentation.) Finally, stylesheets and other XPath or XQuery-based technologies that operate on documents that rely on DITA's class attribute technique are …atypical at best. Provocative words aside, and I did promise a little controversy, I think the specialization idea is pretty damn clever. It might be worth exploring further, in DocBook for example, but I don't think I'd want to de-emphasize the importance of an explicit interchange questionnaire and an extensibility strategy that allows implementations to detect customization failures. I've written about this a little bit before and I've been exploring how this might be accomplished in schema annotations, for example. It's all probably mostly irrelevant anyway. If experience with DocBook is indicative very few users are every going to make any customizations to the markup at all. Sure, some big companies will hire consultants to craft customizations for them, and some ambitious communities will craft them themselves, but they're in the minority. Most users are going to just grab the schema and start using it. So I've rambled around a bit offering some reflections on DocBook and DITA for a while now. I could probably go on for at least as long in any of several directions, but that's not practical, so I guess the real question is, what sort of conclusions can I draw before turning this over to some Q&A? Going forward DITA development may be informed by DocBook's historical stability and design patterns DocBook development may be informed by DITA's innovations and design patterns Some documents are topic oriented and some aren't Transformation may not be the only road to cooperation Cooperation is worth the effort Going forward, I think DITA's development may be informed by DocBook's historical stability and design patterns. Similarly, DocBook development may be informed by DITA's innovations and design patterns. Independent of how they're marked up, some documents are topic oriented and some aren't. It may simply be the case that some of your documents are better suited to narrative DocBook encodings and some are better suited to topic-oriented DITA or DocBook encodings. In other words, transformation may not be the only road to cooperation. Michael Priestley and I have also been discussing how some more dynamic mixing might be possible. No matter how we proceed, cooperation is worth the effort. That's the best I can do on one slide, I think. I hope those all make sense in the context of this presentation. And I hope we can do this again sometime because I really do believe that cooperation is the right strategy. Q&A Me: Norman Walsh, ndw@nwalsh.com This presentation: Other writings on DITA and DocBook: Thank you very much. You may fire when ready!