For what it's worth, I'm appalled too. All the talk about escaping, and even double-escaping HTML markup in RSS some time ago on various RSS lists literally made me sick inside, and I had to take a long break from involving me in anything RSS-related. And now it looks like the same mistakes will be done all over again with Echo.
But to comment more on the technical side of things: As far as I can see, much confusion seems to stem from a a general lack of understanding of what entities (and entity references), characters, (numeric) character references, bytes and CDATA sections really *are* and how they work in an XML framework'. They're all just mechanisms for writing ordinary character data (i.e. text'). (And entities can also be used as a simple macro language for inserting frequently used text blocks easily.)
XML already *has* a well thought-out mechanism for mixing various XML vocabularies. It's called XML namespaces, and nothing could be simpler to use. To embed an piece of XHTML (or any other XML-based content language, e.g. MathML) in a XML document, such as a Echo document, you just include the relevant piece directly (or by parsing and serialising, to get rid of any unresolved entity references), and put a namespace declaration on it. Example:
<description><p xmlns="http://www.w3.org/1999/xhtml">This is <em>my</em> description</p></description>
(Since the content may span several paragraphs, it'll probably be a good idea to use the XHTML body' element as a wrapper element.)
While it's extremely easy to write, it's even easier to parse. All XML parsers support namespaces, and you just have to dump the contents of each element (XSLT example: <xsl:value-of select="description"/>) to have a nice, readable plain-text version (not all RSS/Echo clients will have a XHTML and CSS rendering engine included). And if the RSS/Echo client works by building a XHTML document, and then sending this to the default browsers, you just have to serialise the description' element again (an identity transformation in XSLT, and likely a one-liner in most XML tools/parsers).
Again, nothing could be simpler.
You write
"And if you're expected to recover from broken markup (an expectation that I thought was squashed once and for all in 1998), not only is it much harder still, it's a playground for all sorts of miscreants to devise trojan horses and other mischief."
but the source of the very same article, while sporting an XML doctype declaration
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
, includes around ten structural errors such as
<p>Again, nothing could be simpler.</div>
, which make it malformed.
Check http://snurl.com/1oze .
Preaching well-formed XML will work much better when written in well-formed XML :)
Tobi
P.S. XHTML, just as RSS, is being parsed, processed, transformed, filtered, indexed, etc.
Ah. Bleh. The article is well formed and, in fact, valid. But the talkback comments are not. I'll have to fiddle my CGI script a bit.
Norm
English is not my native language, so I might miss the meaning, implications, and finer nuances of "Ah." and "Bleh.".
I just thought I'd let you know that the page of the article at http://norman.walsh.name/2003/06/30/hardline is/was invalid (since not well-formed).
I hope you didn't take it as an offense, or nitpicking; the report was meant as helpful feedback.
I very much share both your opions: 1. Including escaped markup in RSS is not a good idea; this should be done via namespaces. 2. Everything that says it's XML should be well-formed (IMHO, even valid, eg in respect to some standard).
Tobi
No offense taken, Tobi, I appreciate the report. Most of the uglier problems were in the feedback comments. I've fixed the CGI script that includes them. I've also fixed a couple of other HTML bugs.
I believe all the pages are valid now.
The RSS feed for my utterly unimportant web site uses plain text descriptions (stripping out all mark-up) for just that reason -- I do not want to get emmeshed in the ambiguity of escaped HTML content.
My attempts to create RSS readers quickly have been thwarted, because I cannot process RSS in XSLT (my tool of choice) because it is escaped, and, often, not well-formed. My friends' RSS feeds are generated by a program that takes the first N characters of the HTML data as title -- often slicing a tag in half.
The debate between allowing (or even *requiring*) HTML to be escaped as CDATA is a symptom of the document vs. data views of XML. Having used the phrase "Echo document", naturally you expect the content to be part of the document. People who say "Echo datastream" think the content of the entry "is just data" and should be escaped...
There is a weird minority who think that doing escaping with CDATA (rather than entities) makes it more OK because a human reading the document will be able to see the mark-up as mark-up, even though the processing application cannot. IMHO, the processing application needs to see the mark-up more than human readers do!