Hi Norm, congratulations for a new WG which should create really needed spec. Do you know that ISO started work on the almost same topic as a part of their DSDL project?

http://lists.dsdl.org/dsdl-discuss/2005-10/0005.html

ISO WG should discuss this during their meeting which is usually scheduled to days preceding XML conference.

By making it an XML language:

1/ you make nearly impossible to embed it in an existing document (flat shell like with | and > could fit in a line in a PI)

2/ you ignore the many various existing scripting environment doing this, which are easier to type, understand and will be both more powerful and natural to people using them

The charter seems to be unambiguous about this, this must be XML and nothing else, see how the compact RNG syntax is so much nicer to programmers than its XML counterpart ! Putting a prerequisite something which should IMHO be the result of a test and requiring feedback sounds a bad start. Just for libxml2 there is 2 special purpose programming language (xed + xmlstarlet) and of course binding for most scripting languages. So there is a need... but nobody so far felt that an XML dialect would be of user interest.

Daniel

The language is vastly less interesting to me if I can't manipulate it with XML tools. I don't object in principle to a compact syntax, like RELAX NG's, but I definitely see it as a supplemental format.

Finally! For Xopus we've been waiting for a standardized pipelining language for years.

Did you also consider that with a few extension functions, XSLT 2 could be a pipelining language? But it's probably not the best solution with some important functionality being hidden in xpath expressions. Although it could be an idea to provide an XSLT transformation from your xml pipelining format to XSLT, which would make the semantics of the format unambiguously clear.

Btw, the charter is at this moment behind a username/password.

Limiting the language to be XML sounds exactly like trying to design the 'make' facility with the a-priori that to compile C code the make syntax should be C. Or to take another example that 'ant' syntax should be Java.

This is metadata, this is data, but having the same representation and syntaxic constraints as the targetted data sounds illogical to me. And I see no reason set forward anywhere for this a-priori decision.

If this is final, I will rather watch on the side the outcome of the WG rather than try to influence it, since obviously crucial and IMHO broken decisions have already be done.

Daniel

Great news on the new working group, glad to see this finally happening.

You could possibly consider Ant as a pipeline framework, there are certainly built-in XML processing tasks as well as extensions that support this usage. Currently I typically script XML processing pipelines using Ant. A typical scenario is having one task to generate a stylesheet from a meta-stylesheet, then apply the generatated stylesheet to one or more documents.

I've used this approach to build simple static websites (and also generate JSP pages), complete with multi-pass processing of the source documents with different stylesheets to generate indexes, etc.

I'd put Ant in the same general category as XML Pipeline in that its up to the processor to define the processing order based on some analysis of dependencies. Whereas Cocoon and other frameworks have a fixed processing order.

I made an abortive attempt to begin comparing pipeline frameworks back in 2002, some very brief notes are here: XMLPipelineFrameworks, including a comparison of XML-Pipeline and Cocoon.

Daniel, no decisions have been made. I expressed my opinion. Do you have a particular non-XML syntax in mind?

The charter is public now. Sorry about that, Sjoerd.

Hello,
That's a big news and since the time, we'll be waiting. May be to have a spec now that the idea of "what we should do with" is clearer is better than having a spec that we have to fix a lot.
So i would just mention that Serving XML gives a XML languages too and an implementation. I'm not involved in, but it's worth mentionning it.
Second, making an XML format is good. In fact, if he could stay as simple as XSLT 1.0, no matter. If he begin to be as obcure as XML Schema, the idea of Daniel, to give a compact format, is to take into account. Implementing it in Java is an obvious choice and so we could see how it will interract with SAX and DOM and the brand new StaX.
Because the fact is that a lot of tranformations would not be in XML syntax (XSLT, XQuery, XSP, STX, etc.), but given as programming language code.
To make short, very good news and i'm in.

Only the other day I was asked by a project if an XML 'pipeline language' existed, and had to mutter an answer with a pointer to your note, and I think I may have muttered something about using 'ant'. Now I can say "it's on its way"!

Norm, I don't have a specific syntax in mind, but I have a set of constraints in mind that an XML based language won't handle well:

1/ simple processing model should be expressable in a single line like a shell command with pipe

2/ being able to embed the expression on how to process a document in the document content itself (PI or comment)

3/ language must be able to express complex processing model

4/ the language should be easy to write and modify by an human

an XML format would fail 1/ 2/ 4/, the experience with XSLT does not sound good w.r.t. 3/ not that it blocks writing complex processing but rather it makes somewhat hard to maintain such complex code in an XML format. The main good points of an XML format from this use case perspective is I18N and not having to rewrite a parser. I don't see those processing model being automatically generated or maintained so the advantage of automated processing of an XML format can't compensate to 1/ 2/ and 4/ in my opinion.

Daniel

Congrats on the new working group. It sounds like Martin Bryan from the DSDL committee / ISO side is very interesting in helping with validation management which could be really great! Is a new processing model likely to mean a new XML Pipeline Definition Language? I mostly use ANT for pipelining similar to Leigh and think it must be a very common approach. I look forward to reading Sean McGrath comments on the new working group and hope he gets involved.

Back in 2000 I worked on SAX based pipelining/XML processing framework at Chrystal Software called Eclipse, see: http://hadleynet.org/marc/xml2000.pdf.