A very interesting exercise indeed. What should we take from it? Being an implementer myself, my first reaction was... mild terror: There is no way of stopping Norm from making Calabash the hottest XProc processor out there :)

My main problem with this is - and you made it clear several times in the text - interoperability. That you made this feature optional does not mean that people will not be using it. Once you give them a feature that will make certain things easier (or possible at all), I can't imagine they would resist, just because they should honour standards. And when that happens, other processors are out of the game. Of course they can then start a game of their own, but...

XProc is so much fun to implement - and to extend. I would be the first (or close second) to testify that: Calumet, our processor, is made to be extensible, and its API almost encourages programmers to write plug-ins to provide extension functionality. So I place myself at risk here, too. I have done my best to stay on the good side of the force by supporting only the "standard" ways of extending XProc (extension steps, support for additional URI schemes, etc.), but there is a certain danger that things might go awry if evil programmers really wanted to.

What you just showed is a way to extend or improve (some may think: fix) the core language itself. It makes Calabash a non-conformant XProc processor. Or, as you put it, a processor of a different language - but a language that happens to use the same namespace as XProc and thus some of the pipelines may run with other processors (sometimes they run fine, sometimes fail, and sometimes they produce different results). Interoperability hell.

Suppose the most likely - at least for me - scenario in which the WG agrees there is some merit in your proposal, but that things are not going to change for V1. In that case, is there a way out of this for Calabash? I think there is, and it actually based on standard XProc versioning mechanism.

The specification does not explicitly mention this possibility, but in theory, Calabash could invent its own (Calabash-specific) XProc version, something like: http://xmlcalabash.com/ns/extensions/xproc-1.5.xpl. You could then create the following pipeline:

<p:pipeline>
  <p:import href="http://xmlcalabash.com/ns/extensions/xproc-1.5.xpl"/>
  ...
</p:pipeline>

In Calabash, using this XProc version would mean that your "general-values" feature would be enabled. Other processors would reject this pipeline because they would not support this XProc version. Probably not the best solution, but in my opinion, still better than an ad-hoc command-line switch.

That's an interesting idea, Vojtěch. I was thinking that the pipelines ought to be non-conformant in some way too, to discourage attempts to use them interoperably. Rather than the import trick, I think I'm inclined to change the syntax in a different way, adding an "as" attribute to variable, option, and parameter elements. That'd be non-standard too, so other processors would reject the pipeline.

What I think will really happen though is that I'll turn the feature off again in a release or two. The overwhelming majority of pipelines don't need the feature and the last thing I want is a landscape where XProc aren't interoperable.

As we go forward, we'll see what users ask for and what problems they encounter. If we all have users who need the same thing, then we can use the exproc.org (or expath.org) venues to develop extensions cooperatively.

If XProc is successful enough and popular enough (and I think all the available evidence suggests it will be) to warrant a V.next, I think this feature is a very good candidate for standardizing in that version.

But let's get this version to Recommendation before we worry about it too much!

'end running' around options/params/vars is just an expression that these elements really do not go far enough to do the job for what people probably want/expect.

since you reopened this can of worms I will express how I would like to see things:

* rationalize options/params/variables into one element and keep them as strings ... these elements seem like a compromise

* allow input bindings to be accessible from inside steps who want to access them (p:xslt,p:xquery)

for example:

<p:declare-step type="p:xslt" xml:id="xslt" xmlns:xproc="http://xproc.net/xproc" xproc:support="true">

<p:input port="source" sequence="true" primary="true" select="/"/>

<p:input port="stylesheet" primary="false" select="/"/>

<p:input port="parameters" primary="false" kind="parameter" select="/"/>

<!-- I would allow this //-->

<p:input port="*" primary="false"/>

<p:output port="result" primary="true" select="/"/>

<p:output port="secondary" primary="false" sequence="true" select="/"/>

<p:option name="initial-mode"/>

<p:option name="template-name"/>

<p:option name="output-base-uri"/>

<p:option name="version"/>

</p:declare-step>

then you could just bring in xml as a binding

<p:xslt>

<p:input port="source">

<p:empty/>

</p:input>

<p:input port="name"/>

<p:input port="name2"/>

<p:input port="fragment"/>

<p:input port="stylesheet">

<p:inline>

<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="2.0">

<xsl:param name="name"/>

<xsl:param name="name2"/>

<xsl:param name="fragment"/>

<xsl:template name="cx:main">

<cx:doc>

<name><xsl:copy-of select="$name"/></name>

<name2><xsl:copy-of select="$name2"/></name2>

<frag><xsl:copy-of select="$fragment"/></frag>

</cx:doc>

</xsl:template>

</xsl:stylesheet>

</p:inline>

</p:input>

</p:xslt>

otherwise I am with Vojtech in that such behavior (esp now with the spec so precariously close) should reuse xproc reuse mechanisms.

interesting stuff though