Implementing XProc, IIIhttp://norman.walsh.name/2007/05/13/implXProcIII10472007-05-13T09:29:33-04:00$Date: 2007-05-13 15:52:46 -0400 (Sun, 13 May 2007) $NormanWalsh2007Norman WalshPart the third, in which we consider looping.This essay is part of a series of essays about implementing an
XProc processor.
XProc: An XML Pipeline
Language is a W3C specification for specifying a sequence of operations
to be performed on one or more XML documents. I'm
implementing XProc as
the specification progresses. Elsewhere you'll find background
about
pipelines and other essays
about XProc.As a general rule, steps consume their input and produce output. Consider
this slightly contrived example:<p:pipeline name="pipeline"
xmlns:p="http://www.w3.org/2007/03/xproc">
<p:input port="source">
<p:document href="input.xml"/>
</p:input>
<p:output port="result"/>
<p:load name="loadStyle">
<p:option name="href" value="style.xsl"/>
</p:load>
<p:xslt>
<p:input port="source">
<p:pipe step="pipeline" port="source"/>
</p:input>
<p:input port="stylesheet">
<p:pipe step="loadStyle" port="result"/>
</p:input>
</p:xslt>
</p:pipeline>This pipeline takes an input document on its source
port (which defaults to input.xml if you don't specify a
source),
loads a stylesheet, formats the pipeline's source document
with the loaded stylesheet, and returns the result of that XSLT step on
its result port.Of particular interest is the interaction between the
p:load and p:xslt steps. The
p:load step produces some output and the
p:xslt step consumes it. (I've used a load step because I
want the input to come from a pipe in my next example; in practice,
you'd load the stylesheet directly in the p:xslt
step.)But what happens when we put the p:xslt
step in a loop? Consider
this example:<?xml version="1.0"?>
<p:pipeline name="pipeline"
xmlns:p="http://www.w3.org/2007/03/xproc">
<p:input port="source">
<p:document href="input.xml"/>
</p:input>
<p:output port="result"/>
<p:load name="loadStyle">
<p:option name="href" value="style.xsl"/>
</p:load>
<p:viewport name="format-sections" match="section">
<p:viewport-source>
<p:pipe step="pipeline" port="source"/>
</p:viewport-source>
<p:output port="result"/>
<p:xslt>
<p:input port="stylesheet">
<p:pipe step="loadStyle" port="result"/>
</p:input>
</p:xslt>
</p:viewport>
</p:pipeline>Now, instead of processing the entire document, we
process it one section at a time: the steps inside this
p:viewport get run once for each section in the
source document. What that
means is that the p:xslt step is going to read the
stylesheet several times.Unfortunately, the p:load is outside the loop. It produced
its output, closed up shop, and went home. Somehow, we have to make the
loaded stylesheet available more than once.A really clever implementation might cache the underlying
transformer so that the stylesheet didn't have to be read and compiled
more than once, but let's satisfy ourselves today with just being able
to reread the data.Inputs can come from three places: from p:inline elements
in the pipeline document, from URIs via p:document elements,
or through pipes from other components.The hardest problem is dealing with the pipes. I tackled this issue
by making it possible to “reset” a pipe. If the reset feature is enabled
for a pipe then the pipe keeps a copy of all the events in all the documents
that flow through it. A subsequent “reset” of the pipe tells it to begin playing
from the beginning again.This has to be managed with care. You can't enable the reset after you
begin writing to the pipe, naturally, and for the time being, it's an error
to attempt to write to the pipe after a reset. I'm not sure that's an absolutely
necessary condition, but I'm imposing it for the moment.A p:inline document is treated like a pipe, if the reset feature
has been enabled, it buffers the events before it sends them on. For
a p:document,
if the reset feature is enabled, the document is read into a buffer and treated
like an inline document.One practical consequence of this approach are that steps don't have
to care if they're in a loop or not. A step just does what a step does.
That's good. Another consequence is that pipes can wind up buffering
a lot of data. This is especially true in cases where
nested loops are involved. Given that StAXXMLEvents are no where near the most
compact representation imaginable for XML documents, I think it may become
necessary to allow pipes to dump buffers out to disk when memory runs low.
But that's a struggle for another day.