Accessing Flickr with XProc
I have a hammer.
The Flickr Uploader has never been a model of stability in my experience. Recently, it's become entirely unusable, it simply doesn't work for the current set of pictures that I want to upload.
There are lots of alternative uploaders, but I'm also interested in exploring the capabilities of XProc. Specifications and test suites aside, you don't really know what you can do with a language until you try.
Perhaps uploading photographs to Flickr isn't the sort of application best achieved by XProc, but I'm certain that interacting with real world web service APIs is something that XProc pipelines will need to do.
Uploading photographs requires authenticated access. As a prerequisite for that, you need to go through a little authentication dance with Flickr. I'm not going to try to automate that part; I'll assume you're in possession of an API key, authentication token, and shared secret. Authenticated access is achieved by “signing” the API call; in brief, concatentate the shared secret with the parameters (sorted into alphabetical order) and their values, compute the MD5 hash of that string, and add that value as a new parameter. Can we do this in XProc?
Sorting the parameters
is easy. The only
interesting part of that pipeline is the p:parameters
step.
It's there to turn what might possibly be a sequence of documents on the
parameters
port into a single document for XSLTIf you think using XSLT for the sorting part is “cheating” of some
sort, I don't agree. And if you think that I should just punt and do the
whole exercise in XSLT, I don't agree with you about that, either..
Before we dive into the signing step, there's one more complication.
To simplify processing for the caller, I'm going to allow the “secret”
to be either a parameter or an option. The Flickr APIs
of course don't allow the secret to be a parameter because that would expose
the secret which would defeat its purpose. But from an XProc perspective,
it's convenient to have a c:param-set
that contains the key,
token, and secret which we can just pass around together.
That means the signing step will have to be prepared for the secret to come from either place and will have to explicitly exclude the secret from the signed parameters.
Let's begin! Our pipeline, flickr:sign-api
takes a set of parameters and an optional secret
. It returns
the signed parameters as a c:param-set
on the result
port.
<p:declare-step xmlns:p="http://www.w3.org/ns/xproc"
xmlns:c="http://www.w3.org/ns/xproc-step"
xmlns:flickr="http://xmlcalabash.com/ns/extensions/flickr"
xmlns:cx="http://xmlcalabash.com/ns/extensions"
xmlns:err="http://www.w3.org/ns/xproc-error"
type="flickr:sign-api" name="main"
exclude-inline-prefixes="err" version="1.0">
<p:input port="parameters" kind="parameter"/>
<p:output port="result"/>
<p:option name="secret"/>
The first step is to run the parameters through a p:parameters
step, as discussed above, then we go looking for the secret. This is XProc,
so everything has to be XML. The result of the choose is an XML document that
contains the secret selected from (preferentially) the secret
option or from the secret
parameter. It's an error
to call this pipeline without passing a secret.
<p:parameters name="params">
<p:input port="parameters">
<p:pipe step="main" port="parameters"/>
</p:input>
</p:parameters>
<p:choose name="secret">
<p:when test="p:value-available('secret')">
<p:output port="result"/>
<p:string-replace match="/secret/text()">
<p:input port="source">
<p:inline><secret>@@</secret></p:inline>
</p:input>
<p:with-option name="replace" select="concat('"',$secret,'"')"/>
</p:string-replace>
</p:when>
<p:when test="/c:param-set/c:param[@name='secret']">
<p:xpath-context>
<p:pipe step="params" port="result"/>
</p:xpath-context>
<p:output port="result"/>
<p:variable name="value"
select="string(/c:param-set/c:param[@name='secret']/@value)">
<p:pipe step="params" port="result"/>
</p:variable>
<p:string-replace match="/secret/text()">
<p:input port="source">
<p:inline><secret>@@</secret></p:inline>
</p:input>
<p:with-option name="replace" select="concat('"',$value,'"')"/>
</p:string-replace>
</p:when>
<p:otherwise>
<p:output port="result"/>
<p:error code="err:XX01">
<p:input port="source">
<p:inline>
<message>flickr:sign-api called without secret</message>
</p:inline>
</p:input>
</p:error>
</p:otherwise>
</p:choose>
Next we sort them.
<cx:sort-parameters name="sorted">
<p:input port="parameters">
<p:pipe step="main" port="parameters"/>
</p:input>
</cx:sort-parameters>
Now we have a c:param-set
containing all of the parameters
in alphabetical order. From this we must construct a string that we can sign.
There's no two ways about it, string manipulation in XProc is not as easy as
XML manipulation. Here's what we're going to do: for each parameter (except
the secret
, if it's there) we're going to construct
a dummy document that contains the name and value concatenated together.
If we pass in <c:param name="foo" value="bar"/>
,
we'll pass back <doc>foobar</doc>
.
There's one interesting complication here. We're going to use
p:string-replace
to do the heavy lifting, but that will make the
context node the node that we're replacing. From that
context node, we'll have no way to get back to the c:param
that's
on the current
port. We work around this by storing the
values we need in variables.
<p:for-each name="loop">
<p:iteration-source select="//c:param[@name != 'secret']"/>
<!-- Flickr params should always be NCNames, but just in case... -->
<p:variable name="name"
select="if (contains(/*/@name, ':'))
then substring-after(/*/@name, ':')
else string(/*/@name)"/>
<p:variable name="value" select="string(/*/@value)"/>
<p:string-replace match="/doc/text()">
<p:input port="source"><p:inline><doc>@@</doc></p:inline></p:input>
<p:with-option name="replace"
select="'concat("',$name,'","',$value,'")'">
<p:pipe step="loop" port="current"/>
</p:with-option>
</p:string-replace>
</p:for-each>
The output of the loop
step is a sequence of
doc
documents. We'll turn that back into a single document
by adding a wrapper and we'll prepend the secret value to the
beginning of the document.
<p:wrap-sequence wrapper="inner-wrapper"/>
<p:insert match="/inner-wrapper" position="first-child">
<p:input port="insertion">
<p:pipe step="secret" port="result"/>
</p:input>
</p:insert>
We're getting there! Now we've got something that looks like this:
<inner-wrapper>
<secret>somehexvalue</secret>
<doc>api_keysomelonghexvalue;</doc>
<doc>api_tokenanotherlonghexvalue;</doc>
<doc>methodsomemethodname;</doc>
</inner-wrapper>
Recall your XPath semantics, if we ask for the string-value of that
document, we'll get the string we want hash. We have a p:hash
step,
so we can do that part. But if we replace the wrapper with the hash of it's
string value, the result won't be well-formed because it won't have a document
element anymore. We get around that by adding another level of wrapper first.
<p:wrap-sequence wrapper="wrapper"/>
<p:hash match="/wrapper/inner-wrapper" algorithm="md" version="5" name="hash">
<p:with-option name="value" select="string(.)"/>
</p:hash>
The output of p:hash
is a document that looks like this:
<wrapper>somehexstring</wrapper>
Now if we can stick that back in a c:param
element and stick that
parameter back into the c:param-set
, we're done. We'll use
p:string-replace
to put the string back into an element, but here
we encounter the first really odd wrinkle in the XProc design.
Like before, we need to put the hash in a variable so that we can get
at it when the context is changed by p:string-replace
. But variables
can only appear at the beginning of a compound step. Therefore, we have to introduce
a p:group
.
Note
WTF!? I hear you cry. Yes, well, look at it this way: the order of steps in the pipeline is determined by the connections between them, not strictly by the order in which they appear in the pipeline document. If you put a variable declaration between to steps and those steps get moved, where does the variable go? In order to answer that question, you have to know how it fits into the flow graph. To answer that question, you'd have to parse and analyze the XPath expression that defines its value because that variable's value might depend on the values of other variables.
That was more than we were willing to demand of implementations.
Instead, we simply limited variables to the beginning of compound steps.
That assures that ordinary step analysis produces a consistent result
for variables. Most of the time, it's what you want to do anyway. But
every now and then, you run into this weird situation where you have
to insert a p:group
. Imperfect, I agree.
C'est la vie. Here's the p:group
, the variable, and
the p:string-replace
call.
<p:group>
<p:variable name="hash" select="string(/)">
<p:pipe step="hash" port="result"/>
</p:variable>
<p:string-replace match="/c:param/@value" name="api_sig">
<p:input port="source">
<p:inline><c:param name="api_sig" value="@@"/></p:inline>
</p:input>
<p:with-option name="replace" select="concat('"',$hash,'"')"/>
</p:string-replace>
The last step is to combine this new signature parameter with the original input parameters. Remember that the user may have passed the secret in as a parameter, so we start by deleting that one.
<p:delete match="c:param[@name='secret']">
<p:input port="source">
<p:pipe step="params" port="result"/>
</p:input>
</p:delete>
Then we insert the signature into the resulting c:param-set
and that's what we send out the result
port.
<p:insert match="/c:param-set" position="last-child">
<p:input port="insertion">
<p:pipe step="api_sig" port="result"/>
</p:input>
</p:insert>
</p:group>
</p:declare-step>
The input parameters have been augmented with a signature that will satisfy the Flickr authentication requirements. Now we're read to move on to actually using the API with our signing pipeline.
In a future essay we'll look at making a “Flickr service” pipeline and an upload pipeline.
Comments
Laughing out loud, but if it works . . .