XForms over REST with Marklogic

Researching for an up and coming Marklogic project, I’ve been looking at the REST interface. I’d set one last month up on the laptop so that I could test whether we could push data in from our Apache Camel engine. That worked nicely and the thought occurred to me that I might be able to put an XForms interface directly over the top of the REST endpoints – a ‘one-tier’ application so to speak.

UntitledFor my demo, I took the idea of a herd book rather than the canonical name/address game.
For those who don’t keep animals, a herd book is simply a database of animals you have and what happens to them in terms of breeding, medicines etc. The very simplest ‘toy’ one  you might need for alpacas might be as shown here.

Setup: The Same Origin issue.

To set up for my experiment, all I did was alter my working REST interface in Marklogic admin, so that the ‘modules-database’ was on the file-system. That gave me somewhere tUntitledo put my XForm and the xsltforms library to drive it. I could of course have created a new Application Server on a different port, but that would have put the REST endpoint in a different domain and hence triggered same-origin security.

In a real application you’d use a proxy or CORS via headers or some such, but for this it was simply easiest to run it all together (this came to bite me later in searching though).

Creating documents via POST

First stop, now I had somewhere for my form to live was to get it to store a document. In REST, the endpoint for this lives at /v1/documents where you can PUT a document into the system at an address URI. However, that requires you to crete URIs and often you don’t really care what a new document is actually called, you just need a unique name. Marklogic has a nifty solution to this; you can POST a document instead and get your URI generated for you in a directory you provide (you can read about it here). In essence, you POST the data and you get a header back saying what it got called. Translated into XForms that gives this sort of <submission/>

 <xf:submission id="newanimal"
 resource="v1/documents?extension=xml&amp;directory=/my/herdbook/"
 method="post" replace="none" separator="&amp;" ref="instance('animal')">
 <xf:action ev:event="xforms-submit-done">
     <xf:setvalue ref="instance('control')/uri"
 value="substring-after(event('response-headers'),'=')" />
     <xf:message>
         <xf:output value="event('response-reason-phrase')" />
     </xf:message>
 </xf:action>
 </xf:submission>

Here I POST my form data, in ‘animal’ to REST and give it the target directory. I then use the ‘xforms-submit-done’ event to read the return header out via the event mechanism and put it in a ‘control’ instance, so I know the URI for editing. Success!

Updating documents via PUT

The simplest way to update a document is often just to write over it with a PUT to the same URI. That’s fine for documents if they’re small enough but it’s worth saying that the Marklogic REST interface also supports Partial Updates to both the document and the meta-data which might be better in some cases. Getting back to this form though, now I have a URI, if I update a document  I can simply replace the data with the following <submission/>:

<xf:submission id="upcase"
 method="put" replace="none" separator="&amp;" ref="instance('animal')">
<xf:resource value="concat('v1/documents?uri=',instance('control')/uri)"/>
<xf:action ev:event="xforms-submit-done">
    <xf:message>
        <xf:output value="'Updated'" />
    </xf:message>
</xf:action>
</xf:submission>

This is straightforward apart from the fact that I have to use an in-line <xf:resource/> element to construct the url with the URI I stored earlier. {Note to self. AVT?]

Searching documents

The last step in this little demo is getting lists of animals, searching etc. This bit took the most time due to a problem with how search works. In Marklogic, the modules-database gets used to store all the search options. Now this is great; you can pre-set a load of search templates, give them a name and put that in the search url in ‘options’. However, if you’ve hijacked the modules database for your code, well, you can’t. This is a pain because it doesn’t in my case allow you to override the snippet settings to get particular fields back. Now I thought that was that, until I found out that ML7 had the ability to send <options/> with a Structured Query. (thanks to @adamfowleruk).

Turns out all I needed for search was to POST a Structured Query format <instance/> using the accompanying <submission/> like so:

<xf:instance id="squery">
<search xmlns="http://marklogic.com/appservices/search">
<query/>
 <options>
 <return-metrics>false</return-metrics>
 <transform-results apply="metadata-snippet">
 <preferred-elements>
 <element name='name' ns=''/>
 <element name='dob' ns=''/>
 <element name='colour' ns=''/>
 </preferred-elements>
 </transform-results>
</options>
</search> 
 </xf:instance>

<xf:submission id="list all" action="v1/search?directory=/my/herdbook"
method="post" 
ref="instance('squery')" 
replace="instance" 
instance="results" 
omit-xml-declaration="yes" />

That gets me all the alpacas with their name,colour and dob so I can display them in a list. Note in this example the <query/> section is empty as I want every alpaca. In a real app, of course I could add something like a range query to get a particular animal linked to a search box. A bit of judicious code in some triggers and I can paginate using header data from the reply as well.

Untitled

Lastly, is a <action/> in the  list so that each row in the results triggers a fetch of that document so I can edit it if I want. Given the URI is in the search results, I simply need to copy it over to become my ‘current’ URI and send the instance using GET to /v1/documents. At the moment I send category=content (also the default) as I want the document, but I could expand that to category=meta easily to start working with the meta-data instead.

More to do?

UntitledThat’s pretty much it so far. I haven’t time currently to work this up into a full example at the moment, but it seems a very handy way to quickly put an XForms interface over REST, even if there are compromises, such as making the search a little harder.
Of course I’ve used Marklogic for this, but in theory it’s an approach that could work works with any REST system as long as it will take POST data in XML.
As to whether this is a viable way to write an application? I’d say probably yes for a prototype or something simple, especially as you get the REST endpoints thrown in for free.

Advertisements

XForms Push with MQTT over Websockets

The idea…

I’ve been looking recently at a project that creates pdfs via a ConTeXt typesetter. The front-end is XForms, feeding the backend queue of jobs that get done with a delay of perhaps a few minutes. The problem is how do I tell the web app, that a particular job has finished, because the user isn’t waiting; they’re on to another document.

What I need is some sort of Push Notification that I can integrate into the XForms world.

Of course within XForms it’s possible to do a sort of polling with native <xf:dispatch delay=”number”/> calling a <xf:submit/> to see if anything interesting has happened, but it really doesn’t scale. At that point you need a ‘push’ technology that will notify you when something you are interested happens instead. Handily in HTML5, there is scope for that in the use of WebSockets. These allow you to create a two-way link back to your server along which you can send/receive pretty much anything. Now there is nothing to say you can’t write your own system but  there are plenty of fine libraries out there that will do the heavy lifting for you.

In my case Pub/Sub was the way to go as a behaviour, and MQTT specifically as a protocol because it’s one of the best (if you don’t believe me ask FaceBook). Pub/Sub in brief is a message pattern (2lemetry have a great Nutshell). Publishers send data (of any sort) to one or more topics, e.g. house/temperature/lounge to a Broker (that’s the server bit). Subscribers, without any knowledge of who is doing the publishing, subscribe to the broker to receive data from one or more topics, either explicitly i.e. house/temperature/lounge or by wildcard (+ = 1 level, #= n level), i.e house/# or house/+/temperature.

The Broker mediates between the publishers and subscribers and makes sure messages get delivered. In my use-case, for instance, for a user Fred, working on a Case 00123, you might assume a topic tree something like fred/docs/00123.  When fred logged onto the web app, he’d subscribe to  fred/# and start receiving anything on that branch and down. The reason not to subscribe to say fred/docs/# is that it leaves the door open to using the same tech for something like system messages targeted at him via fred/messages or some way to lock a form via fred/locks/00123 or track form changes via fred/forms/00123 etc etc. Note: One of the nice benefits to doing it the MQTT way is that there’s nothing to stop another system subscribing to say +/docs/# to watch for problems or gather stats!

However, before any of those bells and whistles, perhaps I need to solve the basic problem of getting it to work at all via a small example. All code and runnable examples are as usual on GitHub.

Demo v1: Connect/Disconnect.

Untitled

click to run example.

First thing is a MQTT broker. The nice people at DC Square (@DCsquare)  have a public broker running at mqtt-dashboard.com we can play on based on HiveMQ which I’ll use in the example. Please be nice to their broker and use responsibly. Want to set up your own broker? The Mosquitto Open Source broker comes highly recommended. Second we need a Javascript library to talk to the broker. The Eclipse Paho project has one that is pretty good. To get it you’ll have to clone their repository like so:

git clone http://git.eclipse.org/gitroot/paho/org.eclipse.paho.mqtt.javascript.git

Lastly some code…. I’m going for the simplest xsltforms XForm with my MQTT connection details for the DC Square broker on it’s web socket port, and an initial topic to subscribe to (unused just yet) in the <xf:instance/>:

<xf:instance id="connection" xmlns=""> 
<data> 
<host>broker.mqttdashboard.com</host> 
<port>8000</port> 
<clientId></clientId> 
<initialtopic>xforms/chat</initialtopic> 
<status>disconnected</status>
 </data> 
</xf:instance>

Note the clientId is empty. I’m using a bit of xpath in the xforms-ready event to set this to a random value. This makes sure I don’t get kicked off the broker by someone connecting with the same id. (I’ll come back to this later). I’ve two trigger/buttons to connect and disconnect, which each call Javascript using the <xf:load><xf:resource/> method. Here’s the connect call as an example:

<xf:trigger id="mqttconnect">
  <xf:label>Connect</xf:label>
  <xf:load ev:event="DOMActivate">
    <xf:resource value="concat('javascript:mqttstart(&quot;',host,'&quot;,&quot;',
    port,'&quot;,&quot;',clientId,'&quot;)')" />
  </xf:load>
</xf:trigger>

Having to put in quotes around the params makes it a bit messy and it’s can be a pain. On bigger forms I tend to move this sort of JS stuff up into an <xf:event/> at the top of the form and call it, rather than spread it out through the code. Now I just need some JS to call. The main mqttstart() function sets up a Messaging Client with our parameters. The MQTT library does pretty much everything via callbacks, so most of the code is references to other functions.

function mqttstart(host,port,clientId) {
client = new Messaging.Client(host, Number(port), clientId);
client.onConnectionLost = onConnectionLost;
var options = {
timeout: 30,
onSuccess:onConnect,
onFailure:onFailure 
};
 client.connect(options);
};

For this version they’re pretty simple and I’m not interacting with the XForm the other way at all, i.e.

function onConnect() {
alert('Connected');
};
function onFailure() {
alert('Failed');
};
function onConnectionLost(responseObject) {
alert(client.clientId + " disconnected: " + responseObject.errorCode);
};

Only onFailure() is at all fancy and that’s just to get an error code. I’ll call this Version1 – here it is.

Version 2 : Feeding events back into the XForm.

Untitled

click to run example

Now I know the library works, I need to integrate the results from JS, back to my XForm. This is done by calling an <xf:event/> (note this call format is xsltforms specific) . I’ve got a generic JS function for this that takes the name of the event and a payload for the properties. The code is below, along with my new onConnect() function to call it.

Important! The JS side needs to know what the <xf:model/> id is for the call. I’ve hardcoded it here. Make sure it’s set on your XForm as well or it won’t work.

On the XForms side, I’ve now got the <xf:event/> that sets the connection status in my instance(). To add a visual cue, I’ve added a <xf:bind> to set the instance to readonly once connected  and reset if we disconnect:

function call_xform_event(xfevent,xfpayload) {
  var model=document.getElementById("modelid")
  XsltForms_xmlevents.dispatch(model,xfevent, null, null, null, null,xfpayload);
}
function onConnect() {
call_xform_event("mqtt_set_status",{status:  "connected"});
};
<xf:bind nodeset="instance('connection')" readonly="status = 'connected'" />
<xf:action ev:event="mqtt_set_status">
<xf:setvalue ref="status" value="event('status')" />
<xf:reset model="modelid" if="event('status') = 'disconnected'" />
</xf:action>

This is a much cleaner system than calling JS. I can set any number of properties on the JS side as a JSON string and they’ll appear as event properties on in the XForm.

Lastly, I need to add a subscription. In this case I’ve chosen xforms/chat.  To subscribe I have to wait to be connected and of course I could have done that at the JS level. However, this is about XForms so I’ve added a <xf:dispatch> to my ‘mqtt-set-status’ event to call another event to do it for me if I’m connected.

 <xf:dispatch name="mqtt_sub_first" targetid="modelid" if="event('status') = 'connected'" />

Next, I need to add support for messages arriving from this subscription. This is done by adding another callback into mqttstart() for onMessageArrived and then fleshing it out so that the message and it’s topic can be sent into my XForm:

function onMessageArrived(message) {
  dispatch_xforms("mqtt_message_arrived",{
    message: message.payloadString,
    topic: message.destinationName
    }); 
};

On the Xform side, I now need the event code. I’m going to drop all the messages into a new top-down “messagelog” instance. So it doesn’t escape off the screen, I’ve limited the log to 15 rows before I prune the bottom row. This simply uses a <xf:repeat> to output onto the screen.

<xf:action ev:event="mqtt_message">
  <xf:setvalue ref="message" value="event('message')" />
  <xf:insert nodeset="instance('messagelog')/event" at="1" position="before" />
  <xf:setvalue ref="instance('messagelog')/event[1]" value="concat(now(),' : ',event('topic'),' : ',event('message'))" />
  <xf:delete nodeset="instance('messagelog')/event[last()]" if="count(instance('messagelog')/event) &gt; 15" />
</xf:action>

You can try out v2 here. Once you connect, you’ll see, hopefully, the classic “Hello”. This is what is called a Retained Message and is sent to anyone subscribing as a default.  If you now go to the mqtt-dashboard.com site and use their Publish option, you should be able to publish anything you like to the xforms/chat topic and see it appear on the XForm – neat!

Version 3 : Publishing.

Untitled

Click to run example

We’re almost there. The last thing is how to publish a message to the broker. That’s actually the easiest bit of all. All I need is another instance to hold the topic and the message, so we can put it in the XForm, along with a <xf:trigger/> to call a JS function with the two values to send it. However, I can make it a little bit more interesting by talking about adding QoS and CleanSession.

QoS, or Quality of Service, is a flag that gets sent with a message or a subscription that sets how hard the broker and client work to handle your message.

At QoS 0, the default, the client deletes the message as soon as it’s sent to the broker. The broker in turn only sends it on to clients that are connected – it’s fire and forget.

At QoS  1, the client keeps the message stored until it knows the broker got it. In turn the broker keeps a copy until not only those connected get it, but those not connected currently but subscribed as well – you’ll get the message at least once, but might get a duplicate if the  broker/client aren’t sure.

At QoS 2 the same happens at QoS 1, with added actions by the broker/client to make sure you only get the message once.

So how does the broker keep track? It uses the clientId as the key to handle messages and subscriptions for a given client. When the client connects, there is a flag that tells the broker to either carry on or reset all messages and subscriptions for this client – that’s cleanSession and it’s true by default.

It’s worth knowing about since for a practical system you’d want cleanSession to be false and have a fixed clientId. Also, you’d probably end up with your messages being transferred at at least QoS 1. As they’re just flags it’s easy to add to the XForm for experimentation as version 3

Wrap Up.

It’s been fairly easy to integrate XForms with MQTT as it is with most JS libraries. Now I’ve a working example I should be able to generate my notification system fairly simply. Pub/Sub is an great technology and opens the way for some interesting form/form and form/server interactions.

Resources.

  • MQTT.org All things MQTT and where to go for libraries and brokers.
  • Eclipse Paho. M2M and IoT protocols. Home to the reference MQTT clients.
  • XSLTForms. Great client-side XForms engine.

 

Keyboard events in XForms

A couple of weeks ago I came across a post by Stephen Pemberton on Mouse Events in XForms. It gave me the idea about capturing “key” events to enable my own forms to be a bit smoother. So, perhaps get the Esc key in an <xf:input/> to delete the contents, or Return to trigger a <xf:submission/>. Another was whether I could get the arrow keys to work in a <xf:repeat/>. Now I hasten to add, fiddling with the key events can break forms used on other devices like phones. In my case I’m only dealing with browsers in an office, so your milage may vary but the general idea may be of use.

Untitled

So, over a coffee, I put together a little demo on GitHub using XSLTForms 1.0 that you can run here.
I used Wikipedia as a source of my data, with a hat-tip to Alain Couteres  for his OpenSearch demo showing XSLTForms’ neat JSON handling that I used to get the submission working.
If you fire up the example, you’ll see that you can use Esc and Return in the search box, and if you click into the results, you’ll see you can use the up and down arrows to move in the list. If you get to the bottom of the list and use down-arrow again, you’ll trigger the next page. Up-arrow does the same for prev page. This is how it works:

First of all there is a little bit of XML and a <xf:submission/> that gets data from wikipedia using their api.
Note: Use text/jsonp as the media type. You need it so xsltforms can do it’s magic.

<xf:instance xmlns="" id="query">
 <data>
 <action>query</action>
 <list>search</list>
 <srprop>snippet</srprop>
 <srsearch>xforms</srsearch>
 <format>json</format>
 <sroffset>0</sroffset>
 <srlimit>10</srlimit>
 </data>
</xf:instance>
<xf:submission id="search" method="get" replace="instance"
 mode="synchronous" instance="results" ref="instance('query')"
 action="http://en.wikipedia.org/w/api.php" mediatype="text/jsonp"> 
</xf:submission>

Now, <srsearch/> is the field we’re querying for, so that’s bound in as an input. Normally, I’d use something like so:

<xf:input ref=’srsearch’ incremental=’false’>
<xf:send submission=”search” ev:event=”DOMActivate” />
</xf:input>

This gives you an input box that submits if you leave the field, press return etc. Also, normally there’d be a button as well, just in case. To alter this state of affairs, all I need to do is wrap the control I want to capture in an <xf:group/> and add some <ev:event/>s.
Note: You can’t put your event code inside the <xf:input/> which you’d think would be the place for it – it has to be wrapped to work.  The input now looks like this:

<xf:group ref="instance('query')" navindex="0">
<!-- ESC Key -->
 <xf:action ev:event="keydown" if="event('keyCode') = 27">
 <xf:setvalue ref="srsearch" value="''"/>
 </xf:action> 

 <!-- RETURN -->
 <xf:action ev:event="keydown" if="event('keyCode') = 13">
 <xf:dispatch targetid="searchme" name="DOMActivate"/>
 </xf:action>

 <xf:input ref='srsearch' incremental='false'/>
<xf:trigger id="searchme">
 <xf:label>Search</xf:label>
 <xf:action ev:event="DOMActivate">
 <xf:setvalue ref="instance('query')/sroffset" value="0"/>
 <xf:send submission="search" />
 </xf:action>
 </xf:trigger>

 </xf:group>

If you’re in the input , Esc will delete the contents and Return will trigger a search.  It’s worth noting that the key event is captured regardless and then the if statement acts to control the action, not the other way around. Since I’ve added a ‘Search’ <xf:trigger> now, I can use <xf:dispatch/> in my Return code to send the DOMActivate event to it. That way I don’t have to duplicate the submission code. It’s a great way to say, put prev/next buttons at the top and bottom of a results list, and just have the code in one set.

Next up is the results. I’ve use a standard <repeat/> with a CSS table layout again wrapped in a <xf:group/> with a couple of buttons to page through the results. Since the snippet element comes back as encoded html with the search term highlighted, I’ve used the media type flag in <xf:output/> to display it properly.

As with the input, I’ve caught the keys I want. For example, down-arrow:

<xf:action ev:event=”keydown” ev:defaultAction=”cancel”
if=”event(‘keyCode’) = 40″>
<xf:dispatch targetid=”list_next” name=”DOMActivate” if=”index(‘resultslist’) = 10″ />
<xf:setindex repeat=”resultslist” index=”index(‘resultslist’) +1″ if=”index(‘resultslist’) &lt;= 9″/>
</xf:action>

There are two actions here, driven by appropriate ‘if’ statements. The first calls the Next <trigger/> if you’re at the bottom of the list, i.e. index() = 10. The other increments the list index ‘if” you’re not at the bottom, i.e. index() <=9. Capturing up-arrow simply works in reverse.

There are a couple of things to note here:

  • The use of ‘navindex=1’ in the <xf:group/>. This makes it possible for the list, which isn’t a control, to get keystrokes. It won’t work without it.
  • Each <xf:event/> has ev:defaultAction=”cancel” set. This stops the event from propagating. Be careful if you use this with a input control as normal keys you don’t handle will get cancelled too.

That’s pretty much it and I hope it’s been useful as even if you don’t capture keys, it’s handy to be able to send events to <xf:trigger>s as they’re used so frequently. Enjoy.

DemoJam XForms with interactive Map

At the end of a very enjoyable day at XMLAmsterdam a few weeks ago, there was a DemoJam where you could demo something for 5 minutes. In the spirit, and not because my teenage son wanted an iPad mini and that was coincidently the first prize, I brought along a little demo of a solr query linked to a map rendered in an XForm.

The very worthy winner was Marc van Grootel with his 1:30s video “XML Rock Star“. If you haven’t seen it, go look now – you’ll thank me, and then come back

My demo started out as simple XForm which talked to a SOLR instance to do faceted search on some project data. I used the excellent XSLTForms (rev 580). Initially I returned a list, but one thing that came out of demo’ing was that for things like the projects, which had a location, it was easier to look at a map.

Untitled

It would have been pretty easy to replace the list with a map as an output, but I wantedsome way to make the map part of the search. If you zoomed out you should get more results and if you scrolled results outside the visible map should be excluded. Also, it should work the other way; if you used a facet or a search term, the map should scale to show those results, so the map should be a control and a display.

On the XForm side, the SOLR search is controlled by a ‘query’ instance and results are poured into a ‘results’ instance. The initial query is below:

<xf:instance id="query" xmlns="">
<data>
<q>id:*:*</q>
<facet>true</facet>
<rows>1000</rows>
<sort>id desc</sort>
<start>0</start>
<facet.field>trust_t</facet.field>
<facet.field>area</facet.field>
<facet.field>status</facet.field>
<facet.mincount>1</facet.mincount>
<fq id='0'>location:[50.84757295365389,-5.09765625 TO 54.18815548107151,2.373046875]</fq>

There are three facets and a facet-query <fq/> set to the whole of the UK. This is the one linked to the map, and hence the only one with an id. (The form inserts other <fq/>’s in and out depending on what users selects).

Inside the <model/> there is a custom <event/> triggered by JS on the map side. Note for this to work, your <model/> must have an id. The <event/> updates the filter to the new bounds of the map (i.e. the current ‘corners’) and re-submits the search. All made possible with XForms 2.0 which allows properties in events and explained to me patiently by Alain Couthures.

<xf:action ev:event="callbackevent">
 <xf:setvalue ref="instance('query')/fq[@id='0']" value="concat('location:[',event('latitude'), ' TO ', event('longitude'), ']')"/>
 <xf:send submission="solr"/>
 </xf:action>

The submission calls SOLR with the current query and gets the result set. If there are no problems it calls some JS to re-plot the map from the current results:

<xf:submission id="solr" method="get" replace="instance"
 instance="results" ref="instance('query')" action="http://localhost:8080/lux/collection1/select">
 <xf:action ev:event="xforms-submit-done">
 <xf:load resource="javascript:rePlot()"/>
 </xf:action>
 </xf:submission>

Now to the JS side. It’s a simple thing to add Leaflet.js maps to an XForm; tag a div and call the JS libraries at the end of the <body/> as per their examples. We’ll need a bit of custom JS (only a little bit) called when we move the map or zoom:

function getViewPort() {{
    //Get current corners
    var bounds=map.getBounds();
    var minll=bounds.getSouthWest();
    var maxll=bounds.getNorthEast();
    //Call xsltforms model named 'test' and hence <event/>
    XsltForms_xmlevents.dispatch(document.getElementById("test"),
        "callbackevent", null, null, null, null,
        {
            latitude: minll.lat + ',' + minll.lng,
            longitude: maxll.lat + ',' + maxll.lng
         });
    }
 }

That’s pretty much it. There’s more code on the JS side to call the function when the map moves or zooms and to populate the map from the results; handled very very poorly in the demo by a hidden <ul/>.

This is very much a work in progress but it does expose an interesting technique for interacting between JS and XForms layers. Another point is that it’s also a fix for the issue of not being able to use <xf:load/> with a callback. For example, the geolocation API in HTML5 to get current position could be rendered like so:

<script type="text/javascript">
  function asynch() {
    navigator.geolocation.getCurrentPosition(show_map);
  }
  function show_map(position) {
    XsltForms_xmlevents.dispatch(document.getElementById("modelid"),
        "callbackevent", null, null, null, null,
            {
                latitude: position.coords.latitude,
                longitude: position.coords.longitude
            });
  }
 </script>

If you want to download the full code, it’s available on GitHub . However it won’t work out of the box as it needs the SOLR data. It should be amenable to any similar data though with minor mods. Have fun.

All XForm sites

A few months ago I was looking at an XForm-based menu for a project. Since then I’ve been thinking about the idea of creating a complete site using nothing but the xform system. It isn’t as daft as it might sound at first glance, many sites are basically an initial page with subsequent content AJAX’d into them via something like JQuery. I’ve never been very comfortable with this design and the disjoint between content and the code intercepting events: and not being a fan of JS, perhaps I’m more motivated.

The Simple version

So, in the XForms world, how does this work? The simplest way is to have two instances     in the form: a) One for the menu, b) One for the body data.

 
<xf:instance id="bodydata" xmlns="">
<data>
<title>PlaceHolder</title>
<content>This is the placeholder content</content>
</data>
</xf:instance>

<xf:instance id="navdata" xmlns="">
<data>
<item><title>Home</title><id>1</id></item>
<item><title>Journal</title><id>2</id></item>
<!-- Other menu items -->
</data>
</xf:instance>

<xf:submission id="get-data" method="get" replace="instance" 
instance="bodydata" 
ref="instance('navdata')/item[index('nav-repeat')]/id" 
action="query.xql"/>

Now we can construct a basic site where where the current content of the data instance is <xf:output> into the html. I’m using the mime-type “xhtml+xml” so that it flows with the rest.
The navigation is an <xf:repeat> of <xf:submission> items styled as links using ‘minimal’ appearance. These call a submission event that pulls html into the bodydata instance. Note that since I’ve only got one navigator to call the submission from, I can put the choice logic into that submission, rather than have a separate node with the current value in and having to use the slightly more wordy <xf:trigger> construct.

<div id="navigation">
<ul>
    <xf:repeat nodeset="instance('navdata')/item" id="nav-repeat">
        <li><xf:submit submission="get-data" appearance="minimal">
            <xf:label><xf:output ref="title"/></xf:label></xf:submit>
        </li>
    </xf:repeat>
</ul>
</div>

<!-- The content -->
<div id="content">
<h3><xf:output  ref="title"/></h3>
<xf:output ref="content" mediatype="application/xhtml+xml"/>
</div>

Now, there are obvious issues with this sort of approach:

  • We can’t easily put references to other content in the data. That’s because they wouldn’t be part of the form, just simple links.
  • If we only want to pull in documents, that’s ok. But what if we wanted lists, searches, editing etc, that simple instance is going to get very complicated, or we’re going to end up with a sea of <xf:switch> statements and a load of redundancy.

Luckily, there is at least a partial solution: the Sub-form.

THE SubForm version

Sub-forms allow you to load a complete form into another form. The mechanism is as far as I know due to be in XForms 2.0, but there are several engines that support it in some way already, such as XSLTForms and BetterForm. The basic mechanism is that instead of a submission, you use <xf:load> with the ’embed’ directive and the id of an <xf:group> in the parent. This gets replaced by the subform. Now what is cool is that the sub-form can either interact with the parent model or bring in a model of it’s own with it’s own instances and use that instead (as can the parent).

Armed with the sub-form idea, we can look at our site again and bring in a bit of MVC, i.e the idea of a view. So, imagine that we have a List sub, a Search sub, a View sub, a Edit sub etc, all forms in their own right, pulled into the parent form as needed. For instance we could use a search field in the parent to trigger a <xf:load> to pull in the “Search” sub form. This could have not only the results in it’s own model/instance, but also it’s own submission buttons for <next> <prev>. That’s a great modular approach and of course each sub-form can be tested on it’s own.

So, now we can alter our little site.  We don’t need our data instance now, but we need a destination for the sub-form instead:

<!-- The content -->
<div id="content">
    <xf:group id="subform/>
</div>

We need to change the navigation, so we’ll set up a ‘view’ sub-form instead and some way to be able to send it parameters (the document we want) in the <xf:load>.
First step is replace our <xf:submit> items in the menu with the following:

<xf:trigger appearance="minimal">
<xf:label><xf:output ref="title"/></xf:label>
<xf:action ev:event="DOMActivate">
<xf:load show="embed" targetid="subform">
<xf:resource value="concat('view_doc.xql','?id=',id)"/>
</xf:load>
</xf:action>
</xf:trigger>
Notice that I’ve had to use <xf:resource> as well so I can calculate what id I need as you can’t do that in an <xf:load> “ref” field via concat() as you might imagine.
The XQuery-based subform, view_doc.xql is pretty simple. I’ve used a private instance to hold the data (in some ways this is very much like using sub-routines in a programming language with global and local variables) and hence I’ve had to use another model. Note that I then need to say what model I’m getting my data from in the subsequent <xf:output> or it will try and find it in the parent.
xquery version "1.0";
declare option exist:serialize "method=xhtml media-type=text/xml indent=no process-xsl-pi=no"; 

let $data_collection := '/db/projects/project1/'
let $q := request:get-parameter('id', '1')
let $file := concat($data_collection,$q,'.xml')

let $error :=
<document><title>Oops!</title>
<content><p>I can't find {$file} in the database</p></content>
</document>
let $input := if(doc-available($file) = true()) then (doc($file)) else $error
let $form :=
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:ev="http://www.w3.org/2001/xml-events"
xmlns:xs="http://www.w3.org/2001/XMLSchema"
xmlns:xf="http://www.w3.org/2002/xforms">
<head>
<xf:model id="sub">
<xf:instance>{$input}</xf:instance>
</xf:model>
</head>
<body>
<h3 class="post-title"><xf:output model="sub" ref="title"/></h3>
<xf:output model="sub" ref="content" mediatype="application/xhtml+xml"/>
</body>
</html>
let $xslt-pi := processing-instruction xml-stylesheet {'type="text/xsl" href="/resources/xsltforms/xsltforms.xsl"'}
let $css := processing-instruction css-conversion {'no'}
let $debug := processing-instruction xsltforms-options {'debug="no"'}
return ($xslt-pi,$css,$debug,$form)
It might seem a bit verbose to do this for every view, but of course in the real world I’d template it or use XSLT to do the repetitive bits.
Lastly, we need some way to load initial data into our site. Since we now have a sub-form for showing the data, it makes sense to use that; all we have to do is put a suitable event in the model.
<xf:action ev:event="xforms-ready">
<xf:load show="embed" targetid="subform" resource="view_doc.xql"/>
</xf:action>

Thoughts and Next Steps

It’s possible to do a site as a complete XForm. Using sub-forms makes it feasible, though performance might be an issue and there are issues still to resolve, such as linking in the data. It might turn out that this is simply a good way to design a site and how it behaves.

Of course at the end of the day, in a browser, it becomes the slew of html and js I’ve been avoiding, but at least I didn’t have to write it. So, lets be honest, this isn’t a mainstream technique, but I quite like it. So, I’ll continue my experiments, create more views for listing and editing etc and see how the end result turns out.

Drill-in menu with XForms

As part of an XForms project I’m doing I’ve been looking into menu options and came across the drill-in menu, popularised by a famous range of mp3 players. The menu comes in two parts, a history section at the top and current choices below. As each selection is made, that choice is added to the history and it’s children appear in the menu.

The advantages here are that it’s a compact mechanism for use with any deeply nested structure. Only the current menu is shown at any time and the history gives a convenient breadcrumb trail to that choice-set. If you don’t want to use a tree, linked lists or fly-out, drill-in might be for you. Luckily, it’s fairly easy to do as an XForm and sits nicely on my SimpleXist server.

Note: I’m using XSLTForms here to do my XForms and the SimpleXist set-up described elsewhere. The code expects /resources/xsltforms in the root. 

Data Data Data

The source data for the menu is a standard nested piece of XML. It could be in one piece, a collection, whatever. The important factor is that there is some mechanism to uniquely target each menu item. In my case, I’m using menu.xml and a numeric id; I’ve set the root item to ‘000’.

Next step is some XQuery to put up the menu. I’ve taken the decision to send the whole menu with each submission and get back a complete one to replace the instance. I’m using /db/projects/project3 in the SimpleXist idiom for the data. Here’s menu.xql:

xquery version "1.0";
declare option exist:serialize "method=xml media-type=text/xml indent=yes";
let $data_collection := '/db/projects/project3/'
let $data := request:get-data()
let $q := if (empty($data//prefix)) then ('000') else ($data//prefix)
let $file := concat($data_collection,'menu.xml')
let $head := doc($file)//category[id=$q]
let $this_item := <item id="{$q}" children="{count($head[1]/category)}">{$head[1]/title/text()}</item>
let $hist := $data//history/item
return
<data>
 <prefix/>
<history>
{$hist}
{if ($this_item/@children > 0 ) then ($this_item) else ()}
</history>
<menu id="{$q}">
 {for $z in $head/category
 return <item id="{$z/id}" children="{count($z/category)}">{$z/title/text()}</item>}
</menu>
</data>

The key points here are that that the submission posts the whole menu. The prefix element contains the choice or ‘000’ as default. The new menu is put into $head and $this_item set to the chosen item. Then $history is set up to the current history. Lastly, a new menu is assembled: $this_item is appended to the history if the current choice has a child menu, then all the menu items for this choice are assembled along with a count of children. This last is so the form can display and behave differently for “branches” and “leaves” in the form. The default output looks like so:

Note that the menu isn’t in the format of the original menu; that’s deliberate. The XForm is wrapped around the menu and would be harder to change than the original data. With a bit of XQuery, pretty much any tree data could be coerced into this format.

Simple XForm

Now we have data it’s time to set up the XForm. It’s pretty simple at the moment with one instance to hold the current menu, and one submission to get the next one as shown by drill.xql the meat of which is below:

<head>
<xf:model>
<!-- Instance to hold the menu -->
<xf:instance id="navigation" src="menu.xql" xmlns="" />
<xf:submission id="nextmenu" method="post" 
replace="instance" instance="navigation" 
ref="instance('navigation')" action="menu.xql"/>

</xf:model>
</head>
<body>
<div id="drillin">
<div id="history">
<!-- History Bar -->
<xf:repeat id="hist-repeat"
  nodeset="instance('navigation')/history/item">
<!-- Output as a set of links -->
<xf:trigger appearance="minimal">
<xf:label><xf:output value="position()"/> <b><xf:output ref=".  "/></b></xf:label>
<!-- When we're clicked .... -->
<xf:action ev:event="DOMActivate">
<!-- Rather long-winded way to put current choice in prefix -->
<xf:setvalue ref="instance('navigation')/prefix" 
  value="instance('navigation')/history/item
  [index('hist-repeat')]/@id"/>
<xf:delete
  nodeset="instance('navigation')/history/item[position() &gt;=
  index('hist-repeat')]"/>
<xf:send submission="nextmenu"/></xf:action>
</xf:trigger>
</xf:repeat>
</div>

<div id="menu">
<!-- Menu Items -->
<xf:repeat id="nav-repeat"
  nodeset="instance('navigation')/menu/item">
<xf:trigger appearance="minimal"> 
<xf:label><xf:output value="if(./@children = 0,'>','+')"/> 
<xf:output ref="."/></xf:label>
<xf:action ev:event="DOMActivate">
<xf:setvalue ref="instance('navigation')/prefix"
  value="instance('navigation')/menu/item[index('nav-repeat')]/@id"/>
<xf:send if="@children !=0" submission="nextmenu"/>
</xf:action>
</xf:trigger>
</xf:repeat>
</div>
</div>
</body>

The history div holds the history data as a list of links via a repeat. Position() is used so these get numbered as there is an implied order. There is a trigger hooked into the selections which does two things:

  1. It sets the prefix element to the id of the selected item.
  2. It deletes any item below the selected one in the history.

The deletion is necessary as it lets the history react properly if you select an item in the middle; everything ‘later’ is deleted. Then the whole menu is submitted.

The menu div hold the current menu options, again as a repeat. There is a bit of code in the label to put a ‘>’ or ‘+’ in front of the data depending on whether there are children or not. On selection of an item, again the whole menu is submitted, but only if that menu has children.

A little bit more complicated

Now the basics are out of the way it’s time to put together something a little more interesting. If you want this version feel free to download it here. Initially, I was going to use this menu to pick categories, then I got the idea to use it for navigation instead. Assuming for the sake of argument that a menu item points to some resource NNN.xml, the leaves can now do something useful. Lets start with 000.xml in the database for the root item:

<content>
 <h4>Hello</h4>
 <p>I'm some content</p>
</content>

We’ll need a bit more XQuery, query.xql,  to get a given resource from it’s id. Again  the menu gets sent:

xquery version "1.0";
declare option exist:serialize "method=xml media-type=text/xml indent=yes";

let $data_collection := '/db/projects/project3/'
let $data := request:get-data()
let $q := if (empty($data//prefix)) then ('000') else ($data//prefix)
let $file := concat($data_collection,$q,'.xml') 
let $tail := $data//item[@id = $q][1]
let $strpath := string-join(for $t in ($data//history/item,$tail)
return $t,'/')
let $error :=
<content>
<h2><a name='h1'>Oops</a></h2>
<p>I can't find {$file} in the database</p>
</content> 

let $input := if(doc-available($file) = true()) then (doc($file)) else $error

return
<data>
<meta>{$strpath}</meta>
<content>{$input}</content>
</data>

There is an error message as of course most of the items won’t exist – feel free to add some! One handy additional feature we can get from having the whole menu is a breadcrumb via the history made up of each item, plus current separated by a ‘/’ and put in the meta element. Of course it could just as easily hold other things like creation date etc.

We now only need to be able to tell the XForm menu section to get the resource if it’s a leaf, otherwise the next menu.

<xf:send if="@children != 0" submission="nextmenu"/>
<xf:send if="@children = 0" submission="getdata"/>

Last step is a bit of CSS and some framing :

Note. If you find that all the resource text is coming out without tags, then you need a later release of XSLTForms which has support for mime-types in the output element.

More ideas

Of course it probably isn’t entirely practical to have a menu like this in an XForm for navigation, though it’s an interesting experiment in the ‘All XForm’ direction. However,this is a handy tool even if only for it’s original use of picking categories.

Although this is an XForms program at the moment, there’s no reason why it couldn’t be hooked up using JQuery/CSS to get the same sort of look.