Drill-in menu with XForms

As part of an XForms project I’m doing I’ve been looking into menu options and came across the drill-in menu, popularised by a famous range of mp3 players. The menu comes in two parts, a history section at the top and current choices below. As each selection is made, that choice is added to the history and it’s children appear in the menu.

The advantages here are that it’s a compact mechanism for use with any deeply nested structure. Only the current menu is shown at any time and the history gives a convenient breadcrumb trail to that choice-set. If you don’t want to use a tree, linked lists or fly-out, drill-in might be for you. Luckily, it’s fairly easy to do as an XForm and sits nicely on my SimpleXist server.

Note: I’m using XSLTForms here to do my XForms and the SimpleXist set-up described elsewhere. The code expects /resources/xsltforms in the root. 

Data Data Data

The source data for the menu is a standard nested piece of XML. It could be in one piece, a collection, whatever. The important factor is that there is some mechanism to uniquely target each menu item. In my case, I’m using menu.xml and a numeric id; I’ve set the root item to ‘000’.

Next step is some XQuery to put up the menu. I’ve taken the decision to send the whole menu with each submission and get back a complete one to replace the instance. I’m using /db/projects/project3 in the SimpleXist idiom for the data. Here’s menu.xql:

xquery version "1.0";
declare option exist:serialize "method=xml media-type=text/xml indent=yes";
let $data_collection := '/db/projects/project3/'
let $data := request:get-data()
let $q := if (empty($data//prefix)) then ('000') else ($data//prefix)
let $file := concat($data_collection,'menu.xml')
let $head := doc($file)//category[id=$q]
let $this_item := <item id="{$q}" children="{count($head[1]/category)}">{$head[1]/title/text()}</item>
let $hist := $data//history/item
return
<data>
 <prefix/>
<history>
{$hist}
{if ($this_item/@children > 0 ) then ($this_item) else ()}
</history>
<menu id="{$q}">
 {for $z in $head/category
 return <item id="{$z/id}" children="{count($z/category)}">{$z/title/text()}</item>}
</menu>
</data>

The key points here are that that the submission posts the whole menu. The prefix element contains the choice or ‘000’ as default. The new menu is put into $head and $this_item set to the chosen item. Then $history is set up to the current history. Lastly, a new menu is assembled: $this_item is appended to the history if the current choice has a child menu, then all the menu items for this choice are assembled along with a count of children. This last is so the form can display and behave differently for “branches” and “leaves” in the form. The default output looks like so:

Note that the menu isn’t in the format of the original menu; that’s deliberate. The XForm is wrapped around the menu and would be harder to change than the original data. With a bit of XQuery, pretty much any tree data could be coerced into this format.

Simple XForm

Now we have data it’s time to set up the XForm. It’s pretty simple at the moment with one instance to hold the current menu, and one submission to get the next one as shown by drill.xql the meat of which is below:

<head>
<xf:model>
<!-- Instance to hold the menu -->
<xf:instance id="navigation" src="menu.xql" xmlns="" />
<xf:submission id="nextmenu" method="post" 
replace="instance" instance="navigation" 
ref="instance('navigation')" action="menu.xql"/>

</xf:model>
</head>
<body>
<div id="drillin">
<div id="history">
<!-- History Bar -->
<xf:repeat id="hist-repeat"
  nodeset="instance('navigation')/history/item">
<!-- Output as a set of links -->
<xf:trigger appearance="minimal">
<xf:label><xf:output value="position()"/> <b><xf:output ref=".  "/></b></xf:label>
<!-- When we're clicked .... -->
<xf:action ev:event="DOMActivate">
<!-- Rather long-winded way to put current choice in prefix -->
<xf:setvalue ref="instance('navigation')/prefix" 
  value="instance('navigation')/history/item
  [index('hist-repeat')]/@id"/>
<xf:delete
  nodeset="instance('navigation')/history/item[position() &gt;=
  index('hist-repeat')]"/>
<xf:send submission="nextmenu"/></xf:action>
</xf:trigger>
</xf:repeat>
</div>

<div id="menu">
<!-- Menu Items -->
<xf:repeat id="nav-repeat"
  nodeset="instance('navigation')/menu/item">
<xf:trigger appearance="minimal"> 
<xf:label><xf:output value="if(./@children = 0,'>','+')"/> 
<xf:output ref="."/></xf:label>
<xf:action ev:event="DOMActivate">
<xf:setvalue ref="instance('navigation')/prefix"
  value="instance('navigation')/menu/item[index('nav-repeat')]/@id"/>
<xf:send if="@children !=0" submission="nextmenu"/>
</xf:action>
</xf:trigger>
</xf:repeat>
</div>
</div>
</body>

The history div holds the history data as a list of links via a repeat. Position() is used so these get numbered as there is an implied order. There is a trigger hooked into the selections which does two things:

  1. It sets the prefix element to the id of the selected item.
  2. It deletes any item below the selected one in the history.

The deletion is necessary as it lets the history react properly if you select an item in the middle; everything ‘later’ is deleted. Then the whole menu is submitted.

The menu div hold the current menu options, again as a repeat. There is a bit of code in the label to put a ‘>’ or ‘+’ in front of the data depending on whether there are children or not. On selection of an item, again the whole menu is submitted, but only if that menu has children.

A little bit more complicated

Now the basics are out of the way it’s time to put together something a little more interesting. If you want this version feel free to download it here. Initially, I was going to use this menu to pick categories, then I got the idea to use it for navigation instead. Assuming for the sake of argument that a menu item points to some resource NNN.xml, the leaves can now do something useful. Lets start with 000.xml in the database for the root item:

<content>
 <h4>Hello</h4>
 <p>I'm some content</p>
</content>

We’ll need a bit more XQuery, query.xql,  to get a given resource from it’s id. Again  the menu gets sent:

xquery version "1.0";
declare option exist:serialize "method=xml media-type=text/xml indent=yes";

let $data_collection := '/db/projects/project3/'
let $data := request:get-data()
let $q := if (empty($data//prefix)) then ('000') else ($data//prefix)
let $file := concat($data_collection,$q,'.xml') 
let $tail := $data//item[@id = $q][1]
let $strpath := string-join(for $t in ($data//history/item,$tail)
return $t,'/')
let $error :=
<content>
<h2><a name='h1'>Oops</a></h2>
<p>I can't find {$file} in the database</p>
</content> 

let $input := if(doc-available($file) = true()) then (doc($file)) else $error

return
<data>
<meta>{$strpath}</meta>
<content>{$input}</content>
</data>

There is an error message as of course most of the items won’t exist – feel free to add some! One handy additional feature we can get from having the whole menu is a breadcrumb via the history made up of each item, plus current separated by a ‘/’ and put in the meta element. Of course it could just as easily hold other things like creation date etc.

We now only need to be able to tell the XForm menu section to get the resource if it’s a leaf, otherwise the next menu.

<xf:send if="@children != 0" submission="nextmenu"/>
<xf:send if="@children = 0" submission="getdata"/>

Last step is a bit of CSS and some framing :

Note. If you find that all the resource text is coming out without tags, then you need a later release of XSLTForms which has support for mime-types in the output element.

More ideas

Of course it probably isn’t entirely practical to have a menu like this in an XForm for navigation, though it’s an interesting experiment in the ‘All XForm’ direction. However,this is a handy tool even if only for it’s original use of picking categories.

Although this is an XForms program at the moment, there’s no reason why it couldn’t be hooked up using JQuery/CSS to get the same sort of look.

SimpleXist III

Last time, we finished with a working example on a small XQuery engine based on a simplified eXist install called, unimaginatively, SimpleXist. Now we could leave it there as it’s true that you can write a whole application with just the database and XQuery. However, once forms are involved, it’s so much easier to use XForms rather than native HTML which is of course also the front bit of the XRX architecture. In our case we use XSLTForms to do our XForms on the client side, although there are also good ways to do it server-side with something like BetterForms.

Note: I’m not getting into a XForms / XRX how-to just now. There are numerous resources such as Dan McCreary’s XRX Beginners Guide and there is an excellent WikiBooks XForms section to name but two you can read if unfamiliar with the topic.

Now with our initial project, we used a transform to render the output page at the end of each XQuery. This is a pretty useful idiom and we’d want to keep it going with our XForms. This means that our XSL style sheet needs modifying so that it can cope with two sorts of input, straight xml and xforms.

Simple XForm

Before we can do anything we need the XSLTForms software from the agenceXML site. Download it and put it in /ROOT/resources folder as this probably isn’t the only project that’s going to use it! They also have a simple HelloWord XForm on their site we’ll use as a template. Have a look at the source; what we’re going to do is make that work in SimpleXist. So, first off, create a hello.xql in /project1 and copy in the following:

xquery version "1.0";
declare namespace transform="http://exist-db.org/xquery/transform"; 
declare option exist:serialize "method=xhtml media-type=text/xml indent=yes"; 
let $input := 
<document xmlns:xf="http://www.w3.org/2002/xforms">

<header>
  <title>Hello World in XForms</title>
    <xf:model>
      <xf:instance>
      <data>
        <PersonGivenName />
      </data>
      </xf:instance>
    </xf:model>
</header>

<content>
<p>Type your first name in the input box.<br />
If you are running XForms, the output should be displayed in the output area.
</p>

<xf:input ref="PersonGivenName" incremental="true">
  <xf:label>Please enter your first name: </xf:label>
</xf:input><br/>

<xf:output value="concat('Hello ', PersonGivenName, '. We hope you like XForms!')">
<xf:label>Output: </xf:label>
</xf:output>

</content>
</document>
return
transform:transform($input, "styles/style.xsl", ())

There are a couple of things to note here. First, we’re returning text/xml as the mime-type rather than text/xhtml as that’s what triggers the XForms XSL transform on the client. Next, we’ve got a namespace in our document section, and lastly, we’ve now got a separate <header> element in our XML as the <xf:model> parts of the form need to go in the HTML <head> element in the output.

Now if you try to run it at localhost:8080/project1/hello.xql , nothing interesting happens as we don’t have the XSLT to support it: that’s next.

XForms XSLT

First off, copy this style.xsl over the one in /project1/styles. It’s a small addition to find out if we’re in an XForm or not and send the right processing-instructions back. This is done by testing for xf:model in the header. Note that we’ve used /resources/xsltforms.xsl as we’re using the /ROOT/resources folder.

<xsl:if test="document/header/xf:model">
<xsl:processing-instruction name="xml-stylesheet">
  href="/resources/xsltforms/xsltforms.xsl"
  type="text/xsl"
</xsl:processing-instruction>
<xsl:processing-instruction name="css-conversion">
  no
</xsl:processing-instruction>
<xsl:processing-instruction name="xsltforms-options">
  debug="no"
</xsl:if>

We can also set other specific XSLTForms directives in here, such as setting support for ordinary css files and turning off debug. Using this code, the standard XQuery will be returned as before and the XForms version with appropriate processing instructions.

Now if you run the hello.xql file again you should get a working XForm!

Next steps: HerdBook

Now we’ve a set-up that’s simple, works with XQueries on the file system and supports XForms. Next stop a personal project to exercise the setup a bit more fully. I’ve been looking to scratch this itch for a while – I need a Herd Book. This is a database of all the llamas and alpacas we have, matings, medicines we’ve given them, shows they’ve been to and results. In short everything we have currently written in various calendars, notebooks and a set of index cards. It’s way overdue!

SimpleXist II

A couple of days ago I did a post on a simple eXist setup – lazily called SimpleXist . As I left it, it was a running system but there were a few shortcomings and complaints when it runs :- a) No JSP support,  b) No log4j config, oh and the minor matter  of no eXist client to talk to the database! So it’s time for a bit more configuration and then we’ll take it for a spin with a couple of simple examples.

As per last time if you want the version described here, you can just download it.

Errors & Warnings

This is what I get currently from the system on startup :

macbook:Test tingenek$ java -jar start.jar
2012-01-01 17:35:08.111:INFO::Logging to STDERR via org.mortbay.log.StdErrLog
2012-01-01 17:35:08.237:INFO::jetty-6.1.26
2012-01-01 17:35:08.503:INFO::NO JSP Support for,
did not find org.apache.jasper.servlet.JspServlet
log4j:WARN No such property [append] in org.apache.log4j.ConsoleAppender.
2012-01-01 17:35:10.531:INFO::Started SelectChannelConnector@0.0.0.0:8080

Now the JSP issue is easy, you can either add in the jsp-2.1 folder into Jetty /lib, or just ignore it – it does no harm. The logging just needs a bit more configuration.

The eXist software uses the log4j system to write sophisticated logs for various modules. In our case what we wanted was just simple logging to the console with everything else. It’s easy to setup. First create in /ROOT/WEB-INF a folder called /classes. In that folder put this file, log4j.xml. All it does is create an appender to pipe everything to the console. Note the use of the “priority” setting in the <root> element. This controls how much logging eXist does, at the moment it’s just errors but you can dial it up to “info”, “warn” or even “debug” if you really want to see everything as it happens.

eXist Client

It may not be obvious, but the eXist client is run from the start.jar in the .war file /lib directory. So, first step is to copy it, renaming it to client.jar and add it to SimpleXist. You should now have start.jar and client.jar in the same folder. If you try and run it, you’ll find it complains about not being able to find eXist root and bombs. What’s needed is a small shell file to fire it up, so make a file called called client.sh, and paste the following in:

#!/bin/bash
java -Dexist.home=webapps/ROOT/WEB-INF -jar client.jar client

Now if you run this, it’ll complain about not finding lib files in various folders; that’s because we have a flat lib, but it’ll work and you’ll get the eXist client. Make sure you alter the client URL as we’ve used localhost:8080/xmlrpc rather than /exist/xmlrpc. Also, default .war database install doesn’t have an admin password – remember to set one once you’re in!

Note. I might get around to repackaging the client jar at some point so it doesn’t complain, but at the moment it’s not a priority. 

Before we begin – housekeeping.

As with everything, it’s always good to have some sort of system in place; everyone’s happier when the expected occurs. Now, this is just the way I’ve set this up, feel free to do something different, the key is predictability. So, for SimpleXist there is:
  • A folder off from /ROOT for each project.
  • A /resources folder off /ROOT holds common files, like XSLTForms, TinyMCE
  • A /resources folder in each project holds resources for that project i.e. /css /scripts /images
  • A folder called /styles holds any xslt files.
  • The XQuery files are in each project root.
  • In the eXist database a collection with the same name as the project off from a /projects collection.
Idea! Because we have ROOT and local resources we can have things like a generic /resources/css/site.css in our header to set up things generally, followed by a project specific resources/css/project.css over-riding specific settings.

Getting by without pipelining.

In anything but the most trivial system, we’ll need some sort of framing for the site: the header, footer, navigation etc that we don’t want to repeat per page. Now a common route is to use a pipeline with the XML being turned into HTML through Cocoon,XProc, URLRewriting etc. In SimpleXist this isn’t possible as we’ve got none of those things – though, we could have, but choose not to is more accurate. To get around that we’ll use the excellent transform module in eXist to convert our XQuery into HTML for us at the end of each XQuery using an XSL transform, like so:

First, create a new folder under /ROOT called project1. Now create an index.xql file with the following content:

xquery version "1.0";
declare namespace transform="http://exist-db.org/xquery/transform";
declare option exist:serialize "method=xhtml media-type=text/html indent=yes";
let $input :=
<document>
<content>
<h2><a name='h1'>Test</a></h2>
<p>Hello from XSLT</p>
</content> 
</document>
return
transform:transform($input, "styles/style.xsl", ())

This XQuery sets up a simple bit of XML with a mime-type of html and then calls a transform against it from our /styles folder, returning the results. To get this to work, create a /styles folder in /resources. In /styles put the following style.xsl file. There is also a tiny amount of css so you’ll also need /css and site.css. If you have a look at the XSL file, you’ll see that it simply pulls the document/content element out of the XQuery into the appropriate place in a simple piece of HTML. Now a quick look at localhost:8080/project1 should give you a plain’ish piece of html.

Now this might seem trivial but it just requires a bit more structure and CSS and you’ve the basics of a web site. Even better, if you look at the XQuery, you’ll see that that bit of XML could just as easily come from the eXist db. In fact, you could run a whole site from this bit of code, just by giving it the id of the bit of xml you want to drop in. We’ve got a bit of a WordPress vibe starting 🙂

Using the database.

Fire up your eXist client (you created a password for admin didn’t you?) and go to File->Create Collection. Create a /projects collection and inside that a /project1 collection. In /project1 use File->Create Blank File, call it 1.xml and inside it copy :

<document>
<content>
<h2><a name='h1'>Page 1</a></h2>
<p>Hello from page 1</p>
</content> 
</document>

Make a couple more, excitingly called 2.xml and 3.xml. Now alter the index.xql file to pull the content from the database:

xquery version "1.0";
declare namespace transform="http://exist-db.org/xquery/transform";
declare option exist:serialize "method=xhtml media-type=text/html indent=yes"; 
let $data_collection := '/db/projects/project1/'
let $q := request:get-parameter('q', '1')
let $file := concat($data_collection,$q,'.xml')  
let $error :=
<document>
<content>
<h2><a name='h1'>Oops</a></h2>
<p>I can't find {$file} in the database</p>
</content> 
</document> 
let $input := if(doc-available($file) = true()) 
then (doc($file)) else $error
return
transform:transform($input, "styles/style.xsl", ())

What happens here is that we look for a request parameter ‘q’. If it doesn’t exist, we set it to 1. Next we make up a path in the collection to from the root and the parameter to get a file path like /db/projects/project1/1.xml. Next, we check a document exists for that path, if it doesn’t we use an error doc, otherwise we fetch it. Lastly it gets transformed and returned.

If all goes well, you can now use localhost:8080/project1/index.xql?q=1 or the rather terser /project1/?q=1 to get document 1.xml etc.

This is one of the things I like about the XRX system: simple, clean XML  data and small amounts of XQuery code give you (with a bit more work) the tools to put a site together!

What’s missing now is the XForms side. Next time..

SimpleXist

Over the last few weeks I’ve been looking at introducing the joys of the XRX world using eXist as we look to move away from Cocoon. One problem I’ve found is that eXist defaults to presenting, not only the eXist database, but a cornucopia of XML technologies. It’s an excellent showcase of things like XProc and not one, but two XForms implementations as well as concepts such as having the code and data in the database. But, if you’re used to putting code in a folder it can be tricky to know where to start and this gets in the way of the goal: learning about XQuery, XForms, and the database.

So, I’ve set up a simpler version called SimpleXist that either hides or removes most of the harder topics. The key things I’ve gone for are:

  • Minimal servlets, only XQuery, REST (optional) and XML-RPC (for the client)
  • XQueries run from the disk rather than the database.
  • No URL-Rewriting.
  • Can be locally deployed to a workstation but will work on a server.

I’ve written up the configuration below, but if you want to just download it then here you go. I’ve purposely not taken the route of repackaging jars or reducing their number. Everything is there, just not turned on. It’s all done with configuration and thus saves me embarrassing myself with my limited java skills.

Dear eXist gurus, I know there are other options, you can do a minimal recompile etc, etc. I’ve taken this route as it fits in with what we need, it’s easy to maintain and fits into the deployment and development methods we use. 

Jetty Minimal

First step was a minimal Jetty. It’s not critical to use Jetty but it gives a nice small container to run eXist that works on a desktop. I’ve used a download of Jetty 6 (6.1.26)  as the start point. Put Jetty somewhere; we only need a few files from it and then create a a folder for SimpleXist. In that folder make subfolders for /etc, /lib and /webapps. In the /webapps directory create a subfolder called /ROOT and in that create a folder /WEB-INF. Lastly in that folder create a folder /data.

Keep an eye on the folder structure shown on the left. This is what you’re heading towards.

Now, copy start.jar from Jetty to the SimpleXist folder. Next, Put jetty-6.1.26.jar, jetty-util-6.1.26.jar and servlet-api-2.5-20081211.jar into /lib. Lastly, in /etc put this jetty.xml file.


This is enough to set up a simple server on port 8080, which will run any webapp deployed in /webapps – in our case the default ROOT webapp. This is one of many reasons to love Jetty.

You can try it out with the command  java -jar start.jar . Point your browser to http://localhost:8080. It won’t be interesting or pretty but it should work.

Now we need to set up a simplified eXist to run in our container.

eXist Install

As with Jetty, download a copy of eXist. In this case I used the setup jar so I had a working install of the full system as a start point. Next, in the eXist root directory, run build dist-webapp which will create a .war file in the eXist /dist directory. I’m using war format because a) I like all my lib files in one directory and b) because it makes the app easy to move to something like Tomcat, which we use for production, later on.

Open up the .war with your favorite unzip utility and from it’s WEB-INF folder copy conf.xml and the /lib folder into /ROOT/WEB-INF.  Next,  download this web.xml into WEB-INF

Web.xml is the meat of the system. It sets up:

  • The eXist XQuery servlet to handle anything ending .xqy or .xql
  • The XML-RPC servlet to handle the eXist client connection at /xmlrpc.
  • The eXist Servlet to start the database and respond to the REST commands at /rest
  • The default servlet to handle resources i.e. a catchall servlet for anything else. Note this is Jetty specific.
  • Lastly it looks for index.xql for the welcome file if nothing is requested at all.

Finally, we’ll create the ubiquitous Hello World to test it all out. Make an index.xql file in /ROOT with the following content:

xquery version "1.0";
declare option exist:serialize "method=html media-type=text/html";
let $hello := 'Hello World!'
return
<html>
<body>{$hello}</body>
</html>

That’s it. You can now run java -jar start.jar and you should see the famous message at localhost:8080. The Jetty container will complain a bit about logging, there isn’t a client and a couple of things need tidying up, but basically we’re there. Next time for those bits and some development examples.