Reading usb serial data with Apache Camel

I’ve done a couple of posts on using Apache Camel as part of my home-monitoring system, and why I think it’s a good fit as an IoT switchboard. However, one of the flaws in my master plan is that there isn’t a component for serial communications (Camel MINA does socket comms, but not serial). Note I’m not talking actual RS232 here but the version that comes in USB form. I’m using mostly JeeNodes, which talk RF12 between them, but even in this modern era, without a wi-fi shield or some similar way to get onto the LAN and TCP/IP, then at some point, you’re talking to a serial port on some box into which, in my case a controlling JeeLink, is plugged.

Now, I’ve got around this like I guess most people have, by creating a serial-tcp/ip bridge in code, and getting on with our lives. But, it isn’t, well elegant and it bugs the engineer in me. It would be nice to be able to link directly to Camel and use the considerable advantages it brings, like throttling and queues and all the other goodies. But sadly, my Java isn’t up to creating modules, and it’s a fairly niche use-case. So, up to now I’d pretty much given up on it.

What I hadn’t realised, was that the Stream component, not only lets you talk to the standard input and output streams but also files and urls. Reading further, Stream has support for reading a file continually, as if you were doing ‘tail -f’ on it. Now in Unix, famously “everything is a file”, so in theory, you should be able to read a serial device, like it was a file and if you could Stream should be able to as well. Cue ‘Road to Damascas’ moment.

For my test I quickly grabbed a JeeNode and plugged it into my MacBook. Then I wrote a short sketch that spat out a clock to the serial port:

#include <Stdio.h>
#include <Time.h>
char buffer[9] ="";
void setup()
{
   Serial.begin(9600); // Tried 19200 but didn't work!
   setTime(0,0,0,0,0,0); // reset starting time
}
void loop()
{
    digitalClockDisplay();
    delay(5000);
}
void digitalClockDisplay(){
    //Send a whole line at a time
    sprintf(buffer, "%02d:%02d:%02d",hour(),minute(),second());
    Serial.println(buffer);
 }
}

I could see it running in the Serial Monitor in Arduino, and I could see the device in /dev, but nothing if I tried “tail -f  /dev/cu.usbserial-A900acSz”

Turns out, the default baud rate is 9600 on Macs and also that you can’t alter them using Unix stty commands either!  After reading a whole lot of internet, it seemed that, at least for my demo, I’d just have to stick to 9600 or get pulled into a whole pile of Mac-mess. Also note, at least on a Mac I got /dev/cu.* and /dev/tty.*. Only the cu.* ones seemed to work. I think this is something to do with the tty.* ones expecting handshaking, but I’m happy to be corrected.

Once I’d altered the baud-rate every thing worked fine. I could tail -f onto my JeeNode device and it would happily spit out stuff like this:

00:00:20
00:00:25
00:00:30
00:00:35

On the Camel side, I made a copy of camel-example-console from the /examples folder and modified the camel-context.xml to hold my new routes. First a route to read from my JeeNode – nothing fancy, just get the data and put it in a SEDA queue as a string. Then a route to get it off the queue and send it to the screen via stdout:

 <route>
 <from uri="stream:file?fileName=/dev/cu.usbserial-A900acSz&amp;scanStream=true&amp;scanStreamDelay=1000"/>
 <convertBodyTo type="java.lang.String"/>
 <to uri="seda:myfeed"/>
 </route>

 <route>
 <from uri="seda:myfeed"/>
 <to uri="stream:out"/>
 </route>

Note: scanStream tells it to scan continuously (to get the tail -f effect) and the scanStreamDelay tells it to wait a second between scans.

A quick mvn clean compile exec:java later and it works! So, it seems that I can have my cake after all.

Now, there are caveats. For a start Camel gets pretty upset if you unplug the JeeNode and the device disappears. Also, it seems I’m stuck with 9600, at least on my Mac. But it does work and means a complete Camel solution is possible. Time for more experiments, but at least for the moment my inner engineer is quiet.

PS. It works the other way as well 🙂

<route>
 <from uri="stream:in?promptMessage=Enter something: "/>
 <transform>
 <simple>${body.toUpperCase()}</simple>
 </transform>
 <to uri="stream:file?fileName=/dev/tty.usbserial-A900acSz"/>
</route>

 

Camel on it’s own

In my last post, I looked at Apache Camel as a sort of switchboard, useful in a home monitoring situation; mostly because you could control it just by manipulating the configuration rather than coding. Originally, I was using Camel as a servlet by simply extending the Tomcat Servlet example, which neatly packaged up the end result into a war file, for me to drop onto Jetty. However, if you’ve no use for the web side and just want to load a console app it’s a bit more complicated.

Now there is a Console example: and full marks to the Camel team -it’s one of some 30-odd examples you get to play with. The problem is that mvn compile exec:java works fine, but you’re stuck in the examples folder. What’s needed is a way to make it a proper application with all the dependencies. Now I’m sure this is trivial for the java/maven savvy, but for the rest of us, here’s the recipe:

Step 1 – Add AppAssembler to your pom.

As I understand the Camel/Maven game, and honestly as an occasional java user I don’t really, what’s needed is a method to not only create a jar file and all the dependencies, but also generate a method to kick the whole thing off. A bit of research led me to AppAssembler from CodeHaus. What this does is create a directory for your app, all it’s dependencies and a couple of clever start-up scripts to kick the whole thing off. All you need to do is add the following to your pom file, after the exec-maven plugin (I’ve used the Console example here).

<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>appassembler-maven-plugin</artifactId>
<configuration>
<programs>
<program>
<mainClass>org.apache.camel.example.console.CamelConsoleMain</mainClass>
<name>camel-console</name>
</program>
</programs>
<repositoryName>lib</repositoryName>
<repositoryLayout>flat</repositoryLayout>
</configuration>
</plugin>

The are lots of things you can configure, but of note in this simple setup are:

• Main-class. Set it to the one in the exec-maven plugin
• Name. The bin scripts use this name so you get camel-console.bat etc.
• RepositoryLayout. I’ve chosen flat which dumps all the jars in one directory. Default is to split them into packages.

Now if you run mvn clean compile appassembler:assemble you will find, after a bit of maven’ing, a new appassembler folder will appear under your target folder with /bin and /lib folders. If you fire up bin/camel-console (after chmod +x ing) or bin/camel-console.bat to suit then you should see the Console example running in all it’s glory. Moreover, you can move this directory anywhere and it will still run perfectly. Result.

However, there’s a problem – our camel-config.xml where the <route>s live is now buried in a jar somewhere. As the whole purpose here was to create a configurable camel, we need a small amount of modification so that we can have our xml file in a handy conf directory.

Step 2 – External config file.

First thing is to tell the code that our config file is external. So, open the CamelConsoleMain.java in src/main/java. In there you’ll find a line like so:

 main.setApplicationContextUri("META-INF/spring/camel-context.xml");

We want an external file so we’ll change this to:

main.setApplicationContextUri("file:conf/routes.xml");

Next, create a src/main/config and copy camel-context.xml into it (why will become clear in a mo’). Rename it to routes.xml. Note, the old config file etc will still get packaged up as well – that’s redundant, but I’m making the fewest mods possible here.

Lastly, we’ll leverage our new friend appassembler. Add the following lines to the pom.xml in the appassembler plugin you added previously, below <name> and run the pithy mvn clean compile appassembler:assemble again:

<!-- Set the target configuration directory -->
<configurationDirectory>conf</configurationDirectory>
<!-- Copy the contents from "/src/main/config" to the target -->
<copyConfigurationDirectory>true</copyConfigurationDirectory>
<!-- Include the target configuration directory in the classpath -->
<includeConfigurationDirectoryInClasspath>true</includeConfigurationDirectoryInClasspath>

These properties tell it to copy anything in /src/main/config into /conf directory in the output. Notice I’ve also included the option not to add the /conf folder to the class path. Now, when our app runs it will find our config file in the right, relative, place. Of course, we could just create a /conf dir and drop our config into it once the assembly had happened. But, doing it this way gives us somewhere to put it in src.

Addendum.

So, it’s fairly simple to create a neat, deployable app that we can use on a variety of OS’s  to leverage the power of Camel. If you want to run on a server, a handy App Assembler option is to create Java Service Wrapper scripts so you can start the code as a daemon.

One thing I didn’t handle here is an external log file which is handy if you need to change logging level etc.  It turns out to be as simple as moving log4j.properties to our /src/main/config directory and setting the flag to add our /conf directory to the class path. Works like a charm.