Untangling a String with a Thread (see what I did there…?)

Note: This is a way to process strings as part of WPILibs Command Based Programming Framework…for use with First Robotics. It’s also applicable anytime you have to process information on a separate thread of execuction….I had to learn this a few years ago when processing FIX Messages from the Intercontinental Exchange.

Our FRC team set out to improve the Robot’s vision processing; an off season activity. We found the integration between our OpenMV and the Roborio was, well…confused. We had a really basic json message being passed down the wire from the eyes to the brain.

So what’s a nice way to manage the messages the Roborio reads from the serial port?

What about an ExecutorService with an ArrayBlockingQueue?

Figured it might be a good opportunity to teach/learn about work queues, threads, etc.


Create an ExecutorService (I’m using a single thread for this example…tune as required)…this is a way for Java to manage the creation of threads:

private ExecutorService executorService = Executors.newFixedThreadPool(1);

And then create a queue to place your String in for processing:

private ArrayBlockingQueue visionReaderQueue= new ArrayBlockingQueue(1000,true);

The (1000,true) constructor means your queue can hold 1,000 messages and your want them to be FIFO (first in, first our…the order is guaranteed)
These two fields would be on your Command.  Then create a Runnable class called something like “VisionQueueReader”

import java.util.concurrent.ArrayBlockingQueue;

public class VisionQueueReader implements Runnable {
      private ArrayBlockingQueue queue;
     public VisionQueueReader(ArrayBlockingQueue queue) {
          this.queue = queue;
      public void run() {
          while (!Thread.interrupted()) {
                 try {
                      String messageFromQueue = queue.take();
                      //Here is where you do whatever processing on the json you need to…chance are this class might also have a Static reference to the Subsystem waiting for the message
                 } catch (InterruptedException e) {
                      //this gets tripped when you shut the ExecutorServiceDown…the loop ends
                  } catch (NumberFormatException e) {

and finally…in your command that drops the Strings on to the queue, you need to cleanly shut down the Executor service when the system is shutting down


To start the ExecutorService…in either the init or constructor of your command:

executorService.submit(new VisionQueueReader(visionMessageQueue));

so you basically have this ArrayBlockingQueue waiting for strings….when you put a string on, the queue being managed and run by the ExecutorService grabs it and processes it within its run method.

I think this would be a cleaner way to manage that thread resource and guarantee it’ll stick around rather than spend the resources spinning up another…it also guarantees the order in which the messages come off the queue.

Now, this is just another implementation and subject to scrutiny…try it, see if it works. While you do, ask yourself how you might test it vs. your implementation.

7 Year Itch

Someone asked me the other day whether I had a website or not,

“Sure, yep, I do…durablemethods.com”

He asked how long since I’ve updated it

“Hmmm, 2 years maybe?  I’ve been pretty busy”

He pointed to the date of the last post, 2006.

7 Years?!  What have I been doing for seven years.  At the time of my last post I was doing Websphere Portal Development for a large Utility in Calgary.  I’m still in Calgary but times have sure changed.

What’s the same?

  • Durable Methods still operating in Calgary
  • Durable Methods still write software
  • Durable Methods still work with Java

What’s changed?

  • Durable Methods works in Trading, Energy Trading to be more precise
  • Durable Methods doesn’t build browser based applications.  Durable likes Swing.

Wait…Swing…as in Java-on-the-desktop-ghastly-motif-look-and-feel-why-would-anyone-use-it Swing?


Were you paid to use Swing?  Was there a gun to your head?  Have you gone mad?


Here’s a quick backstory:  When I took this latest gig (close to 5 years ago), there were two of us on the trade floor.  My partner at the time was a UI master while I liked building services.  His UI’s (.Net based) would call my services and the traders would smile.  Then a hedge fund appeared, offered us jobs.  He took his, I decided to stay.  That meant I needed to start cracking off UIs or the traders would stop smiling.  Smiling traders are fun to work with.

I had to choose a technology to build in.

Here’s what I had at my disposal:

  • A single server
  • A floor filled with power horse desktop machines

Having come from a thin client, portal background, I went with JSF on Tomcat.  Icefaces to be more precise.

A failure.

I first built an application that consumed so many resources on that single server that I came to understand the limitations of my approach.  IBM commercials that ramble on about the merits of scalability…this is what they were talking about.  I had a non-scalable solution.  Sure this first UI was great, smiles all around…but I knew the truth.  I knew I couldn’t continue on this path and keep the smilies coming.  So I looked forward by looking back.  I decided to leverage the desktop machines and start writing single and dual tier applications.  I’m not a .Net developer and didn’t want to take on the additional learning curve of a new platform so I stuck with Java.

So I’m going to work in Java, building desktop applications…one question remained…. SWT or Swing?  I spent a day exploring SWT.  I love Eclipse and Eclipse is written in SWT, maybe that’s the way to go.  Maybe it was the path or tutorials and documentation I started with but I quickly found myself immersed in the RCP deployment side of SWT and not into what it’s like to write applications.  Time is money and wasting time means wasting money.  It was my time (in the evenings) and so it was my money.  I remember the night.  I just said, “forget SWT for now, I’m going to hello world a Swing app and see how it feels”

It felt odd.

It felt slow.

But, it was progress…I could see the next step.  And the step after that.  Within the month I had a production application running in Swing.  It was a clever app that heat mapped the open interest in all Natural Gas options traded on the CME.  I had a nifty name (Scout) and it worked.

My next technical horizon had been met.  I was building Swing apps.  Things have changed since then, I’ve integrated the following frameworks with immense success:





JMS?  I took that single server I had and turned it into a messaging hub.  It runs ActiveMQ and is the messaging centre for the trade floor I’m working on.  Classic PubSub, it broadcasts the realtime prices in PJM, MISO, CAISO.  It publishes Hydro Data, weather data…you name it.

So that’s where Durable Methods has been for the past 6+ years. Transforming our context and our technology stack.  It’s been an amazing ride and things only continue to improve.

My plan is to write some articles in the coming months outlining generic uses I have (my work is proprietary and the details can’t be published) for Swing and some jumpstarts to get any Swing-curious developers on their way.

Not quite what I expected…JSF & Validation

So I’m working on a simple search screen (for querying LDAP attributes). There is a Googley type interface (simple box with a search button beside it). The LDAP throws a fit if you search on ‘null’ (the value assigned to a text box in JSF when no value has been entered). JSF field tags support a ‘required’ attribute which will validate them. The validation message is, well, brutal.

‘Validation Error: Input Required’

No big deal…I’ve coded lots of custom JSF validators before (for other types of validation), I’ll put a validation method on the pagecode, reference it in the field and be done with it. Here’s the method I created:

public void validateQueryString(FacesContext context, UIComponent toValidate, Object value) {

String queryString = (String) value;

if (queryString == null || queryString.trim().length()<1){
((UIInput) toValidate).setValid(false);
FacesMessage message = new FacesMessage("I'm sorry, I can't make something from nothing...please enter a search string");
context.addMessage(toValidate.getClientId(context), message);

If you’re not familiar with JSF, let me walk you through…it casts the object being validated into a String and then checks to see if it’s null or empty…if it is, it sets ths validator to invalid, binds a message to the field and drops out of the Validation phase of the lifecycle.

I brought up the form, hit ‘Search’ without entering a value in the field and expected to see my cheeky validation message…I see a roar of red text ripping along the console…the Validation passed, the LDAP is getting queried?!?!  That wasn’t supposed to happen
I checked the method name and confirmed it was the same as the reference on the jsp…yep, matched.

I tried again, only this time, I entered a single space in the text field (knowing I’d trim the string and therefore fail validation)…the method was called, my cheeky message appeared…hmmm.  Validation failed as expected

I was coming to the conclusion that JSF doesn’t event create the field for validation if it’s null when I stumbled upon this tread.

Are you kidding me?!

The part I really don’t understand is the response from an architect:


This is what our architect had to say regarding the validator not getting invoked when no value has been entered in a field:

“JSF validators are *not* invoked on empty fields. The general reasoning is that, if there is no input data, then there is nothing to validate. Therefore, the components are working as they should for this use case.”

Hope this helps

Creator Team.”

I’m scratching my head on this one…not because it’s a road block (I could just live with the Eastern block type validation message), but because it throws off what I believed was a pretty solid validation framework.

JSF is self centered when it comes to validation (it wants to do ALL the heavy lifting) which is something I’ve learned to live with. It makes validation testing a pain but again, you learn to live with the cards you’re dealt. I figured I could live with JSF Validation provided I had full control over how it’s used and now I don’t believe that to be the case.

If anyone out there can shed some light on my interpretation of JSF validation on empty fields, I’d really appreciate your input.

EJPPG0003E: ServletContext lookup for /.CreditCounterPartyReviewLog returned the portal context. It has to be a different one.

I’m working on a Portlet project right now. It helps a group over at TransAlta keep their financial obligations and exposures in check. I like the application…it has great business value, terrific customer participation…a great project.

A second developer was introduced to the project recently. He brought the portlets down from CVS and tried running them on his workstation. The received the following error:

2006.08.23 13:37:49.064 E com.ibm.wps.pe.om.definition.impl.WebApplicationDefinitionImpl getServletContext
EJPPG0003E: ServletContext lookup for /.CreditCounterPartyReviewLog returned the portal context. It has to be a different one.

hmmm, what does that mean? He created an EAR file to associate the project with…ahh, but wait! When Rational Application Developer associates Web Modules with an EAR, it doesn’t give it a context root. It tries using the context root for the Portal itself (uh oh) and then throws the exception found above. Fixing it, as my good friend from IBM Thomas Young discovered, is quite simple. Open the EAR, find the module reference and click ‘Refesh’ located beside the Context Root field.

ReDeploy and run…Right Side, not cropped


Paul Glen wrote a funny little list of things not to do when giving a presentation. While most are fairly obvious (and correct), I don’t quite know if I agree with all of them. I know it’s not a good idea to stray from topic but really, he contradicts himself a little. You should stay on topic but if you follow what Paul says, you should also gauge your audience to see if you’re hitting the mark. In my opinion (and experience) you need to improvise from time to time.

It’s also interesting to note his point on keeping opinions away from a presentation, especially when presented as facts. Mr. Glen goes on to say an audience member gives a presenter 30 seconds before determining whether the presentation is worth their attention (hard thing to prove — sounds a little like an opinion to me…). cover_small.jpgHe was trying to emphasize the importance of a strong opening. For anyone attending Eric Evans present at XPAU in 2004, you’ll probably agree on how inaccurate Mr. Glen’s assessment is. Eric opened quietly, meandered for a few minutes before finding his stride. I was happy to have the chance to meet Eric afterwards (while signing my copy of Domain Driven Design) and let him know how incredible I found his talk. There wasn’t a strong opening and if memory serves, not a very ‘strong’ closing. Just a talk comprised of really, really interesting Domain related experiences and findings.

As with everything, take the list with a grain of salt. It’s still nice to see folks in the industry pointing out the importance of a good presentation. I, for one, love delivering presentations on the things near and dear to me.


My wife and I took the kids out to Nipika for a four day weekend. It’s a wonderful place, I’d highly recommend it to anyone looking to get away from the hustle and bustle of city life for a while.

The last thing I did before leaving was export a .war file for release to my client’s Test Environment. I’m starting to learn the last thing you do prior to holidays is rarely the thing that causes the biggest upset while you’re gone. The second last thing you do prior to holidays…now that’s a different story.

As I prepared to export the .war and send it through the deployment process, I noticed something. I noticed a package within the code base with the following name: “org.transalta.creditservices.managedbeantests”.

I like to maintain two projects for every stream of production code, one for the code and one for the tests. That should explain why a package in my deployment release with ‘managedbeantests’ in its name seemed out of place. It’s not uncommon, it usually means I forgot to change the package name at the time of the classes creation. Since I typically create a class from within the context of my ‘Test’ project, it stands to reason why this happened.

No big deal.

So just before exporting, I refactor the package by changing its name to match the other managedbeans, “org.transalta.creditservices.managedbeans”…done. Export .war, send to Tommy and it’s a 4 day weekend.

I return Monday morning to find a note from Tommy, the deployment administrator. To summarize the content of the note, “Jamie, your deployment didn’t work”.

What?!?! Impossible. It worked when I left, I remember deploying it to my development environment right before packaging it up and sending it to Tommy, he must be mistaken. A thread of emails later, he’s right. It’s not working in either Project Dev or Test. I fire up my development environment to prove that I’m not nuts, that it worked just before I left.

It didn’t work.

again, What?!?!?

I spend 5 minutes thinking about the deployment and what may have gone wrong. I focus on the last thing I did before leaving (packaging the .war). A quick look at the code and I find an empty package, “”org.transalta.creditservices.managedbeantests”. Ahhhhh…a clue.
I moved the class but didn’t delete the package (lucky much?). That was enough to trigger my memory, a path to what may have happened lay ahead…. The class I moved is a JSF managed bean. These are beans used within JSF and registered with the JSF context. That means, you guessed it, an explicit reference to it within the faces-config.xml file (bloody xml!?!?!).
Oh, and by the way, nice exception logging JSF, “cannot instantiate ReviewLogManagedBean” (not even the bean in question, just the first referenced bean in the faces-config.xml file.  Almost as useful as Portal’s ‘AssertionFailed’ exception in the log…thanks guys.

I pop open faces-config.xml and there it is, a reference to the old package location. How foolish could I have been? I changed the reference to the new locations, ran it in Dev (it worked), deployed it to Tommy (it worked) and informed the customer (it worked).

So rather than learn a lesson over and over, I thought I’d jot down my lessons learned on this little adventure.

1) Never make a change between final test and deployment, no matter how trivial you think it is. Make the change, rerun your local tests and deploy (Comp. Sci 101, I know, I know)

2) Refactoring tools within Eclipse 3.02 are good, just not good enough to know about references within an XML file. 3.02, I know, is old but it’s what IBM use for Rational Application Developer (tool of choice here).

3) When a container driven exception is thrown and you’re working with JSF, take a look at the faces-config.xml file. It’s worth a shot and really, that’s the heart of the framework.

4) Never make a change between final test and deployment, no matter how trivial you think it is. Make the change, rerun your local tests and deploy (Comp. Sci 101, I know, I know) [worth a second mention]

5) Don’t release anything within 2 hours of a vacation and leave yourself a note.

6) Don’t fry bacon in a cast iron frying pan over a bbq in the mountains if you don’t want to smell like  bacon for the rest of the day.

A Project of Problems

My friend and colleague Iwona and I were talking this afternoon about our current project. It’s a complex integration exercise, joining 4 very different systems while surfacing a new interface through Portal. Iwona is an excellent team member constructing a data model that is both durable and dynamic.

The conversation came around to how we’ve been tackling the problems as they come. Her preference would have been more upfront analysis and less refactoring. It came out that she believes we’ve been ‘designing the system to solve the next problem’.  I bristled at the suggestion….but only for a moment (and the moment’s gone)

I thought about it and realized that’s exactly what we’re doing and I don’t think I’d have it any other way. I’m not saying upfront analysis is a bad thing. Lightweight design meetings are a great way to start a project. They introduce everyone to the Domain, get lots of foundation knowledge on the table. I love that part of a project. I love getting to know a new Domain, form a common Domain language and figure out where the wins are. Conversations beyond that start returning diminished value.

Under a certain light, projects can be viewed as nothing more than a series of problems needing solved. Isn’t that why we got into this in the first place, to solve problems? I know that’s where the draw was on my 1983 Apple II clone, solving the riddles that made the bloody thing work (I was 12 and it the machine rarely worked without some coaxing…).

You often hear team members  describe a pre-production problem as “Murphy’s Law strikes again” and yeah, I guess that’s one way of looking at it. To me, it means a problem wasn’t ready to appear. Now I know that sounds a bit too, I don’t know, abstract or philosophical to have any meaning. What I mean to say is some problems are revealed only through the solving of others. You’ll never solve them all, you just need to solve enough to provide value to those who feel the problem’s pain the hardest.

So whether you spend the time designing the holes in which the fence post will be placed or drawing a rough idea and putting the shovel into the ground…there are rocks, they weigh a lot and it feels amazing once they’re out.

Remembering Memory Management with CORBA

Java is a wonderful language, especially when compared to a container native language like LotusScript. There are many more options, frameworks, design patterns you can play with. The tools for writing Java agents make the days of writing LotusScript agents seem like forever ago. Have I discovered the silver bullet in the Domino world? Is this my valhalla?


Not by a long shot. Remember memory management? Yeah, it’s back. I tried writing a simple agent that processed a Domino view containing a little more than 20,000 records. The agent uses the POI framework to convert the view into an excel workbook. This is done on a nightly basis. Here’s the thing. Domino’s default heap size is 64mb. My agent was running out of memory. A simple increase in heap size got me around the issue but raised the concern that memory management is something I now need to keep my eye on when writing these Java agents. I know, I know, any (every) developer should always be thinking about memory management but Domino’s LotusScript lulled me away from the concern.

That being said, I still believe this is the way I’ll continue to write my Domino agents from this day forward. My main reason? I can write junit tests for them. A topic for another day…

The Perfect Project

While dodging cars and fighting the wind (I ride my bike to work), I started thinking about what components make for the perfect software development project.

Valauble Business Problem
Business Problems fall into two categories. Those worth solving are one, everything else is another. What makes a problem worth solving? One of two things happens when the system goes live: Money is saved or money is made. One of these should present itself within the first quarter of the code going live. Asking a client to wait longer than that means the project was likely too large in the first place, it should have been reworked. A perfect Agile project would have the business measuring money saved/made from one iteration by the time the next iteration is lighting up.

Easy Access to the Business Experts
I think this one is more common than people think. If you’re about to solve someone’s problem, they want to help. Now, they’re never keen to help you solve a problem not worth solving (see above). A legitimate problem? Everytime.

An Agile project has different demands on the Business Expert, they are asked to behave differently. I have never experienced the Business Expert shy away from this. You are asking them to help you help them. Maybe this post will lead to me meeting a Business Expert from hell, keen on ripping my head off for not knowing how Credit Liquidity or Risk Management work…

Open, Dynamic Team Members
My current project fits this in a lot of ways. Lots of people wearing lots of hats. Business Experts reading API docs and writing tests, Developers meeting with management to discuss ROI — an absolute pleasure. Everyone has interests outside the project: another key. Work is something you do from Monday to Friday, morning to late afternoon. I’m much happier discussing various hikes or how my Leafs did last night than how some developer worked all night on a problem they could have fixed had they discussed it with the other team members. Leave your ego at the door, we all make mistakes, we all miss stuff. When you stop thinking people expect you to be perfect, you’ve taken your first step in perfect’s direction.

Flexible Approach
If the requirement is moving people from Canada’s east coast to west coast and the year is 1871, yeah, not so flexible. You’re building a railroad and that’s that.

It’s 2005. An infinite number of tools to solve the same problem exist. Try one. If it’s going to work, use it. When it stops working for you, change. If change means going back over work that’s been done and refactoring it, do it. Changing your code doesn’t mean it was wrong the first time through, it just means you’ve found a better way of working the problem.

Maturing into technology

I think a career is comprised of steps. Some are small and some are quite big. The small ones tend to be more tangible, things like learning a new language, a new design pattern or some new tool. While they seem big at the time, really, they’re just newer ways of doing something you’re already trying to do. Still a step forward, just a little one.

I think the bigger steps are also changes in the way you do things, just on a much higher scale. Test Driven Development would be an example that comes to mind. Still writing code, just changing the way I think about it.

Maybe that’s a bad example…no, that’s a bigger step than using, say Eclipse, instead of Visual Age for Java but still not a BIG step.

I don’t really have an expression for them but two big steps have come to mind recently. The first is the notion of getting others involved in what you’re working on. I see (and used to be) so many software developers working within their own little world. They sit at their machine and cut code all day. Agile is supposed to alleviate this a little, getting customers involved (a great idea) but leaves an element out. The developer in that case is often still left with a screen full of their own code. Pair Programming is a start, but it’s still missing the step. The step I’m referring to is when a developer wants other people in their code, wants their peers to see what they’re doing. Code Reviews fail in this respect in the same way as pair programming: it’s usually someone else’s idea. Yep, a developer (or at least this one) should want people to look at what they’re doing. Developer’s should stop looking at what they do as being the trade putting insulation behind the gyprock. Developers need to see themselves as (using the same metaphor) cabinet makers (or any trade with a precise skill under constant scrutiny).

That’s the first one.

For the second step, you need to listen to a little story. I’m an alumni of NOLS, having spent time kayaking with them in the Sea of Cortez. The most significant lesson I learned on those travels was the concept of Leave No Trace (lnt). A lot of the lnt princicples can be applied to software developmet. What are some “traces” that I would consider trying not to leave?

-Major server configuration changes
-Additional memory requirements
-newer hardware
-container classpath changes

This list could go on forever. By trying to apply lnt principles to software development, the developer needs to use their mind to devise creative ways around environmental constraints. This way their application can be moved from server to server and deployment won’t be an issue. Deployment can just happen. The tents come down and the next camper doesn’t know you were there (so to speak). I want to continue trying to make this step. With the falling costs of disk space and memory, the temptation will always be to write less efficient code and place the burden on the infrastructure. tsk, tsk I say. Chopping down one little tree does have an impact.

To summarize: Invite lots of people on the camping trip and leave the campsite looking like you were never there.