Untangling a String with a Thread (see what I did there…?)

Note: This is a way to process strings as part of WPILibs Command Based Programming Framework…for use with First Robotics. It’s also applicable anytime you have to process information on a separate thread of execuction….I had to learn this a few years ago when processing FIX Messages from the Intercontinental Exchange.

Our FRC team set out to improve the Robot’s vision processing; an off season activity. We found the integration between our OpenMV and the Roborio was, well…confused. We had a really basic json message being passed down the wire from the eyes to the brain.

So what’s a nice way to manage the messages the Roborio reads from the serial port?

What about an ExecutorService with an ArrayBlockingQueue?

Figured it might be a good opportunity to teach/learn about work queues, threads, etc.


Create an ExecutorService (I’m using a single thread for this example…tune as required)…this is a way for Java to manage the creation of threads:

private ExecutorService executorService = Executors.newFixedThreadPool(1);

And then create a queue to place your String in for processing:

private ArrayBlockingQueue visionReaderQueue= new ArrayBlockingQueue(1000,true);

The (1000,true) constructor means your queue can hold 1,000 messages and your want them to be FIFO (first in, first our…the order is guaranteed)
These two fields would be on your Command.  Then create a Runnable class called something like “VisionQueueReader”

import java.util.concurrent.ArrayBlockingQueue;

public class VisionQueueReader implements Runnable {
      private ArrayBlockingQueue queue;
     public VisionQueueReader(ArrayBlockingQueue queue) {
          this.queue = queue;
      public void run() {
          while (!Thread.interrupted()) {
                 try {
                      String messageFromQueue = queue.take();
                      //Here is where you do whatever processing on the json you need to…chance are this class might also have a Static reference to the Subsystem waiting for the message
                 } catch (InterruptedException e) {
                      //this gets tripped when you shut the ExecutorServiceDown…the loop ends
                  } catch (NumberFormatException e) {

and finally…in your command that drops the Strings on to the queue, you need to cleanly shut down the Executor service when the system is shutting down


To start the ExecutorService…in either the init or constructor of your command:

executorService.submit(new VisionQueueReader(visionMessageQueue));

so you basically have this ArrayBlockingQueue waiting for strings….when you put a string on, the queue being managed and run by the ExecutorService grabs it and processes it within its run method.

I think this would be a cleaner way to manage that thread resource and guarantee it’ll stick around rather than spend the resources spinning up another…it also guarantees the order in which the messages come off the queue.

Now, this is just another implementation and subject to scrutiny…try it, see if it works. While you do, ask yourself how you might test it vs. your implementation.

7 Year Itch

Someone asked me the other day whether I had a website or not,

“Sure, yep, I do…durablemethods.com”

He asked how long since I’ve updated it

“Hmmm, 2 years maybe?  I’ve been pretty busy”

He pointed to the date of the last post, 2006.

7 Years?!  What have I been doing for seven years.  At the time of my last post I was doing Websphere Portal Development for a large Utility in Calgary.  I’m still in Calgary but times have sure changed.

What’s the same?

  • Durable Methods still operating in Calgary
  • Durable Methods still write software
  • Durable Methods still work with Java

What’s changed?

  • Durable Methods works in Trading, Energy Trading to be more precise
  • Durable Methods doesn’t build browser based applications.  Durable likes Swing.

Wait…Swing…as in Java-on-the-desktop-ghastly-motif-look-and-feel-why-would-anyone-use-it Swing?


Were you paid to use Swing?  Was there a gun to your head?  Have you gone mad?


Here’s a quick backstory:  When I took this latest gig (close to 5 years ago), there were two of us on the trade floor.  My partner at the time was a UI master while I liked building services.  His UI’s (.Net based) would call my services and the traders would smile.  Then a hedge fund appeared, offered us jobs.  He took his, I decided to stay.  That meant I needed to start cracking off UIs or the traders would stop smiling.  Smiling traders are fun to work with.

I had to choose a technology to build in.

Here’s what I had at my disposal:

  • A single server
  • A floor filled with power horse desktop machines

Having come from a thin client, portal background, I went with JSF on Tomcat.  Icefaces to be more precise.

A failure.

I first built an application that consumed so many resources on that single server that I came to understand the limitations of my approach.  IBM commercials that ramble on about the merits of scalability…this is what they were talking about.  I had a non-scalable solution.  Sure this first UI was great, smiles all around…but I knew the truth.  I knew I couldn’t continue on this path and keep the smilies coming.  So I looked forward by looking back.  I decided to leverage the desktop machines and start writing single and dual tier applications.  I’m not a .Net developer and didn’t want to take on the additional learning curve of a new platform so I stuck with Java.

So I’m going to work in Java, building desktop applications…one question remained…. SWT or Swing?  I spent a day exploring SWT.  I love Eclipse and Eclipse is written in SWT, maybe that’s the way to go.  Maybe it was the path or tutorials and documentation I started with but I quickly found myself immersed in the RCP deployment side of SWT and not into what it’s like to write applications.  Time is money and wasting time means wasting money.  It was my time (in the evenings) and so it was my money.  I remember the night.  I just said, “forget SWT for now, I’m going to hello world a Swing app and see how it feels”

It felt odd.

It felt slow.

But, it was progress…I could see the next step.  And the step after that.  Within the month I had a production application running in Swing.  It was a clever app that heat mapped the open interest in all Natural Gas options traded on the CME.  I had a nifty name (Scout) and it worked.

My next technical horizon had been met.  I was building Swing apps.  Things have changed since then, I’ve integrated the following frameworks with immense success:





JMS?  I took that single server I had and turned it into a messaging hub.  It runs ActiveMQ and is the messaging centre for the trade floor I’m working on.  Classic PubSub, it broadcasts the realtime prices in PJM, MISO, CAISO.  It publishes Hydro Data, weather data…you name it.

So that’s where Durable Methods has been for the past 6+ years. Transforming our context and our technology stack.  It’s been an amazing ride and things only continue to improve.

My plan is to write some articles in the coming months outlining generic uses I have (my work is proprietary and the details can’t be published) for Swing and some jumpstarts to get any Swing-curious developers on their way.

JSF reference to it’s backing bean missing

Tools can be the bane of my existence sometimes…

IBM’s Rational Application Developer (RAD) handles the voodoo between your JSF JSP and the backing bean (pagecode). It maintains this reference with a commented line atop your JSP that looks something like this:

<%-- jsf:pagecode language="java" location="/JavaSource/pagecode/View.java" --%><%-- /jsf:pagecode --%>

This tells the IBM PageDesigner that the methods/members of that particular page reside in a java file called ‘View.java’…simple enough.

I went to use their tool to add a button on a different JSP (edit.jsp). When I clicked the ‘Quick Edit’ tab (which should create the action method for me in the backing bean), I could only see javascript methods available to me.


sure enough, I check the source of the JSP and wouldn’t you know it, RAD forgot to add:

<%-- jsf:pagecode language="java" location="/JavaSource/pagecode/Edit.java" --%><%-- /jsf:pagecode --%>

to the top of the page. There was no way for the Page Designer to figure out what backing bean to place the new method in.

Hope this helps anyone unfortunate enough to see this issue in their project.

java.lang.UnsatisfiedLinkError: no ocijdbc9 in java.library.path

So I’m trying to configure a servlet running on Tomcat to connect to IBM’s DB2 Content Management system (via II4C). I can get it to run under WebSphere using Rational Application Developer but seeing as I want to buy a MacBook Pro for my development, I figure I need to start using a more vanilla toolset. Martin Fowler‘s Ravine post served as part of the inspiration. I actually believe it’s a good idea to build J2EE apps using a variety of tools…proves the openstandardiness of my work. The fact that I’m calling a client app from a server (II4C is a windows installed app) disproves that a little but whatever…the next stage will be abstracting that beastie behind some enterprise service layer and that’s that

The error comes from the driver’s need for the presence of ocijdbc9 in the container’s library path.  Yeah, I know the use of Oracle’s thin driver eliminates the need to this coupling but in this case, I’m working with someone else’s project…they’re using this driver.

So add <<oraclehomedirectory>>/bin to the container’s Libary path and you’re back in the game…

Words collide

I think one of the most important virtues of a software professional is being fearless. Someone can lack fear for many reasons:

1) They’re naive

2) They’re stupid

3) They’re experienced

A naive or stupid person is either naturally or consciously ignorant of the risk involved with the activity they’re about to embark on. The first can be forgiven until becomes a trend (in which case they’re starting to become the second).

J.B. Rainsberger once wrote (in JUnit Recipes):

Test until fear turns to boredom.

Taken up a level, it means do the little things that help you become fearless. Don’t just act without fear.

Not quite what I expected…JSF & Validation

So I’m working on a simple search screen (for querying LDAP attributes). There is a Googley type interface (simple box with a search button beside it). The LDAP throws a fit if you search on ‘null’ (the value assigned to a text box in JSF when no value has been entered). JSF field tags support a ‘required’ attribute which will validate them. The validation message is, well, brutal.

‘Validation Error: Input Required’

No big deal…I’ve coded lots of custom JSF validators before (for other types of validation), I’ll put a validation method on the pagecode, reference it in the field and be done with it. Here’s the method I created:

public void validateQueryString(FacesContext context, UIComponent toValidate, Object value) {

String queryString = (String) value;

if (queryString == null || queryString.trim().length()<1){
((UIInput) toValidate).setValid(false);
FacesMessage message = new FacesMessage("I'm sorry, I can't make something from nothing...please enter a search string");
context.addMessage(toValidate.getClientId(context), message);

If you’re not familiar with JSF, let me walk you through…it casts the object being validated into a String and then checks to see if it’s null or empty…if it is, it sets ths validator to invalid, binds a message to the field and drops out of the Validation phase of the lifecycle.

I brought up the form, hit ‘Search’ without entering a value in the field and expected to see my cheeky validation message…I see a roar of red text ripping along the console…the Validation passed, the LDAP is getting queried?!?!  That wasn’t supposed to happen
I checked the method name and confirmed it was the same as the reference on the jsp…yep, matched.

I tried again, only this time, I entered a single space in the text field (knowing I’d trim the string and therefore fail validation)…the method was called, my cheeky message appeared…hmmm.  Validation failed as expected

I was coming to the conclusion that JSF doesn’t event create the field for validation if it’s null when I stumbled upon this tread.

Are you kidding me?!

The part I really don’t understand is the response from an architect:


This is what our architect had to say regarding the validator not getting invoked when no value has been entered in a field:

“JSF validators are *not* invoked on empty fields. The general reasoning is that, if there is no input data, then there is nothing to validate. Therefore, the components are working as they should for this use case.”

Hope this helps

Creator Team.”

I’m scratching my head on this one…not because it’s a road block (I could just live with the Eastern block type validation message), but because it throws off what I believed was a pretty solid validation framework.

JSF is self centered when it comes to validation (it wants to do ALL the heavy lifting) which is something I’ve learned to live with. It makes validation testing a pain but again, you learn to live with the cards you’re dealt. I figured I could live with JSF Validation provided I had full control over how it’s used and now I don’t believe that to be the case.

If anyone out there can shed some light on my interpretation of JSF validation on empty fields, I’d really appreciate your input.

Bridging the Gap – The File Upload saga continues…

Last week on LOST….
What’s the issue with a File Upload control in a WebSphere Portlet (and please, if you’ve done something like this and know an easier way, I’m all ears)?

WebSphere portlets aren’t keen (they don’t listen for) forms that use multi-part encoding. The portlet action gets ignored, your file delivered to the heavens and your customer left feeling blue (IBM Blue?).

So the first part of the two part problem is getting the file up to the server. The second involves what to do with it once it arrives. The image files needs to be persisted to the user’s Member record inside LDAP.

Portal provides a nice facade for dealing with Members. It’s called WMM (WebSphere Member Manager). WMM masks things like attribute mapping and LDAP syntax, it’s just the gate keeper. Works nicely…provided you’re within the Portal context (making a Portlet request).

Let me try tying the two issues together with the solution I devised.

The Portlet now calls an external (from the Portal context) Servlet to grab and parse the file (using Apache’s wonderful FileUpload library from their Commons project).

So now I have the file…how do I get it into a valid WMM context (knowing the receipt of the file came via HTTPServletRequest rather than Portlet Request)?

Here’s a little pattern I tried, using Observer. I call it the PhotoBooth/PhotoLab pattern.

When the portlet’s Session bean instantiates from the Portlet, I create an Envelope. The Envelope contains the user’s unique id and a property to house the image (byte[]). The Envelope is then ‘dropped off’ at the PhotoLab, a singleton with a Map of Envelopes. The Map’s key is the unique ID. So the empty envelope has been left at the PhotoLab. When the FileUploadServlet receives and parses the file, it asks the PhotoLab for the envelope (the key was a parameter passed to the FileUpload servlet). The photo gets placed into the envelope and then guess what happens?

Yep, the trusty old Observer pattern kicks in. When the image setter is called on the envelope, the state gets changed (calling setChanged(); ) and then Observers are notified: The Session bean’s update() method gets called and the envelope is dropped of.

The Portlet now has the photo and can use WMM to persist it when the user clicks ‘Save Profile’ (via PortletRequest)

I’m happy with how it works, I just need to clean up the implementation a little bit. Ted O’Grady did a great job of defining the code’s current state as the Speculative Solution…time to clean that speculation up a bit and get the Durable code in place.

2 Portlet Specs and URL encoding…

Still working on the file upload portlet for my current client. I’ve decided to change gears and parse the request in a servlet rather than the portlet itself. Why? I think it will allow me to create a second form on the Portlet’s JSP and encode it in whatever way I need.

The portlet complies with JSR168, not the WebSphere portlet API. Seeing as it’s portal, I need to encode the URL to the servlet…easy enough.

String servletURL=response.encodeURL("/UploadServlet");

and pass this variable into the form’s action attribute and I’m done.

…except…I get a “Page Not Found” error and the following shows up on the console:

Servlet Request Processor Exception: Virtual Host/WebGroup Not Found ….


So I could go into everything I tried in figuring out why this very simple little example didn’t work but I’ll save you the time and anguish. Turns out the IBM’er beside the IBM’er I was working with knew what the issue was….had he chimed in a little earlier (hint for the future Jimmy), we would have saved a little grief but I’m still grateful for his involvement.

Turns out a JSR168 portlet needs an additional bit of magic to encode a url. You need to use the following:

String servletURL=response.encodeURL(renderRequest.getContextPath()+ "/UploadServlet");

with renderRequest doing the heavy lifting. Add that and you’re gold.



Strings…every application has/needs/uses them. I’m using Rational Application Developer, IBM’s IDE that runs up on top of Eclipse. It’s typical IBM software…big, a little slow and somewhat dependable (I believe that’s classified as a backhanded compliment).

Anyhow, RAD (as the cool people call it) has the standard Eclipse based SOURCE options. One being ‘Externalize Strings’. I’ve seen it on the menu a thousand times but never used it. That changed the other day. I had some spare time and some LDAP Attributes referenced from within the code. I highlighted the string “cn” and clicked “Externalize Strings”…biz…pop…bang, a Wizard pops up and presents me with a screen asking what I’d like to use for a key to reference the string (which, ironically enough…is a String). Smokes starts billowing from the exhaust vent, the drive grinds away and presto…a nice little class for grabbing a resource bundle (populated) for the string. My old reference to “cn”

String commonName = getAttribute("cn");

has been replaced with

String commonName = getAttribute(Messages.getString("TransAltaPumaProfile.cn");

so that’s that…mission accomplished, right?

I read an article recently that asked readers what they spend more time doing, reading software or writing it. You know what…the answer is pretty obvious. We read our code far more often than we write it. That’s my first issue with the output of “Externalize Strings”, it doesn’t read nicely. It takes the reader away from the context of their task. Yes you can name the Class for retrieving strings something more meaningful than Messages. A small gain. The Genericists (I believe I made that word up) will tell you not to be too specific with your message bundle…context lost.

That’s my first issue with it. My second, and more significant, issue is the potential for lost time in debugging a change to the ‘key’ string. Whether you like it or not, Java is a compiled language. There are benefits to living with a compiler. It’s loves to complain. If you accidentally (for those of us who make mistakes) change the “TransAltaPumaProfile.cn” to “TransAltaPumaProfile.cnm”, what does your compiler do? That’s right, nothing. It takes the “you want to play with dirty little strings, go for it…I won’t stop you until runtime…but when one of them comes into my room, to play with my stuff…it better be right” attitude. I thought about this and the first thing that came to mind was, “whoa…bad news…I better take those keys, the magic ingredient to the entire ordeal, and put them in an interface somewhere…really lock ’em down”

Yep, “File/New/Interfac….wait a second” what was I thinking? This was all to satisfy some frail bit of whiz bang provided by my IDE.

So here’s what I did. Wrote an abstract class with some public final static strings on them and referenced those in the code. Now, this line of code:

String commonName =getAttribute(Messages.getString("TransAltaPumaProfile.cn");

looks like this

String commonName = getAttribute(LDAPAttributes.COMMON_NAME);

If I accidentally (for those of us who make the occassional mistake) change “LDAPAttributes.COMMON_NAME” to “LDAPAttributes.COMMON_NAMES”, the compiler (Felix) tells me (Oscar) that I screwed up, that the referenced String doesn’t exist and that I need to seek professional help.

I also like this approach over an interface since I don’t need to declare the implementation of the interface in order to use the String. That’s always bugged me. There’s a certain pipeline application out there with a barrage of Strings held within interfaces. I won’t name names but someone had us do a lot of interface String work on that gig…likely an idea that came to him while out for a ‘jog’ (that’s right, I said jog).

File Upload, Portal, JSF and not being surprised…

I’m writing a portlet that allows a user to modify certain LDAP attributes.  It’s a nice little solution (if I do say so myself), wrapping IBM’s PUMA framework with something a little more durable, a little more generic and a lot less IBM.

The customer would like the user to upload their own picture.

“Yeah, shouldn’t be a problem…” I say, recalling the File Upload control on Rational Application Developer’s JSF palette…famous last thoughts.

IBM, in their wisdom and glory, have shipped some JSF components that don’t work with their Portal solution.  They’re available for use in a Portal project, they just don’t work… (bonus points to me for looking it up in the documentation rather than bugging the local IBM on-site consultant).  I guess that makes sense…

This means a trip to Apache and a chance to try the Commons File Upload component (apparently it works with Portal…I won’t get into why I find this entire thing so hilarious).