FitNesse is dead - Long live FitNesse

Wow, we have been so busy for the last 7 weeks i haven't had time to update the blog at all, also it has been pointed out to me that the instructions for getting up and running with FitNesse wont work as is because i made some assumptions, so i will re hash that post soon and add a download that contains a ready-2-run FitNesse instance complete with working examples.

So what have we been up to?

ThoughtWorks are back, and this time people who didn't get exposure to the Agile way on the last project are running the project this time. We have also gone "all out" so we are doing the whole pair programming and BA and QA paring thing which meant we had to radically change our physical environment to accommodate such practices.

The TW test lead on the project had suggested we try their new tool "Twist" instead of FitNesse, and i thought, oh right another sales pitch, and asked them "what is the problem we have today that using Twist will solve". Answer? we have out grown FitNesse.

When i looked at the main fixture we are using for most of our tests (the webtest fixture) we had extended it that much, that 75% of the code in there was ours! So i had to evaluate why are we using FitNesse?? Answer, because it was a quick and easy way (not really that easy) to implement automated browser testing for our last project. Now however, we have a good understanding of driving selenium (born out of fixing the [apparent] inadequacies of webFixture [for our needs at least]).
So with our new skills (the ability to write tight reusable selenium tests), why would we want to preserve FitNesse?

Anyone who has read my ramblings will (hopefully) have read my post [i hate fitNesse / FitNesse Rocks], so you should be aware that we have a torrid love hate relationship with FitNesse already. There are lots of things wrong with FitNesse. A lack of historical test metrics is just one feature, but my main gripe is this:

FitNesse -> Fit -> Fixture -> Selenium

That is just too much overhead (read bloat). If we have the skills to write solid selenium tests directly then that's what we should do:

OurTests -> selenium

It is more important now because we want to run the tests as part of an automated build pipeline, and every millisecond counts. In the time it takes FitNesse to start up, i can run 3 raw selenium tests. ergo i need to take FitNesse and Fit out of the equation.

But wait that's not all...

As i've stated before, not all of my team posses the skills required to dive straight into the Java code and write selenium tests (nor do i want them to, they are testers and not developers after all). This is where FitNesse scores its points. It allows a non coder to write the tests in english prose (or near enough) and a coder and intrument the tests afterwards. However, when you begin to work like this versioning can become problematic very quickly. The problem becomes exasperated when a broken QA test causes the automated build to fail.

So what can Twist do that FitNesse cant?

Twist is built upon the Eclipse IDE with some important additions. Our developers already use Eclipse, and so we have been using Eclipse to maintain the webfixture. ergo, we are comfortable using Eclipse. It has CVS (SVN via an add on) integration built in, so that's versioning problem we had with FitNesse taken care of, but the real power of twist is its ability to allow non tech people (like BAs for example) to write prose like acceptance tests and save them, without writing any code.

A tester or developer can then check out the test on his Twist instance and instrument the test (Note: This can be achieved by recording the test in firefox via the selenium plug-in). So we still have the ability of writing tests in "English", but without having to learn any Wiki mark-up (bonus, also you don't have to be precious about what characters is in your test, as it wont be interpreted as wiki mark-up), and as soon as the test is instrumented it can be saved to the code repository and hey presto the version control is taken care of and the latest version of the test can be used in the automated build.

And finally the lack of historical test metrics. Because we will run the tests as part of the build, historical metrics are taken care of. The build server will maintain the metrics against each build, and the builds will be frequent.

However. Twist is still in Beta. Its a licensed product so we have to stump up some cash, and there will be another learning curve for us to tackle. We have had "fun" today trying to get it to launch selenium using our proxy with a list of hosts to ignore.

Our next challenge is to get the output of the build metrics displayed on a moving message LED display so that the culprit of a broken build will be named and shamed...