Life after ThoughtWorks?

After working with ThoughtWorks I’m completely sold on Agile software development for the type of product we deliver. Which if you know me may be a bit of a surprise as I have a reputation for being a cynical old curmudgeon, and a staunch doubter to boot.

While I am still unsure if agile would work for large scale software projects (think air traffic control) I am certain that there are agile practices (lean for example) that could be pilfered and applied to those lumbering projects too. If you are reading this and you haven’t had any exposure to Agile, I guess the biggest thing I should tell you is that this isn’t some process you just pick up and run with, nor is it a methodology to apply. You can’t learn it from a book, or by a project template; it is quite simply a mindset and as such it presents the practitioner with quite a shift in paradigm. Simply put, you can’t flick a big agile switch and hey presto you are Agile; it is more subtle than that.

I think before I continue it’s important that I clarify something. The term agile is applied to a wide variety of processes, techniques, methods, tools, practices, projects, and phases of the development life cycle; it has become a buzzword used by people trying to paint their work in a new light (who doesn't want to be known as being agile?). It's important, therefore, to set out some basic definitions and context for the use of the term "agile," especially as i will use it constantly throughout this article.

Within the context of software development, the term "agile" (with a small "a") is meant to imply that the development team is nimble, flexible and responsive to the business needs, and that it is able to adopt new technologies and techniques that can improve software delivery. The term "Agile" (with a capital "A") refers to a very specific set of processes (and i use the term process as more of a place holder) applied to software development that have evolved over the past fifteen years or so; including some you have probably heard of, like eXtreme Programming (XP), Scrum, Feature-driven Development (FDD), Crystal, Dynamic Systems Development Method (DSDM) and Lean Software Development. A non-profit organisation The Agile Alliance was created by the ideas people behind most of the Agile processes. The Agile Alliance promotes a set of core values that a process must follow to be called Agile:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

So to be Agile then, a process must support these values (and more), albeit in diverse ways. Some, processes for example like Scrum, address team management, while others such as XP or DSDM, address development activities or other activities of the software development life cycle. It’s worth making a mental note that users of Agile processes do not have to follow all its Agile practices, and neither does the use of one process preclude the use of any other. One thing I learnt is that Agile supports such process change, if a particular way of working is not working then change. In fact many I found that many of the Agile practices are complimentary.

No matter your preference, all of the different flavours of Agile will deliver working functionality in short time-boxed iterations. They implement early and frequent testing. They require lots of involvement from the customer on a frequent if not full-time basis and they assume that the customers requirements will continually change.

So why all the fuss about big "A" little "a" when talking about Agile? There are two main reasons:

1. Companies who adopt Agile processes have to be prepared to completely change the way they not only develop software but how they think, and this change is so big it is very hard to do. However companies that have achieved this shift will appreciate the significant benefits they can reap in productivity, quality and value of the software that they deliver. Notice i didn't say it would be faster. These companies are Agile with a capital A.
   


2. Companies that are not able to embark on this level of change can still become more reactive and flexible in the ways that they build software. They become more agile and begin to realise the advantage that Agile can deliver. This toe dipping exercise can lead to a true Agile team, however the company must understand that it's not the prefferable way, and they would benefit from the gung-ho approach to Agile adoption.

To reiterate what i've said, at a minimum, every Agile process delivers working functionality in short, time-boxed iterations. Agile implements early and frequent testing, and it involves the customer on a frequent if not full-time basis; and assumes that requirements cannot be fully defined at the start of a project, and that the requirements will continually change. The ethos is simple that by using these practices, the development teams will be able to respond quickly to the ever changing customer priorities and feedback, and deliver value to the business. This is often missunderstood as being quicker, I would prefer to describe it as improved time to benefits.

As a tester I have become quite accustomed to being involved late in a project, often right before delivery. I have adapted and learnt how to cope with shortened time frames for testing, and receiving specifications and requirement documentation that don’t actually match the product delivered in QA. Typically the business (or for that matter the project team) usually has little interest in the test teams input, that is until they call us a bottle neck.

Feeling loved and appreciated.

Agile software development for a tester is radically different from the traditional PRINCE(2)/waterfall software development lifecycle (SDLC), because it throws QA right into the heart of project on day one. As Agile testers we suddenly found ourselves being involved in the analysis and design of the product. We became heavily involved in decision-making throughout the whole project and because the delivery of the software is incremental we found that we began testing at the very beginning of the project and that we had to maintain pace and keep step with development to prevent any delay. This is a far cry from waiting for a software deliverable (possibly unfit for purpose) to be thrown over the fence a few weeks before go live.

The paradigm shift I have mentioned is a large one, not just for QA but the whole project team, because our QA team now drives the entire software development process.

Quality assurance, with its focus on preventing defects, is translated into the agile practice of having committed QA resources on the development team that participate in decision-making on a daily basis, throughout the life cycle of the project.

Their input during elaboration and design helps developers write better code. More “what-if” scenarios are considered and planned for, as the collaboration between coders and testers gives the coders more insights than if they were to have planned the work on their own.
Likewise, the testers gain added insight into the expected functionality from the coders and the product owner, and are able to write more effective test cases for the product.
Quality control places its emphasis on finding defects that have already slipped into the system, and working with developers to eliminate those defects. This bug-checking is done within the iteration, using such techniques such as daily builds and smoke tests, automated regression testing, unit testing, functional and exploratory testing, and acceptance testing. Everyone participates – no one is exempt from the tasks of ensuring that the feature coded meets the customer’s expectations.

Your role morphs and evolves.

Through a methodology known as "Story Test Driven Development" the test requirements (aka the acceptance tests) are captured (we currently use Twist) in a test like format and they are then augmented to make them into automated tests. Nothing unusual in that, lots of test teams create automated tests, however here the automated tests are being executed by the development team not the test team and the tests exist before the development team have even created the code for the software the team is delivering. The real beauty of this method is that the development team can then integrate the automated tests into a Continuous Integration environment, where the newly checked in code is built and tested automatically. So the QA teams test suite is run against the code every time a change gets checked in. But wait, that’s not all. The delivery of code into QA can not happen until it has passed all of our tests, which means that we have built the quality in from the very start. Let me state that again, in case you missed it. The development team can not deliver code into QA until it has passed the QA teams tests. That’s a statement that should raise a lot of internal debate with any passionate tester who hasn’t worked in this way before, and it did with us. Its worth me making the distinction here that Test Driven Development (TDD) is not a testing methodology it’s a design methodology. Being testers we have a very different view of the world to a developer, its in very our nature. Working this way felt like we had harnessed our power and put it where it belongs, under the smelting pot of code.
I should also mention here that the developers were also working in a new way, because they worked in pairs (pair programming) and also used TDD at unit level. This means that a developer has to write a unit test before he writes the code for the unit being developed. That means then, that before its delivered to QA, our code has been tested twice. That’s two times tested.

So if development is running the QA teams tests, and the QA team are writing the automated tests (and its blurry line between coding and writing an automated test), and the code is tested before its delivered to QA, a whole load of question begin to bubble up.

Some of early questions we had:

1. Who tests the testers’ tests?
2. This is agile, so what happens if the requirements change?
3. So what do QA test if it’s already passed the QA tests?

On a typical PRINCE(2)/waterfall project the testing team plans as far in advance as it can. We normally try to follow the IEEE 829-1983 standard and document as much as we can.
The documentation covers how we approach the testing and detail the testing activities. These documents are usually created in isolation from the project team and may be published for approval. However in my experience the documents are rarely scrutinised and any feedback given only serves to pay lip service to the processes. Another checkbox checked, and another Gant chart milestone met.

Working in an agile testing environment also has a requirement to define which tools and methods will be used for writing, executing, and reporting tests and determine the best approach to testing, and the scope of that testing. The big difference is that the whole team is engaged in this deterministic process and we found that it was important to engage the developers in this definition, because they would be executing our tests and writing their own unit tests. Moreover thought had to be given to automating the regression testing, something that would happen as part of the continuous integration process.
The business stakeholders were also involved in this process (unfortunately only by proxy through the business analysts, as they are physically located in a different part of the country) as they would help to define and run the acceptance tests. In agile, we (the whole team) all test, but the business accepts.

In short within agile practices, everyone has a contributory part in defining, upholding, and improving the quality of the product.

One of the gotchas was that we found that we needed to become more technical. We had thought that we were already more technically skilled than your average tester, however as we endeavoured to automate our testing we found that we had to skill up and learn not just how to write code (Java for us), but compile the code and version control it. They were steep, steep learning curves, which now we have overcome them, have empowered us, giving us the tools for a brighter, faster and more accurate future. Having said that, it does appear that some of this coding effort could be considered a one off setup task, because we now have a framework of “tools” that cover all of the tasks we need to help us execute an automated test. Couldn’t we have asked the development team to do the tech tasks for us? We did, and they didn’t have any resource free to accommodate our needs as well as those of the agile project. Regardless, we have skills now that we have been able to share with the wider QA team, and they too are seeing the fruits of our initial labour.

Traditional Tools Solve Traditional Problems in Traditional Contexts. Agile Is Not Traditional.

Traditional, heavyweight, record-and-playback tools (like Quality Center) address the challenges faced by teams operating in a traditional context with specialisms. They address the challenge of having non-programmers automate tests by having record-and-playback features, a simplified editing environment, and a simplified programming language.

But Agile teams don’t need tools like these (optimised for non-programmers). What Agile test teams need are tools to solve an entirely different set of challenges that are related to collaborating, communicating, reducing the waste (Muda), and decrease the feedback loop. Ergo, traditional (long standing) test automation tools just don’t cut the mustard in an Agile context because they are designed to solve traditional problems, in traditional contexts and those really are quite different to the challenges faced by Agile test teams. To make it clear, QC & TD isn't going to cut it in Agile.

At the Google Tech Talks December 9, 2005 Elisabeth Hendrickson gave a talk on how as more teams are adopting Agile practices such as XP and Scrum, software testing teams are being asked to become "Agile" as well... View it here

Using Twist With Different Selenium Versions

When you start using Twist you will find that the twist team have baked selenium into twist.
You will also find that the version of selenium that has been integrated has some quirks.

Here is a simple guide for using any Selenium version with Twist.

You can also use this guide to help you implement other drivers like WebDriver.

Step 1
Add the selenium-java-client-driver.jar and the selenium-server.jar of your preferred selenium release (I'm going to be using my old friend selenium.0.9.2) to the Twist project class path, make sure these jars are loaded first in the class path order.


Step 2
You need to create a factory class that will create the Selenium instance for your test suite.

For example, let’s create "SeleniumFactory.java" in the Twist source folder.

import java.io.FileInputStream; import java.util.Properties;
import org.apache.tools.ant.types.Commandline;
import org.openqa.selenium.server.RemoteControlConfiguration;
import org.openqa.selenium.server.SeleniumServer;
import org.openqa.selenium.server.cli.RemoteControlLauncher;
import com.thoughtworks.selenium.DefaultSelenium;
import com.thoughtworks.selenium.Selenium;

public class SeleniumFactory { private SeleniumServer server; private Selenium selenium; }

public void start() {
try {
Properties properties = new Properties();
properties.load(new FileInputStream(getClass().getClassLoader().getResource("twist.properties").getFile()));
}

String[] serverOptions = Commandline.translateCommandline((String) properties.get("selenium.server.options"));
RemoteControlConfiguration serverConfiguration = RemoteControlLauncher.parseLauncherOptions(serverOptions);
String browserLauncher = (String) properties.get("selenium.browserLauncher");
String browserURL = (String) properties.get("selenium.browserURL");

server = new SeleniumServer(serverConfiguration);
server.start();

selenium = new DefaultSelenium("localhost", serverConfiguration.getPort(), browserLauncher, browserURL);
selenium.start();

} catch (Exception e) {
throw new RuntimeException(e);
}

public void stop() {
try {
if (selenium != null) {
selenium.stop();
}
} finally {
if (server != null) {
server.stop();
}
}
}

public Selenium geSelenium() {
return selenium;
}


Step 3
We now need to remove any bean definitions with id="seleniumFactory" and id="selenium" from the applicationContext-suite.xml file.
You will have to manually edit, the "applicationContext-suite.xml" of the twist project, and add the following bean definitions.

Step 4
The workflows will now have to depend on the Selenium interface.

For example,
public NewWorkflow(Selenium selenium) {
this.selenium = selenium;
}

With the above changes, your scenarios will be now use the selenium server and driver jars included in “step a”.

Note:

The selenium options are not the same between releases. The beta-2 release of Selenium RC contains some significant changes you should be aware of. You will need to account for these and change these values in "twist.properties" as required.

For example:

selenium 0.9.2 config : selenium.browserLauncher = *firefox
selenium 1.o Beta 2 config : selenium.browserLauncher = *firefoxproxy

selenium 0.9.2 config : selenium.server.options = -port 4545 -avoidProxy
selenium 1.o Beta 2 config : selenium.server.options = -port 4545 -avoidProxy -honor-system-proxy -singleWindow

For a full list of the changes take a look at http://clearspace.openqa.org/community/selenium/blog/2009/01/13/selenium-rc-beta-2-goodies-and-gotchas

Choosing Twist - Part two

The next Big Thing

Enter our next big project. We (the QA team) held a huddle to talk about the capability of Fit and FitNesse for the new project. We all agreed that we would sooner avoid it if at all possible, in fact the language used during the meeting was much stronger that that. It was around that time I was shown a demo install of Twist, and immediately asked the question to the guy showing me “what is the problem I have with FitNesse that Twist will resolve?”, he looked at me as if I was crazy then reeled off all of the pain points that we knew and then some.

It’s worth having a look I thought, and installed the 30 day trial. One of the guys who had been very instrumental in fighting the FitNesse issues also installed it, and we said we would take a look at it and then compare notes.

At first we didn’t get it, but some things are very apparent with Twist.

For those of you that don’t know, Twist is built on top of the eclipse IDE, I should mention here (for existing Eclipse users), that it’s not currently available as a plug-in for an existing eclipse install (however that feature is in the programme plan). That means it comes with the entire feature set that eclipse does, e.g. integration with source control (CVS or SVN for example) albeit via a plug-in. The team write their tests, run them locally to make sure they work, and commit.













Search
You can search across your code, files or project. Search and replace across an open file or the whole project.







Keyword completion.

This last feature is biggie for us. If you create a method call UserOpensUrl then the next time you type v-e-r and press Ctrl+Space it pops up a list of matching methods.

straight away people can see what methods that have the phrase “ver” in them have already been written.

If you have used eclipse before, then all the usual features and short cut keys you love are available (I’m still learning new ones every day).

But what about twist? What does that bring to the party?

The most striking thing is the WYSYWIG scenario editor.














If you have grappled with fitnesse then the ability to make text Bold and Italic with the click of a button is great.




Not having to remember any mark-up language allows you to concentrate on the task in hand, writing tests.

The scenarios are broken down into several areas. The first part is where you write your test prose, the purpose or intent of the test. The second part is where the test proper is written.



Tests are currently written using either bullet points or tables.

And just like fitnesse a passed test turns green.




The stats for all the tests are given in a separate panel

The scenario editor allows a scenario to have one or more tags associated with it. These can be tags such as “QA Complete” or “shopping cart” or “smoke” or all three or none. These tags come into their own when you only want to run particular types of test. E.g. if you only want to execute tests that are QA Complete and are for the Shopping Cart area of your site; this is especially useful for CI builds.

Tags can be applied at a scenario level (while you are editing a scenario) or then can be applied or removed en masse.



When you have written your tests they need to be instrumented.
This can be achieved in a number of ways. Either right click and select Quick Fix from the context menu


Or use the quick-key combination of Ctrl+1 and you are presented with another context menu

“Create method” simply creates an appropriately named empty method ready for you to work on.

Again this is great where you don’t have development skills within your team as your team can focus on writing the scenarios and the dev team work on instrumenting the empty methods.


Choosing Record from the context menu starts a selenium server (which only currently supports Firefox) and fires up firefox. The record system differs a liitle bit in that it first runs though any previous steps in test if you have written any, then begins recording at the new point in your test, and completes the method.


I have only touched on a few good points here, those which counter the main pain points we had with Fitnesse. One of the communities biggest gripes with Twist is the lack of good documentation. Which when you consider it is a paid for product is a little shameful.

i should also point out that while i was writing this post, ThoughtWorks Studios release the first GA version of Twist, which i have yet to try out....

Ferrograph SDX - scrolling LED sign

One important aspect of being Agile is have short feedback loops. Wherever possible we aspire to reduce the feedback times, and one aid in decreasing feedback time is a feedback monitor. Having a some sort of visual indicator for the state of the project is obviously valuable. Its believed that if used correctly it can boost productivity and morale within the team considerably, however there is a certain amount of Kudos to be gained for having anything more than a VGA screen.

If you have a quick Google for Extreme Feedback Devices (no i'm not talking about a Marshall JCM800, 4x12" cab and a Gibson Les Paul) will turn up a large amount of hits for glowing orbs, VGA screens, lava lamps, lots of bespoke devices powered by Atmel AVRs, devices controlled by X10, water features, gummi bears and so on. One post by Dirk Ziegelmeier caught my eye, he had used a scrolling LED message board.

I picked up an old Ferrograph Aurora 64 SDX moving message / LED wall board, the type that is typically used in a call centre environment. In fact our call centres employ these boards to show the inbound call queue, and average call waiting times etc. for the Avaya Index ACD system.

The Ferrograph SDX boards turn up quite frequently on Ebay, and i had (somewhat naively) assumed that there would be enough info on the internet for me to be able to integrate the sign into our build monitor. Like i say, i had assumed, and as Eric Bogosian put it so eloquently in "Under Siege 2: Dark Territory (1995)" , "assumption is the mother of all f**ck ups".

The sign came with a short pig tail of cat5 cable terminated in an RJ45 plug. I have seen these signs listed on eBay as being "networked", this is an incorrect assumption being made by the seller as this is not a network cable but a serial cable. In my case the sign had been configured for RS422. Being a hardware geek more than a software geek, meant i had the sign opened up and running on RS232 in a matter of minutes.

Inside the sign is a main-board, at the end of which there is an RS422 header and RS232 header. Although there didn't seem to be a pin out that i could see printed on the PCB i traced back a little bit to work out which pin was which (a guesstimate based on where the pins ended up), i soldered up a 9pin D-connector to three wires (gnd, Rx, Tx) and stuffed it into the back of my PC. I took a guess at 9-n-1 (as its fairly standard HW setting) and sent "hello world" using the Alpha 1-byte protocol. The sign displayed nothing, nada, zip, bupkis, diddly-squat, zilch, bugger.

A long hard trawl of the internet turned up nothing. A few posts on various "Q&A" sites asking for info on these boards, but little else. A few lunchtimes of hunting and i came across a posting on makezine where Robert Coward said he had successfully reverse engineered the SDX signs.

I dropped Robert a line, and he very generously sent me the secret i needed to know for me to be able to get "hello world" onto the sign.

Because the Ferrograph SDX signs were designed for call centre ACD systems using the Avaya Index telephone system, it expect to be sent the phrase "SDX" at the start of every packet. It was that one nugget of information that allowed me to get going with the SDX wallboard but it was Roberts reverse engineering that really enabled me to push the sign to its limits - which took all of about 5 minutes!

There are many things wrong with the SDX sign, in fact possibly more defects than working features. I'm guessing that when Ferrograph tested the sign they only tested the features that would be needed for the sign to operate with the ACD system and not the features that are typical of the Betabrite or Alpha LED moving message led displays.

Anyhoo, long story short, Robert has created some new firmware ADF for the board, that does the following amazing things:

* Conforms to the publicly available Alpha protocol. It works with ready made applications, and it's easy to write your own applications for it.
* It has lots of fancy effects (snow, dissolve, drop down, cursor wipe and many more), and smooth continual scrolling, as well as all standard wipes/scrolls.
* Supports small/large normal and fancy character fonts; one or two line operation for small fonts. Double wide, flashing and colour shift flashing options available.
* Automatic word fit and centering to ensure a presentable display at all times.
* Many colour combinations, including three colour rainbow and two colour stripes.
* Full support for pictures and animations; configurable animation speed.
* Sophisticated & flexible memory management.
* Full real time and date display support in various formats.
* Ability to set messages to appear and disappear at certain times/days.
* Control of two optically isolated misc IO signals available on some Aurora 63 units, allowing automatic control of external buzzers, lights and even mains appliances (via suitable interface hardware) in synchronisation with message displays.
* Serial readback for message data and system/error status.
* Works on RS232 or RS485 interfaces.
* Serial timeout message option to display an error or blank message if host serial comms fails.
* Automatically detects Z8S180 CPUs on modern units and switches to high speed internally for maximum performance. Also automatically handles boards hard wired to double speed operation.

I'm sure you will agree that its some features list. Robert has turned what was previously a skippable board into a fantastic fully featured LED scrolling message board that would cost thousands of pounds.

A big feature here is that the with the ADF firmware the sign talks Alpha protocol, 1-byte, 2-byte and 3-byte. The Alpha protocol is publicly available and therefore there is much info available on the internet about it including sample code.

For example, in Linux i can:

echo "_01Z00_02A0hello world_04" > /dev/ttyS0

I contacted Robert again and purchased a copy of the firmware off him, which he supplied on an EEPROM ready to drop straight into the mainboard of the sign.
I powered up the sign and put and sent it "hello world" and basked in the 3 colour glow. within minutes a small crowd had gathered around my desk, Ooh-ing and Ahh-ing at the effects. You see if you sent the old SDX firmware "hello world", that's what you got however with Roberts ADF firmware, the sign centres the text and positions it in the middle of the two rows, then applies various effects to the text. The crowd were pleased.

Ok, so know i needed to make this sign work hard, i needed it to parse the cctray.xml that is generated by Cruise. Surely this is something i'd find ready made on the t'interweb? Well yes and no. Lots of people are parsing the xml that is output from Cruise Control, but we are using Cruise, and cctray.xml was what i wanted to parse (at first at least). But that as they say, is another story.

If you have a Ferrograph Aurora (Aurora 62, 63 or 64, new or old hardware type) SDX display / wallboard and you are interested in Roberts firmware you can contact him directly via email: robertcoward{at}gmail{dot}com