. 5
( 10)


lem with telling developers what he felt they needed to know about the quality
of the application. He also tuned in very quickly to where he could add value. He
was surrounded by automated tests, but he saw developers being “too busy” to do
the kind of exploratory testing that a professional tester is so good at. He put the
application through its paces as a skeptical user would, creating interesting sce-
narios and unusual (but not unrealistic) usage sequences. He considered usability,
Agile Testing: How to Succeed in an Extreme Testing Environment

noting how many mouse-clicks it took to perform common operations. He measured
performance. He changed browsers and browser settings, and tried the application
through a mobile phone. He checked how good our graphics looked at different
screen resolutions.
Importantly, John brought lists of his observations, bugs, and anomalies to our
planning meetings for triage. That way, the developers could comment on likely
solutions and estimate how long it would take to address each issue. Edgar was then
able to balance the cost and bene¬t of these against other features he wanted to add
to the product functionality.

17.4 Results of the Agile Approach
Developers became great advocates of testing and helped to advance the state of test
automation in a testing services company. We observed extremely low rates of ¬eld
defects. The product was sold to many blue-chip companies and won an Innovation

Product Success
The ¬rst version of TestStrategist was brought to market after six months of develop-
ment, and the ¬rst sale made soon afterwards to a large multinational company. We
settled into a regular rhythm of bringing out a new release every three months. Only
once did we have to delay a release, by one week. It was surprising how disappointed
we were as a team over this “slip” “ a sign of how proudly we regarded our record of
delivering working software on time.
Edgar, as our internal customer, had the excellent idea of strengthening this
concept by establishing a Client Advisory Board, where one key user representative
from each of our clients could attend a quarterly planning meeting. This was partly a
customer relationship exercise, but it was also a genuine opportunity for our clients
to in¬‚uence the ongoing design and development of the product, as well as get a
preview of the features in each new release. This regular obligation to “face the
buyers” lent a great sense of importance to each release and certainly heightened
the team™s commitment to quality and testing.
The product continued to enjoy success in the market and sold at a healthy price
to big-name companies, including four in the U.K. FTSE top 10. It was also being
used internally as a delivery dashboard on several of our company™s test consultancy
engagements. In 2005 the product won the Information Age award for “Most Effective
Use of IT in Professional Services.”

Test Infected
The proof of the success of the process was that we were delivering valuable software
to paying customers and satisfying their new requirements regularly. Customer
satisfaction was high and we had very low rates of ¬eld defects.
Test-Infecting a Development Team

However, the most remarkable sign of agile success was how we worked as a team
and how it changed our view of what was important in software development. As a
development team, we had made a genuine transition where we no longer thought
of testing as someone else™s job. It was part of how we created good software.
We treated our growing test suites as genuine assets that returned real value on
our investment. They served as both a guidance system during development and a
reassuring safety net during refactoring. Quality was no longer optional, and the
“cost” of writing tests was an essential element that was factored into the estimate
to develop each feature.
We didn™t think about writing code without writing unit tests ¬rst. We found TDD
not only a natural way to develop, but also a safe way to develop. It was likened to the
habit of putting on a seat-belt before driving: once you™ve formed that habit, it feels
unsafe not to do it. We were test-infected, to borrow Erich Gamma™s phrase [63].

Converting Unbelievers
Not only that, we were developers surrounded by a company of testers, and we wanted
to be the advocates of this new age of collaboration-for-quality. We were convinced
that the more developers learned about testing, and the more testers learned about
development (especially the tools and processes that support developer testing), the
better the software industry would become.
Monique, who had at ¬rst been the most vociferous skeptic, became a proud
ambassador of agile and TDD. From joking to her husband that she was joining a
cult, she would later be trying to teach him (and anyone else who would listen)
the virtues of the agile way of working. Owen “ who had cycled his roles through
tester, Build Captain, UI designer, and developer “ showed that his abilities and the
opportunities offered by an agile project were not just a natural ¬t, but a virtuous
cycle of reinforcement. It seemed that the more he learned, the more opportunities
arose for us to improve, and the better and more agile we became as a team.

All true agile projects blur the line between development and maintenance “ releasing
live software early and following it with regular incremental feature releases. Rather
than having a ¬xed project end date, the product owners can “call time” on the
project when the expected returns to be gained from further development do not
exceed the development team cost.
While it is one thing to draw the line on product development economics, it
is more dif¬cult to account for the value that accrues in a high-performing team.
Over and above turning out software, our team was adding value to the company
in the form of new skills, innovative ideas, advanced research, high morale, and
leading-edge development and testing techniques.
When the time ¬nally came for the management to take the decision to wind
down the project and disband the team, the sense of loss ran deep. The team had been
Agile Testing: How to Succeed in an Extreme Testing Environment

through a great deal and had bonded closely. What should have been a fond and cele-
bratory farewell to the project instead felt like an unceremonious and rather sudden
breakup, where cold economics had taken priority over human concerns, without
any consideration for how the investment made in the team could be retained.

17.5 Lessons Learned
Be pragmatic and business focussed. Recognize and respect what is important.
The most surprising results of this experience were how we overcame the initial
skepticism of new processes, converted the unbelievers, opened their minds to the
bene¬ts of testing, and made them hungry for other ways to get quality information
that is timely, relevant, self-evident, and helpful.
For other teams embarking on this journey, these are the main lessons that we
would share.

It™s OK to be agile about your agile adoption. You don™t have to take on all
the practices at once. Starting with the fundamentals (short iterations, simple
design) will get your team into a steady rhythm, and supporting practices will
follow naturally. Understand and prioritize other practices, then fold them into
your process gradually. However, do keep the bigger picture in view: read books
to learn how the method works holistically and how the individual practices
support each other. Don™t expect being “half-agile” to work in the long run.
Start from a position of trust. If most of the team are new to agile development
and testing, ensure that the team members who are experienced also have the
people skills to understand the challenges and concerns the others will face, and
the patience to help them come up to speed. If the whole team is new to agile,
create an atmosphere of mutual respect and safety by getting some kind of team-
bonding activity scheduled as soon as possible. Train the team together. Make it
clear to all that having doubts and asking “stupid” questions is OK. In the long
run this is much better than allowing people to carry on with misconceptions. In
all cases, allow extra time for pairing and agile retrospectives to enable the team
to “own” their process and feel comfortable with all aspects of it.
Respect the testers. Ensure testers are treated as equal members of the team, and
remove any barriers that hinder fast and easy communication between customer,
testers, and developers.
Respect the tests. Do not get into a mindset of “the code is the concern of the
development team, the tests are the concern of the test team.” There is only one
team. That team creates tested code. Tests and code are of equal value and both
are the concerns of the whole team.
Respect the build. Use continuous integration and other automated testing pro-
cesses to your advantage and have them provide fast feedback about code quality,
either daily or on every check-in. But don™t squander the value of that informa-
tion by failing to respond as quickly as it comes. A “broken” build, including any
Test-Infecting a Development Team

failing test that previously passed, should always be the top priority for the team
to ¬x.
Respect the team. The people involved in any agile project are likely to work
more closely than in nonagile projects. Remove impediments that prevent them
from working well together. Encourage them to run their own retrospectives to
improve their own practices. Be supportive of each other when occasionally life
gets in the way. Don™t mess with a team that is already working well together “
the economics of headcount rarely stand up against the power of good team-
18 Agile Success Through Test Automation:
An eXtreme Approach
Jon Tilt, Chief Test Architect, IBM

“Automation, Automation, Automation” was the battle cry of our leaders as we embarked on
our ¬rst “iterative” mission with a large team of twenty testers; within our agile iterative test
approach, we have invested heavily in test automation, unit test, and continual integration.
After a number of years of operating this strategy, we had reached a point where the
regression test took up to ten days to run through (35,000 tests), and we had to create an
approach we called “laser-guided testing” to pinpoint the best set of tests to run each build.
We can now run a complete targeted regression in less than a day!
We saw many bene¬ts of our agile automated approach, but there are costs and pitfalls
that are worth considering ahead of time.

18.1 Introduction
My name is Jon Tilt; I have been involved in software development for twenty-
¬ve years and speci¬cally in software testing since 1996. I started testing in the
functional test team of MQSeries, a large middleware product. Since then I have
progressed through test management, test project lead, and test architect roles. I
currently work for IBM™s Software Group under the WebSphere brand and am the
Chief Test Architect across a number of Enterprise Service Bus (ESB) products
that generate in excess of one billion dollars of revenue each year. The products we
deliver are primarily enterprise middleware offerings (application servers, messaging
backbones, etc.) that many major Fortune 500 companies bet their businesses on.
This case study focuses on a project that I worked on that started about ¬ve years
ago. It involved the delivery of a new ESB component using an iterative development
approach; this was the ¬rst time our team had tried this method. Our team™s focus
was automation, automation, automation, from developer test through to system
test, with strong leadership support and team buy-in to achieve this.
This case study focuses on the challenges of getting the project started and how
we almost became victims of our own success. After several years of this strategy we

Agile Success Through Test Automation: An eXtreme Approach

had reached a point where the regression test took up to ten days to run through
our 35,000 tests. In short, we had reached a point where we had too much testing!
Fortunately, our story ends happily, as we “innovated” our way out of automation
overload. We created an approach we called “laser-guided testing” to pinpoint the
best set of tests to run against each build. We can now run a complete targeted
regression test in less than a day.

18.2 Overview of the Testing Challenge
We had been commissioned to develop a new component, written in Java, as part of
a much larger middleware product. The component approached two million lines of
code and had a delivery team of one hundred software engineers. The component is
delivered as part of a Java Enterprise Edition Application server.
Our primary challenge was to fully integrate testing into the psyche of the whole
delivery team. Our guiding principle was that “Everyone owns the quality of the
We needed to be able to continuously verify the product, ¬nding defects as early
in the process as we possibly could. By focusing on the quality of our drivers we
strove to make later phases of testing more effective and ef¬cient.
In order for this strategy to be effective we needed to front-load our testing as
much as possible.
We also had to ensure the software performed correctly on a number of different
platforms (Mainframe, Intel, UNIX, zOS, Windows, AIX, Solaris, HP-UX, Linux, IBM
i series).

18.3 De¬nition of an Agile Testing Process
The development process introduced time-boxed iterations and full automation to
our team for the ¬rst time. The key points include the following:

Time-boxed iterations
Each iteration had to deliver “testable entities;” content was agreed jointly
between developers and testers.
Continuous integration (at least twice-daily builds).
Test automation included
clear direction from leadership team (good communications about what to
test and when to test it);
dedicated automation infrastructure resource “ people and hardware; and
developer testing, incorporating automated unit test, run as part of the build
process; “stop the line culture” for unit test failures; rivalry between teams to
have best code coverage of unit tests; charts to demonstrate results shown in
department meetings; and function testing.
Focus on end-to-end test automation (not just the execution phase).
Agile Testing: How to Succeed in an Extreme Testing Environment

Over time, the number of tests increased to a point where a regression run took
ten days!
System test was largely manual for “consumability.”
Humans tend to understand variations better than machines.
User-like tests (documentation, GUI).
Laser-guided testing “ invented to reduce the burden of our regression suite
(laser guided testing is explained in detail in the next section).

The test automation was separated into two parts, which I will call developer
testing and functional testing.

Developer Testing
Developer testing was a big cultural change for our team. We put a huge amount of
emphasis on ¬nding defects as early in the process as possible, when they are the
cheapest to ¬x. Under the banner of developer testing, I include the build process
itself (compiling and merging the whole product twice a day), automated unit tests,
and, ¬nally, a build veri¬cation test (BVT) that was run against each driver to verify
basic functionality.
For the ¬rst time we actually measured the unit test code coverage using the
open-source tool EMMA [69]. This was a real revelation, as we now had evidence
of the rigour of each developer™s testing. At ¬rst we exposed a number of groups
who claimed they were thoroughly unit testing, but in reality had not added the
tests to the automation. With leadership support this was quickly recti¬ed. At each
end-of-iteration department meeting, we displayed the coverage ¬gures for each
subcomponent team. At ¬rst this was used as a stick, but very quickly it became a
carrot as teams openly competed with each other to achieve the highest ¬gures.

Functional Testing
When we built the test team, we made sure that from the start we had an experienced
automation specialist who was responsible for creating the infrastructure for running
all our tests. We de¬ned automation from the beginning to mean “end-to-end”
automation. That meant that from a single button push, the infrastructure had
to be capable of installing a new driver, con¬guring the environment as required,
running the appropriate tests, evaluating the outcomes, and storing the results in
a database. Too often I see teams interpreting automation to mean automate the
execution step. While this has some bene¬t, the real costs of testing tend to come
in the setup “ especially in a multiplatform, multimachine environment such as
our own.
Once we had the infrastructure in place it was up to the testers to write the
material. We instigated a policy that a test was not deemed complete until it had
run inside the infrastructure. This did lead to some heated debates with the project
team, who of course wanted to see test progress. However, in the long run it proved
to be the most effective way to ensure everything was correctly automated.
Agile Success Through Test Automation: An eXtreme Approach



Code Change

System under Test

Dark grey represents the recent code changes to the system under test (SUT),
Light grey represents the code covered by each test suite (S1-S6)

18.1 The principles of laser-guided testing.

With a team of up to twenty testers we managed to accumulate vast numbers
of tests over the course of several releases. While this mass of collateral was a real
bene¬t to the project, it was beginning to take on a life of its own as it got to the stage
where a complete regression run was taking in excess of ten days to complete. The
overhead of feeding the automation beast was now becoming a considerable burden
to the team.
This is when test specialist Ian Holden and test manager Dave Dalton devised the
concept of cumulative test analysis “ a way of identifying the most effective tests to
run, based on information about how they have run in the past, what system code
they cover, and what has changed in the last product build (see Ian and Dave™s paper
describing the process [70]).
Since this was going to be a radical new approach to the way we did our testing,
we needed to sell the concept to our leadership team. The ¬rst step was to rename
“cumulative test analysis” to “laser-guided testing” or just “targeted testing,” which
I felt conveyed better the exciting approach to what we wanted to achieve.
At its core, targeted testing is very simple. You need two pieces of information:
¬rst, a map of what product code each test covers, and second, you need to know what
code has changed in the product since you last tested it. If you have this information
then it is possible to select the most effective tests to run against a particular product
Figure 18.1 provides a simpli¬ed graphical overview of the principles of laser-
guided testing.
In this example we can use targeting to select and prioritize the suites that need
to run:

S1 and S2 must run.
S3 and S5 may add a small amount of value.
S4 and S6 need not be run as they add no value to this change.
Agile Testing: How to Succeed in an Extreme Testing Environment

Where defects are found

16% Developer



18.2 Breakdown of defects found by testing phase.

I have deliberately simpli¬ed how targeting works, and when I do this everyone
points out that there may be changes elsewhere in the product that have an impact
on a piece of code that hasn™t actually changed. This is of course true and has been
taken into account in the targeting algorithms; however, those details would take
another chapter at least and I won™t go into them here.

18.4 Results of the Agile Approach
Overall our approach to agile testing has been very successful. Figure 18.2 shows the
breakdown for who ¬nds defects during the product life cycle. (When these ¬gures
were gathered, the component had been out in the ¬eld for over a year.)
Nearly a third of all defects discovered are found by developer testing (compile,
unit test, and BVT), which is when they are cheapest to resolve. Adding in the defects
found by our functional test automation brings the percentage of defects found by
automation to well over 50%.
After our ¬rst product release, I wanted to work out how successful our automated
unit test strategy had been, so I did an analysis of the defects found by the test team
and compared them to the code coverage ¬gures of each of our subcomponent unit
tests. The results are summarized in Figure 18.3.
I believe this graph really shows the effectiveness of unit testing; the subcom-
ponents that had the lowest automated unit test coverage had the highest number
of defects found by the test team, and the subcomponents with the highest code
coverage had the lowest number of defects.1
The total automation of our functional testing has also proved very successful as
we now have an excellent regression capability, despite its initial high cost and the
fear at one stage that it was going to devour all our resources.

1 There are some anomalies in between the two extremes and these would need more detailed investigation.
There may also be some correlation between the complexity of the subcomponent and the number of
defects, but this is another area for future investigation.
Agile Success Through Test Automation: An eXtreme Approach

Test Defects vs Code Coverage %

30 100
25 80


CC %
15 50
10 30
0 0







































Test defects CC

18.3 Analysis of defects found by test team versus code coverage.

The introduction of laser-guided testing meant we were able to reduce our team
of regression testers (the team that was responsible for preparing the hardware,
kicking off tests, and chasing test failures) by four people and move them into front-
line testing. We were also able to reduce the cycle time of a complete regression
run from ten days to only one day. The important thing to understand here is that,
while we no longer run the entire regression suite, we believe what we do run is able
to assess the quality of the build to the same extent but in a dramatically reduced

18.5 Lessons Learned
Potential adopters of such approaches should not be overawed; this isn™t rocket
science. Here are some simple guiding principles that we identi¬ed:

The earlier we ¬nd a problem, the less it costs to ¬x it.
Successful agile quality is not a technology issue, but a cultural one:
Everyone in the team is responsible for the quality of our component.
De¬ne the automation vision early and stick with it.
Start your automation with something simple and build on it.
Remember that over the course of time your automation will grow; plan for this
and don™t let it consume you.
When starting out, be very clear on why you are embracing automation and early
testing. The cost factor is the primary driver and helps get buy-in across the
whole team.
Agile Testing: How to Succeed in an Extreme Testing Environment

We spent a lot of time reiterating to the delivery team that everyone was respon-
sible for the component quality and that we could achieve high quality through
There is always the temptation to architect yet another automation framework
at the start of the project. Don™t! Look around at the simple tools that already
exist (like JUnit) and build on them as required. Developers and testers are much
more likely to use something simple that is quick and easy to use.
Once you have some automated tests in place, start measuring their effectiveness.
There are many code coverage tools in the market, but start with something
simple like EMMA (for Java coverage) to give you a picture of what your tests
are doing. We started our measurements at a very crude level (just measuring
class coverage). This made it easy for us to achieve high scores early and psycho-
logically helped gain acceptance from the team. Later on we switched to more
¬ne-grained analysis (method and branch coverage) but only after the process
had become well established.
All our ¬gures were graphically displayed for all to see “ a great motivator!
We generated over 7,000 unit tests alone that ran as part of every build.
Do remember “ code coverage itself does not improve product quality, actions
based on its results can!
When your automation is running regularly and growing with each iteration,
remember to keep an eye on its size and how long it takes to run. While the
perception is always that lots of tests mean lots of good testing, beware of the
automation beast and how it will end up consuming all the time and resource you
We have had great success with our laser-guided targeting of tests, but it is also
worth considering regular refactoring of your test suites. This way you can purge
anything that is not adding value and help to keep your regression capability lean
and agile.
19 Talking, Saying, and Listening: Communication in
Agile Teams
Isabel Evans, Principal Consultant, Testing Solutions Group

This chapter describes techniques to improve the interactions between people during agile
The quality of communication is vital for agile projects; agile methods rely on close and
collaborative working, so the relationships between team members strongly affect success.
Agile methods emphasize people but they do include process; the reaction to and success
of the processes depends on the people. We can use special techniques to enhance the
communication between people, thus enabling our agile processes to be effective and ef¬cient.
This case study provides examples of problems with communication that I have observed
in a number of agile projects and some useful and practical techniques that can be used to
address and overcome these communications issues.

19.1 Introduction
My name is Isabel Evans. I™m a principal consultant at Testing Solutions Group Ltd
and I have more than twenty years™ experience in the IT industry, mainly in quality
management, testing, training, and documentation. During that time I have worked
as a practitioner and a consultant with clients in many sectors, including ¬nancial,
communications, transport, and software providers.
I™ve been a member of various working parties and groups to contribute to im-
provement in software quality and testing, and my book Achieving Software Quality
Through Teamwork was published by Artech in June 2004. I™m interested in people
and processes, how they interact, and how those interactions affect the success of
projects and businesses. Agile methods emphasize people but they include process;
the reaction to and success of the processes depend on the people.
In this chapter, I want to think about techniques we can use to enhance the
communication between people to enable our agile processes to be effective and
ef¬cient. This is not a speci¬c case study dealing with one customer or project but
is distilled from a number of such engagements I have worked on and from which I
have documented examples of issues with communication that affect agile testing.

Agile Testing: How to Succeed in an Extreme Testing Environment

19.2 Talkin™ a Lot . . . Communication Problems in Agile Testing
Good communication is essential for success in any project, whether that is written
or verbal communication. IT is about information, from the conception of the initial
project through to the closedown and decommissioning of the system.
The means and media of communication are different in traditional and agile
projects. In traditional projects, the key communication methods and media are
written or electronic documents and meetings. The communications can be slow,
bureaucratic, and hierarchical in nature, although that is not always the case. For
many traditional projects, hierarchical power structures supported by words on
paper de¬ne “the truth.” Unfortunately, much of the real truth about the project
may remain hidden in people™s heads; the hierarchy may inhibit communication,
and documents have gaps which are ¬lled by people™s assumptions.
One reason for adopting an agile approach to projects is to speed up successful
delivery into production, and this may be done by reducing hierarchy, reducing
documentation, and reducing meetings. Communication is intended to be direct,
honest, speedy, productive, and egalitarian. The agile project is intended to over-
come communication problems. In agile projects, a typical approach might include
reduced, less-formal documentation with speedy meetings and verbal decisions.
Key words for successful agile projects are team work, collaboration, communi-
cation, trust, and respect. But these qualities need to be worked for; just putting a
group of people together does not make them a team. So let™s start by looking at
some ways that communication may go wrong.

Some People Just Prefer a More Formal Style
It might be that people have misunderstood or don™t trust the agile methods; they
may not realize that the agile approaches are not just an ad hoc approach but require
the discipline of keeping one™s teammates informed. If a team or team member
is not used to working this way or if a manager is not sure how to manage an
agile team, then the discipline of information exchange may be missed. People who
prefer to communicate by email, written report, or formal presentation may be
uncomfortable with the direct interpersonal interactions required; they may be shy,
or lack con¬dence, or prefer to put their thoughts in writing in order to review them
more easily before showing them to others. Additionally, some people want to make
a statement of fact or a decision once only “ they want to issue their decision (email,
report) and that™s it “ job done! If someone feels forced to take part in agile processes
against their will or disagrees about how to organize the team, then they may opt
out from the collaborative process.

Some People Are Good at Talking but Not So Good at Listening
Those of us who generate ideas are often noisy and talkative, and we can dominate
meetings. But sometimes we forget to listen to our quieter colleagues. We are only
communicating in one direction, and our teammates cannot get us to listen. By
Talking, Saying, and Listening: Communication in Agile Teams

sheer will power we move the meeting to the conclusion we want. But supposing
it is the wrong conclusion? Or supposing there is more than one noisy person in
the room and we don™t agree with each other? Discussions may degenerate to an
inconclusive argument, and the rest of the team members mentally pack their bags
and leave. They no longer want to contribute.

Some People Just Don™t Get On
If team members do not get on with each other, and there are personality clashes,
it may become dif¬cult to collaborate. This can be exacerbated if anyone abuses
the collaborative approach “ if meetings allow the “loudest voice” always to win, if
there is bullying, or people fall into abusive behavior. This can happen if anyone
adopts parent“child relationships in the team, as described in Abe Wagner™s book
The Transactional Manager [71]. We think we are exchanging facts, but actually our
exchanges become loaded with emotions, and not always constructive ones. We end
up telling each other off, or capitulating when we are challenged incorrectly. Wagner
suggests that:

We are all six people rather than just one and that some of these six “inner people” are
effective in dealing with others, but some of the “inner people” are not so useful.
The six inner people or ego states Wagner identi¬es are as follows:

1. The natural child is an effective ego state that acts spontaneously, expresses feelings,
and has need for recognition, structure, and stimulation.
2. The adult is an effective ego state that is logical and reasonable; it deals in facts rather
than feelings.
3. The nurturing parent is an effective ego state that is ¬rm with others, but also
understanding, sensitive, and caring.
4. The critical parent is an ineffective ego state that uses body language, gesture, and
tone of voice to “tell others off” perhaps by sarcasm, pointing the ¬nger, or raised
5. The rebellious child is an ineffective ego state that gets angry and stays angry, is very
negative, does not listen, and may deliberately forget things or procrastinate.
6. The compliant child is an ineffective ego state that blames itself, uses a soft voice,
whines, is very careful and self-protective.
Communication between the effective ego states is generally useful . . . If we cross into
the ineffective ego states, we will argue, whine and blame without communicating or
changing anything; in fact we may make things worse [72].

People Use Information as Power
For some people, information is the source of their power. By hiding information,
they maintain their power base in the organization. The mistaken belief that they
are uniquely able to solve certain problems “ perhaps particular system knowledge “
means that collaboration will be a threat to them. They may express this by saying
Agile Testing: How to Succeed in an Extreme Testing Environment

that no one else has the knowledge to solve those problems. Or, they may cite loss
of con¬dentiality and security requirements as a reason to withhold information.
These may in some circumstances be valid arguments, but they may be red herrings.

Methods of Communication May Be Inappropriate
The number, style, length, and recordkeeping for meetings may not favor collabo-
ration. Recordkeeping may be inadequate or overdetailed; either extreme prevents
good information exchange.

Environment Is Not Conducive to Collaboration
The project may not be sited or equipped in a way that fosters good communications
and collaborative behavior. For example, if there is no team room or if the team is
on a split site, this will reduce face-to-face communication. In The Mythical Man
Month [73], Fred Brooks pointed out what a short distance away people need to
be before they stop communicating well. A colleague has just observed this on a
project; the developers and testers were moved to only 100 meters apart in the
same open plan of¬ce, but they had reverted to communicating only by email.
Attempts to run agile projects without co-located teams will require far more effort
to maintain communication. Organizations attempting to run projects across split
sites, multiple countries, and multiple time zones are setting themselves additional
challenges for communication and collaboration. Part of the environment is the
equipment available to the team; if there are problems with computer equipment,
communications technology, and information storage and retrieval, the project team
will not be able to communicate well.
None of these problems are unique to agile teams; the same problems apply to
traditional projects. It is just that in traditional projects it might be easier to hide
the consequences for longer.

19.3 Something to Say . . . Techniques to Free Communication Logjams
If the problems are the same, so are many of the solutions. The methods I am
going to suggest are tried and tested inside and outside IT projects. Because all
humans carry taken-for-granted assumptions about the world, it is easy to believe
that our communication preferences are the correct ones. We need to understand and
empathize with others; they may have different preferences. As the team interacts,
it will need to build understanding between the team members to help build shared
assumptions and hence trust [74].

Although many taken-for-granted assumptions are shared, there will be differences
between cultures, organisations and family/friendship groups. Just because you “always
do it that way” it does not mean it is the right way. One of our assumptions can be that
others will share our taken for granted assumptions. If we discover that other people have
different core beliefs, we are surprised and may dismiss their views as wrong “ it is hard
to see other people™s view.
Talking, Saying, and Listening: Communication in Agile Teams

We may also assume that other people communicate in the same way as we do, that
they share our sense of humour, use body language in the same way, and share our
preference for written, pictorial or verbal messages. I recently had a discussion with a
colleague because we could not understand each other. It turns out that we used two key
phrases in opposite ways, both equally correct but meaning the opposite to each other:

• I had used “I think” to mean “I am not sure” but my colleague had used “I think” to mean
“I am certain.”
• My colleague had used “I feel” to mean “I am not sure” but I had used “I feel” to mean
“I am certain.”

These communication differences are increased by cultural differences between organ-
isations, and between colleagues from different types of organisation, and from different
countries [72].

The following sections review a couple of techniques that I have found valuable
in freeing communications logjams, speci¬cally De Bono™s Six Hats in a Meeting and
cause and solutions analysis with Ishikawa Fishbones.

Technique 1: De Bono™s Six Hats in a Meeting
Sometimes collaboration can be held up by con¬‚ict “ personality clashes, insuperable
arguments, and endless, unresolved con¬‚ict. We ¬nd ourselves always contributing
in the same way “ pessimistically, optimistically, coldly, or emotionally! And we
notice that other people are equally predictable. As a consequence, we ¬nd that the
meeting dissolves into acrimonious chaos, with people no longer on speaking terms.
Can we do anything about this?
Edward de Bono [75] decided that we could, and the Six Thinking Hats was the
result. I ¬rst came across it as an idea for improving software projects from Jens
Pas™ EuroSTAR presentation and workshop [76] as a means of introducing emo-
tional intelligence to rationally biased software projects “ in other words, allowing
fact-based people an opportunity to also express emotional views. The “six hats”
encourages people to take different roles at their meeting. It allows them to deal
with one problem at a time and to focus on resolving problems. You can refer to the
hats in two ways: You can de¬ne the thinking process that™s required in a given sit-
uation, or you can comment on how someone is communicating without appearing
critical. So you might be struggling with an apparently insurmountable problem,
and call on the team to put on their green hats to generate some ideas. Or you can
ask someone who™s getting negative to take their black hat off for a moment “ this is
a more neutral request!
By using the hats we set rules for behavior. Everyone wears the same colour hat
at the same time, and the hats are not associated with particular people. This allows
meeting members to move outside their perceived stereotypes and allows time for
different, sometimes dif¬cult types of communication. De Bono discussed the order
and way the hats are used; it depends on the meeting and the problem, as well as the
Agile Testing: How to Succeed in an Extreme Testing Environment

team™s experience in using the techniques. Suppose we have a meeting to discuss a
design for an interface.
We might have a meeting agenda like this:

Blue hat to set the scene “ why are we here and what do we want to achieve?
White hat “ what facts do we have that everyone can agree on?
Red hat “ do we like or dislike the design “ what is our gut feel about it?
Yellow hat “ what are the good things about the design?
Black hat “ what disadvantages can we see?
Green hat “ can we identify new ideas to help us overcome the black hat points?
Red hat “ how do we feel now?
Blue hat to close the meeting “ what are the next steps?

The advantages de Bono identi¬es [75] are

powerful, focused working, which is time-saving because the meeting is not ¬lled
by confrontational arguments;
removing of ego by removing confrontation, allowing the meeting to deal with
one thing at a time.

All the ideas from all the people at the meeting are treated as parallel rather than
confrontational. Once a complete picture has been arrived at using the Six Hats, then
it is easier for people to agree to a solution or make a decision. We can use the hats
to change focus: “It looks like with the yellow hat thinking we™ve identi¬ed some real
advantages to the approach we™re discussing. Let™s just try some black hat thinking “
what are the disadvantages?” You can ¬nd more information about de Bono™s work
on thinking, including the six thinking hats, on his Web site [77].

Technique 2: Cause and Solution Analysis with Ishikawa Fishbones
Cause“effect analysis helps us look for the root causes of problems, using cause-
and-effect diagrams, sometimes called ¬shbone diagrams because of their shape or
Ishikawa diagrams after the person who ¬rst developed them, but now widely used
in identifying quality and other problems [78, 79].
Put simply, this is a structured form of gathering and sharing information, either
in a brainstorm meeting or as work is going on, day to day. It is used in factories,
with a blank ¬shbone on the wall in easy reach by a production line, so that anyone
can add an idea about the cause of a problem or a potential solution. Periodically,
the group working on the production line will discuss the information and ideas
gathered in the diagram and decide on problem solutions.
This open approach with simply documented ideas to which anyone can con-
tribute lends itself to lean manufacturing and to agile project teams. In both cases,
the teams are empowered to analyze and change their own processes to facilitate fast
The diagram is simply a means of helping us think about and classify the causes
of problems and their underlying root causes. The ¬rst step is to draw a ¬shbone
diagram with the effect (which is the problem or symptom noticed) on the head of
Talking, Saying, and Listening: Communication in Agile Teams

Fishbone diagram before start of discussion about customer complaints
Environment Methods
We forgot to tell everyone
that the bug fix was in

Fix was made to earlier

A defect has

Plant Equipment Materials

19.1 Ishikawa ¬shbone diagram.

the ¬sh. Then, label the ribs of the ¬shbone to show typical areas of root cause. You
can choose your labels. Typical ones are “The 4 Ms “ Manpower, Machines, Materials,
Methods” [78] or the “PEMPEM” labels, which give People, Environment, Methods,
Plant, Equipment, Materials [79], or you could make up your own labels.
In a traditional project, a brainstorming meeting would be called to generate
causes for the effect, grouping the ideas on the ¬shbone to show how they relate
to each other. In an agile team, the ¬shbones could be started on a whiteboard or
¬‚ipchart in a brainstorming session during team meetings or between team meetings
as people have problems to solve; ideas can be allowed to incubate naturally. Even for
a traditional project, we would want to put the diagram on a public board so other
people can see it, comment on it, and add to it, and so that the people can suggest
other causes [79]. The diagram in Figure 19.1 shows the very start of a discussion of
a problem: the ¬rst two ideas have been added. Notice that “ because we are looking
for a root cause “ we are not just adding ideas at the major bones. We backtrack,
adding one or more causes until we reach a root cause; a defect has reappeared
because a ¬x was made to an earlier version, because we forgot to tell everyone that
the bug ¬x was in, because we did not realize that other people might change this
code module . . . and so on. We add ideas under each of the headings. Although a
quick look tells us this is highly likely to be a con¬guration management problem,
the breakdown helps us see whether we need to look at an improved process, tool, or
education of the team as they move from a traditional approach (only I can change
this code) to a collaborative approach (we work on this problem together).
We then “reverse” the ¬shbone, to go from a problem to a solution. To do this we
draw the diagram in reverse, write our proposed solution in the box, and then under
our ¬shbone headings discuss whether the proposed solution will work:

What actions do we require under the 4 Ms or PEMPEM or our own titles to make
sure the solution works? (Note: whichever titles you use, make sure you have
Agile Testing: How to Succeed in an Extreme Testing Environment

a place to look at time, money, and staf¬ng, which will all constrain the solu-
What advantages or positive affects will the solution have?
What disadvantages or negatives can we identify?

You can use different-color pens for the advantages and disadvantages, so you
could match pen colors to the de Bono Six Hats R colors, using yellow for optimistic
and black for pessimistic views [72].

19.4 Something Worth Listening to . . . Understanding Each Other™s Viewpoints
There are several methods of understanding viewpoints; these include psychometric
methods such as the Belbin Team Roles [80, 81, 82] and the Myers Briggs Type
Indicator (MBTI [83, 84, 85]). These are well explained elsewhere, so I shall only
summarize them here.
Belbin discussed the way people behave when working in teams and suggested
that a balance between the roles supports success.
For example, plants have new ideas while completer-¬nishers want to ¬nish to
¬ne detail. If you have too many plants in the team you™ll never ¬nish anything, but
if you only had completer-¬nishers, you might not have so many new ideas. Each
person in the team takes an assessment, and this provides a picture of the strengths
and weaknesses of the team as a whole [72]. Most people will score to show a mix of
the team roles.
In contrast, the MBTI looks at people™s personalities and suggests that commu-
nication styles and preferences follow certain patterns. A person may be introvert
or extrovert, for example, and that will alter the preferred amount of interaction.
Another factor is the degree of analytical or empathetic approach that someone
prefers. The four pairs are as follows:

E and I “ Extroversion (prefers social interaction) versus Introversion (quiet and
S and N “ Sensing (experience and practicality) versus iNtuition (novelty and
T and F “ Thinking (analytic or critical approach) versus Feeling (sympathizing
and appreciative approach);
J and P “ Judgement (¬rmness and control) versus Perception (¬‚exibility and

With each person being assigned one value from each pair, this gives Myers Briggs
sixteen distinct personality types. Beware of how you interpret these:

The communication preferences of these groups may be radically different. However, if
you are going to look at MBTI, beware of stereotyping! It is worth noting that just because
someone is in a particular job, it does not mean they are a particular type, and just
Talking, Saying, and Listening: Communication in Agile Teams

because someone appears to be a particular type does not mean they will be a success in
an apparently related job. Remember, if 10% of computer staff are INTJ, then 90% are
not INTJ. Similarly, being a woman does not mean you cannot be an ESTJ. As with all of
these types of test, make sure it is administered by someone who is suitably trained and
can interpret the results properly [72].

Notice how use of the Six Hats and the Ishikawa ¬shbones would encourage people
out of their roles as they allow everyone to contribute ideas, facts, and commentary
in a controlled way that might be more acceptable to some personality types; an
introverted team member could add his or her ideas quietly to the ¬shbone rather
than having to participate in a direct discussion. The roles and latest thinking
are described on the Web sites in the references. There are other useful models
such as looking at attitudes to innovation [86] and learning styles [87] which are
summarized in [72].

19.5 Starting the Conversation . . . and Keeping It Going
In addition to thinking about personality differences we should also consider orga-
nizational viewpoints.

Technique: Weaver Triangle
All the communication points from traditional processes “ reviews, entry and exit
criteria, coverage measures “ still have something to offer in terms of clarifying
that everyone in the agile team has the same goals for the project. To help us
with this, an “on one page” method of capturing goals for a project is the Weaver
Triangle. This was originally developed by Jayne Weaver for use with not-for-pro¬t
organizations, and we then adapted it for use in IT and business projects [72].
Using the triangle, the group identi¬es and agrees the aim of the project (why it is
being done) and associated indicators of success, then the objectives of the project
(what is to be done) and associated targets. This helps identify where stakeholders
have different aims for the project. The form is used to encourage teams to focus
on an overall aim or goal, and to show pictorially how the aims and objectives ¬t
Some ground rules of this approach are as follows (see Figure 19.2):

The aim should answer the questions “why are we doing this?” and “what differ-
ence will this make?”
The speci¬c aims should break down the overall aim into a small number of
detailed aims.
Each speci¬c aim also answers a “why?” and “what difference?” question.
In order for the aims to be achieved, something needs to be done, so each speci¬c
aim must be associated with at least one objective that answers the question
“what do we need to do in order to meet the aim?”
Agile Testing: How to Succeed in an Extreme Testing Environment

19.2 Typical Weaver Triangle.

Objectives, which may be projects within a program or parts of a project, depend-
ing on their size.
Each objective must be focused on achieving at least one of the aims, otherwise
there is no point doing it.
Aims are measured by indicators that measure whether we are making the dif-
ference we intended.
Objectives are measured by delivery targets such as savings, number of people
affected, and delivery dates.
Indicator and target measures should be linked to measures used generally in
the organization; for example, you could show how these measures link to the
organization™s balanced score card.
Consensus is required between the stakeholders. This is not done by the managers
and told to everyone else; it needs contributions and discussion from the whole

Technique: Think Group Viewpoints
In Reference 72, I have discussed that there are ¬ve viewpoints in any IT project:
the Customer, the Manager, the Builder (including all the analysis, design, and
development groups), the Measurers (including testers and reviewers), and the Sup-
porters (including IT infrastructure). Each of these groups holds particular views
about quality, has speci¬c information for the other groups, and requires particular
information from the other groups (see Table 19.1).
Table 19.1. The ¬ve IT project viewpoints

Acceptance criteria When involved Additionally
Group Quality Risks Constraints traditionally involve them in
and plan are . . .

Customer User (¬t for purpose) Impact on organization Business, service Realistic, timebound Preproject All project stages
level, time, cost At go live
Post go live

Manager Value (can we afford it?) Impact on other projects Cost, time, skills, Realistic, timebound Preproject
resources Project
At go live

Builder Manufacturing (defect Likelihood“technical Technical skills, Achievable, realistic Project Preproject
and process) infrastructure Go live Postproject

Measurer Product (ISO 25000 Likelihood“previous Technical skills, Speci¬c, measurable Project (sometimes late) Preproject
attributes) failures, predictions infrastructure Go live All project stages

Technical skills, Achievable, realistic Go live Preproject
Impact on existing systems
Supporter User (¬t for purpose)
infrastructure Post go live All project stages
and infrastructure
Product (ISO 25000
Future growth
Agile Testing: How to Succeed in an Extreme Testing Environment

19.6 Conclusion
Only by considering the communication needs of all the people involved in the
successful delivery of software can the team succeed in delivering what its customer
needs to meet the correct quality attributes.
For the agile team to succeed, equipment, infrastructure, approaches, and tools
used in the project must support disciplined communication.
The team members need time and opportunity to understand each other™s com-
munication styles and to build trust. An understanding of how others think and
feel will help team members interact more effectively, and use of techniques such
as De Bono™s Six Hats, Ishikawa ¬shbones, group viewpoints, and Weaver Triangles
provides opportunities for structured interaction to improve communication. An
understanding of the different viewpoints in the organization allows the team to
draw the right representation into the team.
Agile teamwork is not necessarily a natural way of working for all of us, but we
can all improve our communication by understanding ourselves, empathizing with
others, and using techniques to structure our communication processes.

Chapter Acknowledgements
Aspects of this chapter are based on conversations with Elisabeth Hendrickson, and I
also bene¬ted from her excellent workshop at the London SIGiST (British Computer
Society Specialist Group in Software Testing) event in 2007, where I was able to
observe and take part in the group interactions. Figures and tables are based on [72].
Comments on communication and on agile projects from Graham Thomas and others
informed my thinking.
20 Very-Small-Scale Agile Development and Testing of a Wiki
Dass Chana

This case study reviews an agile approach to the development, testing, and delivery of a Wiki
in very short timescales, with chronically low budget, and with very few resources. There were
a number of potential risk issues with the project, such as dif¬culties with co-location of the
customer and the project manager, but which were successfully mitigated by intelligent use
of collaborative technologies such as instant messaging and teleconferencing.
Overall, the project was very successful, delivering precisely the system the customer
needed, and with good quality. The only signi¬cant issue that could be reported was that the
original delivery timescales of four weeks overran by one week, although this was actually
due to deliberate enhancements requested by the customer.
I also make some suggestions for improvements to the agile development and test process
that I would use on future projects.

20.1 Introduction
My name is Dass Chana and I am a computer science student. This case study
describes a project I undertook as an industrial placement student working for a very
large IT company. I was assigned as a resource to a large distributed team who were
responsible for providing technical support to the company™s business partners, and
tasked with setting up and delivering a Wiki (for internal use within the team) under
very challenging timescales, with extremely low budget, and with very few resources.
I was given the goal of getting the Wiki up and running in just four weeks (although
in the end I actually took ¬ve weeks due to enhancements to the system that were
requested as the “customer” saw what the Wiki could do for them).

20.2 Overview of the Testing Challenge
At the risk of starting a circular argument, Wikipedia [36] de¬nes a Wiki as

a collection of web pages designed to enable anyone who accesses it to contribute or modify
content, using a simpli¬ed mark-up language. Wikis are often used to create collaborative

Agile Testing: How to Succeed in an Extreme Testing Environment

websites and to power community websites. The collaborative encyclopaedia, Wikipedia,
is itself one of the best-known wikis. Wikis are used in business to provide intranets and
Knowledge Management systems.

The manager of the team I had been assigned to, as a work experience student, gave
me the goal of implementing a Wiki to provide the team with a resource for posting
team news and success stories. In effect my teammates were also the customers/users
of the Wiki.
The inevitable requirements creep1 would eventually lead to the purpose of the
Wiki being extended to include information on best practices for the team, de¬nition
of team roles and responsibilities, team structure, and reporting/liaison lines (in
effect, a process/framework de¬nition for the team “ yet another recursive aspect of
the project and this case study write-up)!
Given the nature of the system I was tasked with developing, the chaotic nature
of the requirements (often driven by two different managers simultaneously, plus
the ad hoc requests/comments from the team; i.e., the users), the lack of avail-
able resources (i.e., low/no budget, plus limited access to other human resources),
and the need to employ a formal method to develop the Wiki, I determined that
adopting an agile approach would be the best solution to developing and testing the
Also, using an agile approach to developing and testing the Wiki seemed a partic-
ularly appropriate solution, as the origin of the word Wiki comes from the Hawaiian
wiki wiki, which is translated as “fast.”

20.3 De¬nition of the Agile Development and Testing Process
My agile process incorporated the following characteristics:

RUPesque “ Having been exposed to the Rational Uni¬ed Process (RUP [7]) at
college, I employed a fundamentally RUP-like approach to the iterations (that I
termed the RUPesque method). I also included elements of the V-model [4] to
enable me to convince my manager that I was doing some formal planning of the
project. (It seems managers like to be reassured about these things.)
A small team (not co-located; see Section 20.5 for details of how I overcame the
issues of lack of co-location) comprised
the customer/user (basically the team I had been assigned to during my
industrial placement),
the project manager (my boss),

1 In my young and innocent state, I did begin to wonder if the “Requirements Creep” might actually be
a role held by one of the team members “ perhaps the person responsible for coming up with new and
crazy requirements.;-)
Very-Small-Scale Agile Development and Testing of a Wiki

the developer and tester (me), and
ad hoc resources (a member of the team as and when needed for pair devel-
opment and test “ see later discussion).
Short iterations (matched to a week-long cycle to coincide with and to piggyback
on an existing weekly team telephone conference call on Friday afternoon to allow
the users [i.e., the team members] to review the previous week™s development
and provide comments for the next week™s work).
Agile requirements “ capture and maintenance (documented using an agile
requirements tool [61]).
Rapid prototyping “ I relied heavily on a prototyping approach in which I would
produce simple mock-ups (essentially just demonstrating the navigation and the
basic features requested by the users) to show to the users in order to obtain their
feedback on the suitability of the proposed system (formally during the Friday
conference call, but also informally [but then documented in the requirement
tool] via the SameTime instant messaging system [a facility available under Lotus
Pair development “ To make particularly fast progress, I organized a number
of intensive sessions where I worked with an experienced member of the team
at the same workstation, bashing out code, and producing very good results
(see Section 20.5). Pair development was particularly valuable where I had very
challenging timescales and proved to be an excellent technique for very intensive
development and testing and rapid delivery of good-quality software.
Pair testing “ Again, working with my colleague, we identi¬ed suitable test sce-
narios, test data (including boundary and partition values), and employed state-
transition techniques to design tests to verify correct navigation [4]. I have broken
this out separately from pair development, because I was so impressed by what
turned out to be a particularly effective means of reducing defects.
Manual tester packs “ I don™t subscribe to the popular notion that all students are
lazy, fun-loving, hedonistic parasites (well, not completely). To avoid additional
wasted testing effort, and to ensure I wasn™t stuck in the of¬ce after 5 p.m.
or (God forbid) testing on the weekend, I made sure that testing was as slick
and ef¬cient as possible. My main solution was to ensure I got maximum reuse
from the tests that I (and my colleague, during pair testing) created. To this
I created a simple test script template in the IBM Symphony word processor
package and adopted a consistent script naming convention (a unique identi-
¬er formed from the iteration number, the Wiki page [and subpage], and an
incremented number “ e.g., 02-TeamOrgChart-01).
To promote test reuse, I ensured that all test scripts that were developed were
¬led (both electronically as Symphony ¬les and as paper-based copies in my
test reuse folder, which was a ¬le box with separate tabs into which the tests
for separate iterations were stored).
Agile Testing: How to Succeed in an Extreme Testing Environment

Regression testing was achieved by photocopying those tests that were
required from earlier iterations and including them in the testing pack for
the current iteration.
I employed test refactoring to ensure the testing load did not grow to unman-
ageable volumes; the earlier tests were reviewed and omitted if it was felt that
that aspect of the system being tested was now stable (in practice, that there
had not been a defect observed since the last but one iteration).
Acceptance testing did involve creating some new tests (by reviewing the
requirements and identifying areas that needed additional test scenarios) but
also involved the reuse of a random selection of manual test scripts from the
previous iteration.
In retrospect, in order to save even more of my valuable time and to reduce
effort, I would have used automated testing tools if we had had budget and/or
access to them. (Ironically, I discovered a day after the end of the project,
we could have obtained a thirty-day temporary copy of Rational Functional
Tester [62], which would have been very useful for recording and playing back
web and Java tests overnight and/or over the weekend!)
Continuous integration and code refactoring “ Although this may sound like
a virtue, this was in fact a necessity. As I worked across the weekly iteration, I
incorporated changes as and when they were received, and as and when I was
able to make them, republishing them in the live system. In fact, as I describe
later, this is one of the few areas I feel could be improved (again, perhaps by tool
support), allowing me to rerun the test suite overnight following changes, or
even over the weekend.

20.4 Results of the Agile Approach
The adoption and use of an agile approach to developing the Wiki proved to be very
successful. In terms of the general success criteria for the project, I am able to
provide the following information.

Budget “ Although there was no explicit budget to meet (except for the implicit
time I spent performing the work and that of my colleague, who provided ad
hoc pair development and testing support across the later weeks of the project),
I needed no additional effort or resources (aka money) to complete the project.
Since I had planned to complete the work in four weeks, but actually took ¬ve,
you might argue that I had overrun by 25%. However, in reality the additional
week was an extension that I negotiated with the customer and project manager
to allow the incorporation of further features that the customer requested after
seeing what the system could do for them as they observed the development of
the prototypes and full functionality of the evolving Wiki.
Timescales “ As I described earlier, the initial four-week allocation for the project
grew to ¬ve weeks. This is a dif¬cult issue to deal with in this case study. For
Very-Small-Scale Agile Development and Testing of a Wiki

while I was delivering customer satisfaction by implementing additional features
that they requested (and which I formally agreed with both the customer and my
project manager), we were not really operating in a commercial environment.
My time was in effect free to my boss (aka the project manager), the software I
was developing was not a commercial bespoke system (in fact, it was only really
intended for internal use by our team), and bottom line, was not commercially
critical to the success of our company.
What I can say is that, if this had been a full-blown commercial project to deliver
a business-critical system, with contractually binding budgets and timescales, I
would have looked to agree the requested changes from the customer with my
project manager and, if appropriate, would have proposed an additional charge
to the customer to cover the additional work.
Quality “ Overall, the quality of the delivered system has been good. So far there
have been no serious defects identi¬ed, and ¬ve minor issues reported to me
following ¬nal delivery, requiring trivial reworking and retesting of the system
(about two hours in total, or about 1.85% of the total project time).
I can report that, in the initial iterations of the project (when there was no
pair programming and test resource available), we identi¬ed about half a serious
defect per week and some six minor issues. During the last two weeks, when I
was able to implement pair development and test, this fell to an average of zero
serious defects per week and three minor issues. Although the sample size is a
little small for formal statistics, my feeling is that we “doubled the quality” by
employing pair development and test “ which is also my estimate for overall
productivity (i.e., more than twice the productivity compared with single-person
Customer satisfaction “ The delivered Wiki has been a great success, with not only
my own team using the system, but with our project manager also providing links
to the Wiki to the managers of other groups we work with within the organization.
(There is de¬nitely an element of showing off here, although my manager justi¬es
sharing access to the other groups as “showcasing our capabilities.”) In formal
The system has been acceptance tested against all of the requirements in
the requirements management tool and we can demonstrate 100% coverage
against the requirements.
En route to completing the project, the use of rapid prototyping meant that
we kept closely to the needs of the users (who could provide rapid feedback
on the direction we were taking).
The formal weekly testing of the system (plus regression testing) meant that
we started testing early and continued throughout the project. I am proud
to say that this approach meant that, once a defect was identi¬ed, it was
¬xed quickly and with thorough regression testing did not reappear again in
subsequent iterations.
Agile Testing: How to Succeed in an Extreme Testing Environment

20.5 Lessons Learned
What Worked Well
With the project complete, the Wiki delivered and working, and having had the time
to review what I did (is this what they call an agile retrospective?), I would like to
report the following positive aspects of the development process:

Pair programming and pair testing worked very well. I estimate that working
alongside another member of the team in developing and testing the Wiki was
more than twice as effective as working on my own (in terms of timescales and
lack of reworking). The obvious and tangible bene¬ts included:
Being able to keep each other focused on the task at hand (made it harder for
me to keep checking out my auctions on eBay or my mates on Facebook and
Second Life).
Filling in for each other™s knowledge gaps “ There was often something I
couldn™t do that my esteemed colleague did know about, and vice versa “ this
meant together we had a more complete coverage of the knowledge of the
development environment capabilities.
Providing a testing dummy “ I have often experienced the situation where you
have observed a bug, spend ages trying to understand what is going wrong,
but don™t seem to be able to make any progress. Then a mate comes past who
you grab and ask for their help, and even though they don™t really say a word,
the act of explaining what is going on actually gives you the eureka moment
where you spot the problem. At this point they usually stand up and wander
off, feeling smug about having done so much to help you! This experience
is just part of the pair development and test paradigm, and happened to us
several times each session.
Solving that developer testing thing “ I ¬nd it hard to test my own work. I seem
to focus on showing that the code works the way I expect it to work, rather than
focussing on showing that the code doesn™t work. This is a subtle difference
between a developer™s approach to testing compared with a tester™s approach,
but in practice it seems to make a big difference (more on this later). With
a pair testing approach, I found that together we would come up with more
test scenarios and more thorough scenarios (maybe ¬nding more error con-
ditions to test against, for example). The other person also keeps your testing
“honest” “ no cutting corners or making rash assumptions about how good
your code is. Where I had tested on my own, there were always lots of issues
reported by the team when they made their weekly review of the system. On
those weeks where I had been able to work with a colleague on pair program-
ming and testing, we calculated that there were only about 50% of the bugs
The weekly iteration cycle “ While I don™t believe this will work for large
projects where they have large tasks to complete, I found this to be a highly
effective technique to adopt on my project. Because we were running to
Very-Small-Scale Agile Development and Testing of a Wiki

a weekly cycle, everyone seemed to get into a rhythm in terms of their
responsibilities and tasks:
I knew I had to review the requirements (existing and new requirements)
every Monday morning and ensure the requirements tool was kept up-to-
The team knew that they would get the latest version of the system to review
each Thursday afternoon, and also that they had to have their comments
ready for the teleconference on the Friday afternoon.
Similarly, I had access to the project manager (my boss) on the Friday
conference call, as well as the customers/users (the rest of our team), and
could take direction from them on priorities and goals for the next week,
as well as, where appropriate, to make requests for additional resources
(such as a pair developer and tester).
Monday afternoon to Wednesday was development and testing time.
That left Thursday morning to do some formal testing to ensure I had met
the requirements for the week, and to regression-test the Wiki to make
sure I hadn™t introduced any deleterious changes; I had to be thorough and
I had to make good progress to hit the Thursday lunchtime “code drop”
deadline to deliver to the users.
Lack of co-location with users “ The fact was, that although we weren™t physically
co-located, the company we worked for provided extensive team working support
facilities. I could always ping one of the team using our company instant messag-
ing system and could always publish the latest version of the Wiki for anyone to
take a look at some aspect of the functionality I was working on. I don™t believe
I could have developed the system as quickly, with as high a level of quality, or
with as close a match to the users™ needs working on my own in isolation and
only having contact with the users once a week.
Lack of co-location with the project manager “ Again, although not physically
co-located, I was able to quickly and easily ping my project manager using instant
messaging. My feeling is that the project timescales, quality, and match to the
users™ requirements did not suffer through lack of face-to-face contact with my
project manager.
Tool Support “ I found that having access to a requirements tool was very useful:
It provided me with a simple and effective solution to documenting require-
I could use the tool to publish the requirements to my project manager and
the users so that they could review them and provide me with their comments.
Keeping the requirements up-to-date was straightforward, and I had a built-in
trace of how the requirement had changed and who had requested the change.
This proved to be invaluable several times, as guilty parties in the team denied
having suggested a particular change “ the tool never lies.;-)
I would have liked to have had access to testing tools as well “ such as the
sort that can automatically scan a web application to identify problems (such
Agile Testing: How to Succeed in an Extreme Testing Environment

as missing links, widows and orphans,2 security issues). The reality was that
this project was too small and had no funds to allow us to use such tools.
However, having seen such tools demonstrated, I would certainly investigate
their use on other projects that I may get involved in. (I am sel¬sh, and
anything that saved me time and effort on testing would be something I
would be very interested in using!)
And last but de¬nitely not least “ rapid prototyping. I found this to be a
really useful means of increasing my ef¬ciency and keeping to my timescales.
Rather than just assume I knew best and spend ages developing a complete
solution that it turned out nobody wanted, very quickly producing a simple,
navigation-only model and running it past the team meant that if I had gotten
it wrong, I had only spent a couple of hours of time on the task (and had gained
the big bonus that I now understood better what was actually needed). So,
prototypes are a quick, simple, and easy way to check you are developing the
right thing and if not, to not waste too much time and effort, with the bonus
of getting a clearer insight into what the users actually want.

What Would I Change in the Future?
Although the project completed successfully (admittedly, with a deliberate ¬fth
iteration overrun), with the bene¬t of 20“20 hindsight, the one area I might change,
if I was still working in the team and there was a requirement to run a similar project,
would be in the area of continuous integration and refactoring.
What I would have liked to have been able to do was to undo changes I had made
in a formal, safe (i.e., allowing me to restore a change), and repeatable manner. For
example, if a particular request for a change had resulted in a catastrophic problem
in the system, it would have been nice to have easily restored the system to its
prechange working state. Also, it would have been nice to have been able to retain
those successful changes that might have been made in parallel with the problem
change while undoing that change.
Given the relatively low size and complexity of this project, this was in practice
a relatively small problem (although I did spend several frustrated sessions in the
evening, and on one occasion across a weekend, ¬xing such issues). However, I can
see where this would be an extremely serious issue with larger and more complex
applications and larger teams.
This seems to me to be largely a con¬guration management problem, and my
solution to this issue on future projects would be to employ tool support, using
either a shareware tool off the Internet for the sort of small project described in this
case study or a more capable and robust proprietary product (such as [88]) for larger,
industrial-strength projects.

2 Widows and orphans refer to web pages that no longer have any links pointing to them, or links where
the original page they pointed to is no longer present.
Very-Small-Scale Agile Development and Testing of a Wiki

Perhaps the most innovative thing I might try in the future, if I was involved
in a similar project, would be to use some of the new hi-tech means of online
collaboration, such as team meetings in Second Life [64]. For example, I have heard
of Second Life design review meetings where the team could actually walk through
a 3D representation of the design diagrams and provide instant online feedback on
the information displayed. While Second Life might not be a perfect solution to the
problem of not being able to get together for face-to-face meetings, I really believe it
is an option well worth exploring (assuming my older colleagues actually have
accounts on Second Life)!
Finally, my only other wish “ and again while this was not really an issue on this
small project (I can see bene¬ts for larger and more complex projects) “ would be to
consider automated software testing. In the case of this project, a web testing tool (to
check navigation, usability, and security) might have been very useful and, for larger
projects, some form of record and playback tool (such as [62]) to regression-test
later builds and releases could save huge amounts of valuable tester time and effort
(avoiding the unwelcome prospect of spending too much time manually testing the
software in the evenings and at weekends, when I could have been out partying
21 Agile Special Tactics: SOA Projects
Greg Hodgkinson, Software Development Process Evangelist, Proli¬cs

Service-Oriented Architecture (SOA) projects [89] require speci¬c qualities in the solution that
they produce around ¬‚exibility and extensibility. Most agile methods don™t focus on providing
method support to address these needs, or at least not to the degree required for an SOA
project. This case study will describe my process con¬guration that borrows from the best of
agile techniques and combines these with SOA development techniques to support projects
that produce SOA solutions. And most importantly, this has been tried and tested on our SOA
projects for many years with resounding success.

21.1 Introduction
My name is Greg Hodgkinson and I work for Proli¬cs, where one of my roles has been


. 5
( 10)