<<

. 8
( 10)



>>

the status of the project (such as the lava lamp solution covered later in this
section and in Chapter 17), ¬xing defects can become a matter of both urgency
and pride, with the team working hard to achieve a “green lava lamp state”
again.
My Agile Process
227


Collective code and test ownership “ Collective code ownership is another best
practice that a number of the agile methods promote (such as XP). From a
software quality perspective, a number of the case studies describe the bene¬ts
of the analogous practice of collective test ownership and its role in encouraging
test refactoring (that is, all members of the team have the opportunity to modify
contents of the test suite for the purposes of improving the quality of the tests
and/or their ef¬ciency). Where this practice is adopted, consider the role of change
and con¬guration management tools to trace the changes and to allow changes
to be undone where necessary.
Agile exploratory testing “ This is a valuable addition to other testing practices
being used on a project. An experienced test practitioner will frequently ¬nd
defects that a developer or testing tool cannot, using their knowledge of the sort
of errors developers often make (such as incorrectly coding logical boundary
conditions), knowledge of how users often abuse systems (such as performing
unexpected navigation or entering inappropriate values), and that mystical tester
ability termed error-guessing [4].

Agile Process and Project Management
Co-location of project stakeholders “ This fundamental principle of agile devel-
opment and testing is the ideal solution for organizing an agile project team. Co-
location of the project stakeholders supports the practices of early involvement of
test resources, as well as pair testing (see Section 25.3) and early test case design.
Although an essential agile best practice, this approach becomes increasingly
dif¬cult to employ where team sizes become very large, or impossible where
it is necessary for some of the team to be at a different site or even offshore.
Advice and guidance on what to do under these circumstances is provided in Sec-
tions 25.5 and 25.6, respectively.
Agile estimation “ Estimation is a key aspect of agile projects, and methods for
estimating project and iteration velocity are well understood and used. Although
most projects employ estimation successfully, a number of the case studies
reported issues with agile estimation and stressed the importance of employ-
ing previous experience; the most accurate estimates for any particular project
are made with the assistance of metrics collected previously on that project.
For example, metrics collection and analysis can help a project leader identify
resources who regularly overestimate or underestimate the effort involved in a
particular task.
On larger projects, with longer timescales and/or the additional challenges
of not having a co-located team, there may be value in employing automated
distributed support for estimation (such as [63]).
Progress measurement “ Typically calculated agile metrics such as velocity are a
valuable source of information for stakeholders to be able to understand project
progress. Many of the case studies also report the value of collecting and mea-
suring a number of quality management metrics as well. Examples include test
Agile Testing: How to Succeed in an Extreme Testing Environment
228


case coverage versus requirements, defect detection and distribution rates, code
coverage, and code complexity metrics. The availability of such metrics can be
used to improve the effectiveness of the testing process, such as identifying which
aspects of the application are more prone to defects, which testers are most effec-
tive at ¬nding bugs, and supporting the comparison and analysis of historical and
current data.
On the smallest agile projects with relatively simple tasks, daily stand-up
meetings can be a particularly effective means of reviewing such data. With
growing team sizes and increasing project complexity, more formal means of
obtaining estimates of productivity and calculating progress are needed. While
spreadsheets may provide a simple and frequently used solution to recording the
estimates and actual effort needed to compete a task, larger and more complex
projects may need to employ tool support (such as [60, 63]) to collate such data.
Progress visualization “ Although it is important for the project leader to have
access to accurate and up-to-date project data, there is also a great deal of value
in displaying progress information to the wider project stakeholders. Examples of
progress visualization solutions drawn from the case studies include traditional
approaches such as pinning up project burndown charts in shared project areas
and using distributed electronic data (such as the graphical code coverage and
complexity information provided through the continuous integration and build
solution described in Chapter 14).
Visualization of project data means that the project stakeholders can make use
of this information to make decisions about how to improve their performance.
For example, in the case study described in Chapter 18, displaying a chart showing
the breakdown of defects detected versus the testing phase in which they were
found provided developers with the encouragement to improve the effectiveness
of unit testing. Similarly, the automated switching on of the red lava lamp when
a build failed due to detection of a defect provided a massive motivator for the
team to ¬x the problem and get the green lava lamp back on before the red wax
became hot enough to melt and ¬‚ow!
Process improvement “ While process improvement should apply to all agile
projects, the scope of this activity will change depending on the size, duration,
complexity, and importance of the project. For small projects, agile retrospectives
held at the end of each iteration/sprint (and particularly heartbeat retrospectives
[54]) may be suf¬cient. As project size, duration, and complexity grow, more
formal approaches to process improvement may be appropriate; for large, long-
duration, and complex projects, or projects being run within an organization that
has corporate interests in process improvement, a scheme such as CMMI [37]
might be employed.
Everyone is a software engineer “ To avoid the adversarial mindset that is fre-
quently reported between developers and testers, a number of the case studies
reported the bene¬ts of replacing the titles of those practitioners responsible for
development and testing with the title of software engineer. This practice was
My Agile Process
229


shown to provide bene¬ts in one of the case studies by improving productivity and
lowering quality issues through closer working, and in another by encouraging
those staff performing development tasks to seek out staff with quality assurance
skills for advice. Combined with the notion that software quality is everyone™s
responsibility, this practice can lead to signi¬cant improvements in the quality
of the delivered software.

Agile Requirements Management
Use cases and user stories “ Strongly consider having a test practitioner work
alongside the analyst and customer during the process of requirements elici-
tation to review the requirements for testability, to help identify errors in the
requirements, and to create the test cases in parallel with the use cases or user
stories. Designing test cases based on use cases is a very natural process, because
the use cases which describe a series of actions leading to a speci¬c outcome
map very easily into test scenarios, which specify the navigation that needs to be
performed to produce some expected result.
Look beyond requirements to customer needs “ This practice provides valuable
guidance about what to focus on when trying to understand precisely what the
customer needs. An overly formal approach to requirements elicitation may under
certain circumstances miss those aspects of the proposed software system that
will deliver genuine value to the customer.
Being aware that you cannot know all the customer requirements at the
start of the project and adopting an approach in which you re¬ne the customer
needs at each iteration will help address this issue. The feedback obtained using a
rapid prototyping approach may also help in further understanding the customer
needs.
Ensure all requirements have tests “ To ensure complete test coverage of the
requirements, each requirement must have one or more associated tests. On
smaller agile projects, simple tool support may be used (such as a spreadsheet
that lists the requirements, with a corresponding column to show the associated
test case). For larger agile projects that have access to requirements management
and testing tools, you should look for integrated solutions that are able to map
test cases back to speci¬c requirements to demonstrate test coverage.

Agile Communication and Meetings
Agile project start-up meetings “ Agile projects should begin with a start-up
meeting in which issues such as the team structure, project planning, risks
and dependencies, architecture, and product release planning are discussed and
agreed. Ensure quality management issues such as testing roles and responsibil-
ities, test planning, and test resourcing are also covered in this meeting.
Agile iteration start-up meetings “ Each project iteration should begin with an
agile start-up meeting, whose agenda should include a discussion of the priority
of the features that should be implemented within the iteration, agreement on the
Agile Testing: How to Succeed in an Extreme Testing Environment
230


iteration plan, discussion and agreement of the success criteria for the iteration,
and identi¬cation of any iteration-speci¬c risks and dependencies. On a Scrum
project, for example, this is analogous to a sprint planning meeting.
Daily stand-up meeting “ During the course of the iteration, the project team
should get together for a short (typically ¬fteen-minute) stand-up meeting for
the purposes of discussing three things: (1) what progress they made the previous
day; (2) what they plan to achieve today; and (3) what issues, problems, risks, or
dependencies might prevent them from achieving their plans.
Agile retrospectives “ Retrospectives should be held at the end of each iteration to
ensure that the lessons learned during that iteration are discussed and recorded.
These could be as short and simple as heartbeat retrospectives [54], or more
formal (perhaps following the guidelines provided in [72, 99]). Ensure quality
management issues are covered in the meeting, for example, by reviewing the
success of automated testing tools, highlighting lessons learned in trialing some
new testing technique, or reviewing how well testing plans were adhered to
during the iteration.
An agile retrospective should also be held at the end of the project to ensure
that the lessons learned on the project are discussed and documented. This will
be particularly useful if there are likely to be similar or related follow-on projects
with the same customer.

Agile Automation
Automated unit test “ From the analyses of the case studies, automated unit
testing is a universally successful agile practice; all developers should be provided
with access to automated unit testing tools (such as [50]). Such tools are simple to
use, provide good results, and can be easily integrated into automated continuous
integration and build solutions, where the unit tests can be automatically run
to verify the quality of the code. For small- to medium-sized projects, free open-
source unit test tools may well be suitable. For larger projects, review the needs
of unit testing tools to integrate with other tools and/or test harnesses being used
on the project and consider the purchase of commercial products if appropriate.
Automated con¬guration management “ All agile development projects should
consider the use of con¬guration management tools as a general best practice. For
small- to medium-sized projects, free open-source con¬guration management
tools may well be suitable. For larger projects, review the needs of con¬guration
management tools to integrate with other tools being used on the project and
consider the purchase of commercial products if appropriate.


25.3 Agile Best Practices for Small-Sized Projects
The best practices documented in this section have been shown to have value when
used on small agile projects. Combine the agile practices documented in the following
sections with those described in Section 25.2 to provide a set of best practices for
use on small projects.
My Agile Process
231


Agile Development and Testing for Small-Sized Projects
Pair testing “ Where there are suf¬cient resources on the project, consider the
value of pairing a testing resource with a developer to provide software quality
advice and guidance during development. Be careful not to give the impression
that the tester is performing a “quality police” role. This practice works par-
ticularly well if you have avoided the “us and them” issues often seen between
developers and testers by encouraging these practitioners to view themselves as
software engineers.
Continuous integration “ Although this practice works particularly well where
there are a number of developers writing and testing code, its value on smaller
agile projects, perhaps used in combination with tool support, should also be
carefully considered.
Rapid prototyping “ Consider the role of this practice on small agile projects but
ensure that, if used, the prototypes are developed in such a way as to maximize
their potential reuse in developing any deliverable code produced as a result of
the prototyping exercise; that is, be careful not to waste time writing code that
isn™t actually used in the ¬nal system.

Agile Process and Project Management for Small-Sized Projects
Metrics “ There may be some value on a small agile project in collecting simple
metrics to support the analysis of quality management information, such as defect
detection rates. Without having such metrics available, it will be very dif¬cult to
measure the bene¬ts of any test process improvements one might make.
Since the activity of collecting and analyzing metrics involves time and effort,
the use of metrics on a small agile project must be carefully considered. In the
event that collecting and using metrics is thought to be of value, any metrics
scheme should be very clear about the purpose the metrics are being collected
for and should be focused on collecting just the data required for that purpose.
The bene¬ts of any adopted metrics scheme should be reviewed during agile
retrospectives.

Agile Communication and Meetings for Small-Sized Projects
Improve interpersonal communications “ Even on small, co-located agile
projects, effective interpersonal communication is essential to the success of
the project. In fact, in small teams, poor communications problems can have a
disproportionately high impact. For example, it is possible for one particularly
forceful personality to dominate meetings. Similarly, it is quite possible for the
least assertive member of the team to have the most important contribution to
make but not be given the opportunity to make it! Make use of the communica-
tions techniques described in Chapter 19 (as well as in [72, 99]) to signi¬cantly
improve the quality and effectiveness of communications in small agile projects.
Interim iteration meetings “ Although of more value in medium and large agile
projects, in projects involving longer iterations, or where some important stake-
holder cannot be co-located, there may be some value in holding a meeting
Agile Testing: How to Succeed in an Extreme Testing Environment
232


midway through an iteration or at some other appropriate point(s) to review
progress, review risks and dependencies, and/or discuss lessons learned.
Reasons for holding such a meeting might include the availability of a par-
ticular stakeholder (such as a non-co-located project sponsor) at a speci¬c point
in the iteration, the occurrence of some event external to the project that may
impact project progress, or the detection of some signi¬cant risk or dependency
that needs to be reviewed. However, the scheduling of additional meetings should
always be challenged on a smaller agile project, since it is quite possible that the
daily stand-up meetings will be suf¬cient.
Agile project closedown meetings/extended retrospectives “ Consider holding a
brief agile project closedown meeting if there are outstanding items not covered
in the agile retrospective. If this is likely to be the case, consider combining the
retrospective and closedown meetings into a single meeting that includes the
agenda items from both.

Agile Automation for Small-Sized Projects
Role of static analysis tools “ Consider the use of one of the open source static
analysis tools that are available to check the quality of the code generated in the
project. These tools can be used to detect issues or errors in the code that need
to be addressed prior to execution, with a number of these tools also capable of
being customized to check project or organization issues, such as adherence to
a particular coding standard or company style.
Test harness tools “ Where a signi¬cant need exists to test software compo-
nents that interact with other components that have not yet been developed, the
potential use of test harness tools should be considered. In addition to tools that
may have been developed in-house, there are a number of free test harness tools
available (e.g., [51]) that can save the project time and effort.
Requirements management tools “ The majority of small agile projects employ
simple, cheap (or free) solutions to record and use project requirements; very
often, a simple spreadsheet will suf¬ce. Where more rigor is required on a project “
to formally document a customer™s requirements or to be able to trace the origin
of some speci¬c requirement, for example “ a more robust and reliable solution
many be required. The use of a commercial requirements product should be
considered carefully on a small agile project, because such tools may need to be
purchased and, even where free, are likely to incur training and familiarization
costs.


25.4 Agile Best Practices for Medium-Sized Projects
The best practices documented in this section have been shown to have value when
used on medium-sized agile projects. Combine the agile practices documented in
the following sections with those described in Section 25.2 to provide a set of best
practices for use on medium-sized projects.
My Agile Process
233


Agile Development and Testing for Medium-Sized Projects
Pair testing “ With increasing numbers of staff on the project, there is increasing
value in pairing a testing resource with a developer to provide software quality
advice and guidance during development. This practice will work particularly well
if you have adopted the approach of encouraging developers and testers to see
themselves as software engineers, avoiding any “us and them” issues. Where you
do observe push-back from developers who perceive the tester as being a member
of the “quality police,” consider involving the test resource in a reviewing role
toward the end of the workday.
Continuous integration “ As the number of developers on the project team
increases, this practice becomes increasingly valuable with individual developers
submitting completed code to the growing code base. Using tool-based contin-
uous integration, it is not only possible to automate the build process, but it is
also possible to automatically run other tools, such as unit test, code coverage,
code complexity, and functional/regression testing tools.
Combined with innovative techniques for visualizing the results of a build,
such as the graphical code coverage and complexity information provided through
the continuous integration and build solution described in Chapter 14 and the
practice of ¬xing all defects as soon as they are found, this becomes a highly
effective and ef¬cient means of building defect-free software.
Test refactoring “ Where the testing suite becomes large and complex, dif¬cult to
manage, and/or takes an unacceptably long time to execute, you should consider
the value of test refactoring. Removing test scripts from the test suite (but not
deleting them, so that they can be reused later in the project if required) and/or
editing or modifying them can result in a leaner, meaner, and more effective test
suite. Care must be taken to target only those tests that are no longer being used,
that are out of date, or that test low-risk or tried-and-tested elements of the code
base. Combined with the practice of identifying targets of test, test refactoring
provides a valuable solution for improving the ef¬ciency of your test effort.
Identify targets of test “ In the event that the suite of test scripts becomes suf-
¬ciently large that there are dif¬culties in completing functional and regression
testing within allocated timescales, you should consider the use of this practice
to assist you in selecting an effective subset of tests to run that will still provide
you with good results from a quality perspective. Chapter 18 provides excellent
advice and guidance on this best practice.
Code coverage metrics “ Consider the use of this practice as part of the process
of ensuring good test coverage of the software. Where there are large numbers
of tests to run and the application under test is large and/or complex, you should
consider the use of a code coverage tool. Such a tool can also be of value in
assisting you to identify targets of test (see previous section) as well as assist-
ing you to target additional testing effort in those areas of the software where
more thorough testing is needed. Combined with the automated build process
and sophisticated graphical solutions for visualizing code coverage described in
Agile Testing: How to Succeed in an Extreme Testing Environment
234


Chapter 14, this practice can quickly and easily highlight code coverage issues
before they seriously impact the project.
Rapid prototyping “ This practice can be of great value in exploring the customer
requirements, gaining valuable feedback, and ensuring that the ¬nal system
matches the customer needs. Where used, ensure that the prototypes are devel-
oped in such a manner to maximize the reuse of the code in developing the ¬nal
system; that is, don™t waste time and effort writing code that isn™t actually used
in the ¬nal system.

Agile Process and Project Management Practices for Medium-Sized Projects
Be receptive to the reuse of traditional testing techniques “ With increasing
project size and complexity, the bene¬ts of traditional project and test manage-
ment practices should be considered. For example, there are a couple of case
studies in which the V-model [4] has been shown to have helped with test plan-
ning in agile projects, and where elements of PRINCE2 [22] have been of value
in agile project management. Where you have very experienced testing staff
involved in your agile project, be receptive to any suggestions they may have
about traditional practices that may bene¬t the current project.
Availability of agile process guidance “ On a medium-sized agile project, with a
moderate team size, where team members leave and/or join the team frequently or
one that involves new and inexperienced agile practitioners, you should strongly
consider the need to make formal guidance on the agile process being used
on the project available to the team members. Effective solutions can include
distributing process summary cards as and when needed [34] and publishing the
process guidance online [7], for example.
Metrics “ Metrics can be used for a variety of purposes in medium-sized agile
projects, such as ¬ne-tuning estimates, publishing data on some aspect of the
project, and supporting process improvement. Because the activity of collecting
and analyzing metrics involves time and effort, the use of metrics on a medium-
sized agile project should be considered with care. Any metrics scheme that is
implemented should also collect data recording the time and effort expended on
metrics collection, and this information should be used in agile retrospectives
and iteration start-up meetings to determine the bene¬ts of, and the continuing
role and use of, metrics on the project.
Reduce documentation overload “ As agile projects increase in size, it is very
likely that the amount of documentation used on the project will also grow.
The time and effort taken to compile, distribute, maintain, and make use of
these documents have signi¬cant implications for the agility of the project. In
medium-sized projects where the volume of documentation is likely to be an
issue, strategies should be investigated for managing the problem. Daily stand-
up meetings have been proposed as one solution to communicating information
that might otherwise have been typed into memos and/or emails; automation
can offer another (requirements management tools allow information on the
customer needs to be recorded once and accessed many times, for example).
My Agile Process
235


Co-location of project stakeholders “ Although co-location is the ideal option
for organizing the team in a medium-sized agile project, as the team size grows,
ensuring their co-location becomes increasingly challenging. This will be partic-
ularly true where the customer representative is an important individual whose
time is valuable and who has commitments involving them in other roles outside
the agile project. Where it is not possible to ensure all members of the team
are co-located, other strategies for improving communications should be con-
sidered (this subject is covered in greater detail in Section 25.5).

Agile Communication and Meetings for Medium-Sized Projects
Improve interpersonal communications “ Good interpersonal communication
is essential to the success of agile projects, but with increasing project size,
larger teams, and the potential for some of the stakeholders to not be co-located
for part of the project, maintaining good communication becomes increasingly
challenging. During meetings and in other project communication, techniques
to encourage all members of the team to participate must be used, because
all members of the project have a valuable perspective and must be given the
opportunity to contribute. The communications techniques described in Chapter
19 (as well as in [72, 99]) can all be used to signi¬cantly improve the quality and
effectiveness of communication in medium-sized agile projects.
Interim iteration meetings “ For medium-sized agile projects there may be value
in holding a meeting midway through an iteration to discuss progress, review
risks and dependencies, and review outstanding tasks. This would be particularly
appropriate where one or more key stakeholders are not co-located but could
make time to be present for such a meeting. One of the case studies describes
the bene¬ts of scheduling such a meeting to coincide with another regular event
or meeting to enable particular staff to attend both events (Chapter 20). On a
Scrum-style project, this goal might be achieved through a Scrum of Scrums
(in effect, a higher-level meeting where the attendees are representatives drawn
from other Scrums), scheduled at some convenient point during the iteration.
End-of-iteration retrospective “ These longer meetings (of up to four hours™ dura-
tion) provide an opportunity for the team members to re¬‚ect on what worked
well in the iteration, what could have been done better, and to document the
lessons learned as part of the task of process improvement. Ensure that qual-
ity management topics are on the agenda at such meetings and that effective
communications techniques are employed [99].
Agile workshop meetings “ As the project size expands, team numbers grow,
and the complexity and duration of the project increases, there is likely to be a
requirement for larger and more formal meetings. For example, RAD includes the
notion of the joint application design workshop, which is attended by customer
and developer representatives, chaired by a facilitator, and documented by a
scribe. While such workshops can last up to ¬ve days for particularly demanding
tasks in large and complex projects, clearly the smaller the agile project, the
greater the drive for more focused meetings of shorter duration.
Agile Testing: How to Succeed in an Extreme Testing Environment
236


The role of such workshops in medium-sized projects can be of signi¬cant
value as an intensive means of discussing and documenting an important aspect
of the project, but their duration must be challenged in order to ensure they
do not adversely affect project progress. Techniques exist to focus the meetings
(such as the ¬fteen-minute and 80/20 rules [102], and these can be used to ensure
such meetings do not exceed the time allocated to them.
Agile project closedown meetings “ With increasing project size and growth in
team numbers, there may be signi¬cant value in holding a project closedown
meeting above and beyond the end of iteration retrospective meetings. An agile
closedown meeting can be of value by allowing the lessons learned from earlier
iteration retrospectives to be collated, providing an overall project perspective on
what worked well and what could be improved (Chapter 15). From a Scrum per-
spective, such a meeting could be supported using a Scrum of Scrums approach,
with one or more representative from each retrospective providing their input
on the key lessons learned.
Ensure quality management issues are covered in the meeting, such as review-
ing the success of automated testing tools, highlighting lessons learned in trialing
some new testing technique, or reviewing how well testing plans were adhered
to during the iteration.

Agile Automation for Medium-Sized Projects
Tool selection process “ For those medium-sized agile projects where there will
be signi¬cant requirement for the use of automation, it may be necessary to
undertake a formal tool selection process. This process should be as lightweight
as possible and could rely heavily on the previous experience of the team members,
who may already have preferences for particular tools. Any tool review, evaluation,
and selection process must take into account the integration and interoperation
needs of other tools already selected or in the process of being selected for use
on the project.
Although open-source products may be acceptable on small-sized agile
projects, their use may need to be challenged on medium- or large-sized projects.
In particular, issues such as availability of support, experienced users, training,
upgrade, and maintenance should all be considered when making any decisions
about a particular tool. Company standards and guidelines on tool selection and
use may also need to be considered. Jon Tilt (Chapter 18) offers this advice on
tool selection:
There is always the temptation to architect yet another automation framework at the
start of the project. Don™t! Look around at the simple tools that already exist and build
on them as required. Developers and testers are much more likely to use something
simple that is quick and easy to use.
There may be some bene¬t in terms of reduced time and effort in using the tools
evaluation framework presented in [4].
My Agile Process
237


Static analysis tools “ While the use of one of the free static analysis tools
should be considered to check the quality of the code generated in the project,
the use of more professional commercial static analysis products should also
be investigated, particularly if the tool needs to integrate with any existing or
planned tools in use on the project.
Test harness tools “ Where there is a signi¬cant need to test software compo-
nents that interact with other components that have not yet been developed, the
potential use of test harness tools should be considered. In addition to those tools
that may have been developed in-house, there are a number of free test harness
tools available (e.g., [50]) that have been reported to save the project time, effort,
and money.
Functional test tools “ For small agile projects it is likely that the volume of
functional and regression testing can be satis¬ed using a manual test approach.
As the project size increases, it may still be possible to manage the increasing
manual testing load using good test management techniques (such as reusable
test scripts and test packs). At some point, however, when the duration needed
to execute the manual test suite no longer ¬ts into the planned timescales,
when there are frequent builds and releases of the software, and/or when the
maintenance of the manual test suite becomes an issue, it may be necessary
to consider the use of a commercial capture“replay tool. Such tools record
the navigation and veri¬cations employed in a manual test in the form of an
automated test script that can be replayed against later builds and releases to
perform regression testing (see [62], for example).
Requirements management tools “ Although small agile projects may be able to
employ in-house or open-source requirements management tools, medium-sized
projects are more likely to require an increasingly rigorous, robust, and reliable
solution. A number of commercial products are available (see [59], for exam-
ple) that provide facilities for documenting, utilizing, and maintaining project
requirements. In selecting such a product, be aware of company standards and
guidelines, the knowledge and advice of colleagues, and the need for the tool
to integrate with existing or planned project tools. For example, a requirements
product may need to integrate with a test management tool for test planning and
design purposes.
Build management tools “ Build management tools automate the process of gen-
erating a build, bringing together the units and components required to assemble
the application under development. Used in combination with a continuous inte-
gration approach and potentially incorporating automated testing, these tools
can provide a powerful solution to building and testing the software being devel-
oped. While the use of one of the open-source build management tools may be an
option to consider in relatively small agile projects, as the project size, number
of developers, and complexity of the software increase, there is a growing need
to consider employing one of the more robust and reliable commercial build
management tools.
Agile Testing: How to Succeed in an Extreme Testing Environment
238


Change management tools “ Where agile teams are relatively small and co-
located, simple communications between the stakeholders may well be suf¬cient
to ensure that requests for changes to the software being developed are imple-
mented. With increasing team size, complexity of the system under development,
and potential lack of co-location of key stakeholders, the role and use of a change
management tool becomes more compelling. On small agile projects, in-house
or open-source con¬guration management tools may be an option, but as the
project size increases, the use of more robust and reliable commercial products
(such as [95]) needs to be considered.
Defect tracking tools “ Having spent signi¬cant amounts of time, effort, and
money tracking down bugs in the software, it is essential that these defects
get ¬xed. Similarly, when customers report defects in the software following its
release, it is important that these are ¬xed by a patch or in the next release of the
software. To ensure the formal logging of defects, their correction, and retest,
some form of defect tracking mechanism must be employed on the project.
In a small agile project with co-located stakeholders, which employs an
approach where all defects are ¬xed immediately, the use of a commercial defect
tracking tool may be unnecessary. However, as project size increases “ with
growing numbers of developers and testers, a large code base, and correspond-
ing defect reports “ a rigorous commercial solution (such as [52]) will become
increasingly necessary.
Often, change management tools will also be used for defect tracking (since
a defect is arguably a special case of a change request). Whatever defect tracking
solution is selected, ensure it integrates with the functional, regression testing,
and test management tools already being used or planned for use in the project.
Process enactment tools “ For larger agile projects, particularly those involving
new or inexperienced agile practitioners or where project stakeholders are dis-
tributed across a number of separate sites or even offshore, it may be a challenge
to ensure that all stakeholders follow the chosen process correctly.
Process enactment solutions (see [63], for example) provide an integrated
development environment in which the users are constrained to follow agile prac-
tices. Such environments typically incorporate sophisticated facilities to promote
effective communications across the project (which can include the offsite and
offshore team members). Developer estimates and actuals can also be captured
in real time to enable the project leader to accurately report on project progress.


25.5 Agile Best Practices for Large-Sized Projects
The best practices documented in this section have been shown to have value when
used in large-sized agile projects. Combine the agile practices documented in the
following sections with those described in Section 25.2 to provide a set of best
practices for use in large-sized projects.
My Agile Process
239


Agile Development and Testing for Large-Sized Projects
Pair testing “ In combination with the practice of involving testing practitioners
as early as possible in the software life cycle, strongly consider pairing testing
resources with developers to provide software quality advice and guidance during
development. Combined with the practice of encouraging developers and testers
to see themselves as software engineers, pair testing can be a particularly effective
practice to improve the quality of delivered code. Where you do observe push-
back from developers who perceive the tester as being a member of the “quality
police,” consider involving the test resource in a reviewing role toward the end
of the workday.
Continuous integration “ The practice of continuous integration becomes
increasingly valuable with a large team of developers writing and submitting
completed code in parallel with the growing code base. A tool-based continuous
integration solution should be adopted to automate the build process and also
run other tools such as unit test, code coverage, and functional/regression testing
tools to verify the quality of each build as it is generated.
Combine continuous integration and automated build with innovative tech-
niques for visualizing the results of a build, such as the red“green lava lamp
solution described in Chapter 17 (where a red lava lamp would come on if a build
failed). Ensure all defects that are highlighted through this approach are ¬xed as
soon as they are found to support the process of building defect-free software.
Test refactoring “ For a large agile project, with a large team and a complex appli-
cation, there is also likely to be a large and complex test suite. With good test
management practices, the test suite may be well structured, perhaps involving
subsuites linked to particular functional areas of the application under devel-
opment, or associated with testing code developed within a speci¬c iteration.
However, if the test suite is not well structured, or there are dif¬culties in com-
pleting the regression testing within acceptable timescales, test refactoring must
be considered. Removing tests2 that cover some aspect of the application under
test that you have a high level of con¬dence in, or modifying existing tests, may
improve the ef¬ciency of the regression test. This practice should be combined
with the practice if identifying targets of test.
Identify targets of test “ In the event that the suite of test scripts becomes suf-
¬ciently large that there are dif¬culties in completing functional and regression
testing within allocated timescales, you should consider the use of this practice
to assist you in selecting an effective subset of tests to run that will still provide
you with good results from a quality perspective. If you have a well-structured
test suite “ perhaps involving subsuites linked to particular functional areas of
the application under development, or associated with speci¬c iterations “ it may
be possible to simply leave a particular subsuite out of the current regression

2 Ensure the scripts are removed, but not deleted; they may need to be reused in a later testing phase.
Agile Testing: How to Succeed in an Extreme Testing Environment
240


test (particularly if the tests are associated with a tried and trusted part of the
application under test). More rigorous approaches are also possible for selecting
the individual scripts to omit from any particular test, involving code coverage
metrics and statistical calculations (see Laser-Guided Testing in Chapter 18, for
example).
Code coverage metrics “ When running a large test suite against a complex
application under test, it is very dif¬cult to have con¬dence that each and every
code statement has been covered in the test, that all subroutines have been
tested, or that all possible logical paths through a conditional statement have
been exercised. The use of code coverage tools and the metrics they produce will
give you an accurate picture of just how thorough your testing has been. Such
tools can also be of value in assisting you to identify targets of test (see previous
section).
Combined with the automated build process and sophisticated graphical solu-
tions for visualizing code coverage described in Chapter 14, this practice can
quickly and easily highlight code coverage issues before they seriously impact
the project.
Rapid prototyping “ The practice of rapid prototyping can be of major value in
a large agile project “ exploring the customer requirements, gaining valuable
feedback, and ensuring that the ¬nal system matches the customer needs. It is
important to make sure that the prototypes are developed in such a manner that
maximizes the reuse of the code in developing the ¬nal system; avoid wasted time
and effort writing code that isn™t actually used in the ¬nal system. In a large agile
project, also look for opportunities to share the use of a particular prototype with
other developers, who may be able to reuse the code in the completion of their
tasks.

Agile Process and Project Management for Large-Sized Projects
Be sensitive to the reuse of traditional testing techniques “ In large agile projects,
with large (and potentially distributed) teams and involving complex software
development, you must consider the value of reusing traditional development
and testing techniques where appropriate. V-model [4], for example, is a tried and
trusted solution to organizing testing plans that has been shown to have bene¬ts
when used in agile projects, and particularly when applied to test planning in
longer iterations.
Many agile methods reduce the number of testing phases used in a traditional
test process (such as XP), but, depending on the project, it may be necessary to
increase the number of testing phases. For example, in developing an application
that needs to integrate and interoperate with other existing systems, you may
have to consider introducing a systems integration test phase to ensure these
requirements are properly tested.
Similarly, with increasing project size and complexity, the bene¬ts of tradi-
tional project and test management practices should be considered. For example,
My Agile Process
241


elements of the PRINCE project management method [22] have been shown to
have been of value in agile project management. Where you have very experienced
testing staff involved in your agile project, be sensitive to any suggestions they
may have about traditional practices that may bene¬t the current project.
Progress measurement “ For large agile projects with large teams and extended
timescales, the availability of accurate information on progress is of essential
importance, particularly where timescales and budgets are likely to be tight
and meeting planned milestones and delivery dates are critical. Add to this the
likelihood that it may not be possible for the whole team to be co-located on a
large project, and collecting and analyzing progress data quickly becomes a major
challenge. While simple ¬fteen-minute stand-up meetings may be suf¬cient in
small agile projects to discuss progress, such techniques can become unwieldy
and inef¬cient in large projects.
The role and use of tool support must be considered in large projects. While
simple tools, such as spreadsheets, may provide a practical and inexpensive (in
terms of tool acquisition costs) solution to collecting and recording estimates of
the time and effort to complete a task and the actual time expended on the task,
they must rely on timely and accurate reporting of progress and estimates for
completion. Commercial products are available that automatically collect data
on estimates and actuals as tasks are planned and completed. These products do
not have to rely on the vagaries of optimistic or pessimistic estimates or on the
need of the developer to submit these estimates; they can provide the project
leader or manager with an accurate view of the state of the project in real time
(see [63, 65], for example).
Ensure that effective progress measurement is combined with effective meth-
ods for progress visualization.
Progress visualization “ In addition to the importance of making project progress
information available to the project leader, it is essential that all the stakeholders
on the project have access to key project information. With increasing numbers
of staff working on a larger agile project, and with the likelihood that not all of
the stakeholders will be co-located, providing accurate and effective information
to the team is challenging. Project and/or iteration burndown charts that are
attached to the wall in a shared project area are unlikely to be effective under
such circumstances, and more sophisticated means of sharing project data are
needed.
Consider publishing project information via the Internet or a local intranet,
particularly if the tools that have been selected for use on the project have a web
interface, which allows stakeholders to remotely access requirements informa-
tion, for example (see [61]). Other tools can be used that allow the setting up of
a project portal, where customized views of project data can be displayed [103]
and could also be adopted as a means of delivering agile process guidance to the
stakeholders. This is an area where thinking out of the box can bring signi¬cant
results, such as the more unusual display methods described in the case studies,
Agile Testing: How to Succeed in an Extreme Testing Environment
242


such as the solutions described by David Evans in his case study (Chapter 17)
that were used to show the quality results of the code integration process.
Availability of agile process guidance “ A very signi¬cant number of the case
studies point to agile project success being linked to the availability of well-
trained, well-motivated, knowledgeable, and experienced agile practitioners. It
may be a relatively simple task to obtain the small numbers of such individuals
for modest-sized agile projects, but trying to staff a large agile project entirely with
such superdevelopers and testers would be both a major recruitment challenge
and have serious cost implications for the delivery of the project (although it
could be argued that a large project staffed entirely with the cream of agile
practitioner-hood would be bound to complete early and under budget “ if we
ignore the issues of obtaining such a team in the ¬rst place).
The reality on projects of any signi¬cant size is that they will almost certainly
be staffed with a mixture of experienced practitioners and less-experienced staff.
Also, over the course of an extended project, it is very likely that staff will leave
or join the team fairly frequently. Under such circumstances care must be taken
to ensure project staf¬ng is organized in a way that ensures a good mixture of
experienced and inexperienced staff on tasks, and to encourage skills transfer and
mentoring. It is also critical to have good agile process guidance readily available
to the team for reference and for skills enablement purposes.
Effective solutions can be as simple as providing process guidance on ready
reference sheets (such as those used with [34]). Internet-based portal solutions
can also be used to deliver process guidance to the team via web browsers [103]. In
large agile projects that are staffed with a high proportion of new or inexperienced
agile practitioners and/or those that involve offsite or offshore teams, there may be
value in using a process enactment“style product that provides a customizable
agile development environment, and which itself embodies and enforces the
principles and practices of whatever agile approach has been selected for use on
that project ([63], for example).
Metrics “ It is strongly recommended that a metrics scheme be set up and run
for a large agile project. A metrics scheme can provide value to the project in a
number of areas, including assisting with more accurate estimation, supporting
process improvement, and feeding information to the project portal (or other
means of distributing key project information). Given that there is already likely
to be signi¬cant project activity associated with collecting information about
estimates and actuals, productivity, defect detection rates, code coverage, and so
forth, the additional effort expended in ensuring that this information is reused
in a wider metrics scheme should not result in a large overhead.
When deciding what metrics to collect, ¬rst make sure you are clear about
the goals of the scheme and precisely what it is you intend to investigate or
analyze, since this will guide your selection of data. Make sure that the effort
expended collecting and analyzing the metrics is recorded in the metrics scheme
(to determine how much time and effort it costs to run the metrics scheme).
My Agile Process
243


Review the bene¬ts of the metrics scheme in the project retrospectives and
challenge the value of continuing to run the scheme and, where appropriate,
change how the scheme is run. Consider the role and use of tool support for
the scheme, particularly if an existing product that is being used on the project
can provide access to the data needed. [4] contains comprehensive guidance on
setting up and running a metrics program.
Reduce documentation overload “ The time, effort, and cost required to com-
pile, distribute, maintain, and make use of documents on a large project have
signi¬cant implications for the agility of the project. Also, paper-based solu-
tions are notoriously error-prone, involving the potential loss of critical project
information.
While daily stand-up meetings may be a solution to communicating informa-
tion that might otherwise have been typed into memos and/or emails for small
and even medium-sized agile projects, such techniques can become unwieldy and
inef¬cient in large projects.
Information that might, in traditional projects, have been recorded and dis-
tributed using paper-based solutions, such as project progress information, pro-
cess guidance, and reports, can all be managed using appropriate tools. Require-
ments management tools provide a thorough and effective means of documenting
and maintaining project requirements, while a web-based project portal [103] can
be used to provide real-time graphical displays of project progress and instant
access to agile process guidance, as well as graphs, bar charts, and pie charts
showing defect rates, code coverage, and metrics revealing where in the project
life cycle the most defects are being detected.
Co-location of project stakeholders “ Although this is the ideal option for orga-
nizing the team in small- and medium-sized agile projects, as the team size
grows, ensuring their co-location becomes increasingly challenging. This will be
particularly true where the customer representative is an important individual
whose time is valuable and who has other demands on her or his time.
For those large projects with stakeholders that are offsite, effective solutions
need to be found to ensure that they are able to communicate with other team
members who are co-located as ef¬ciently as possible. A range of solutions can
be investigated, such as simple instant messaging systems; Internet telephony,
with or without webcam technologies; full-blown videoconferencing facilities; or
even use of Web 2.0 technologies, such as virtual project meetings in Second Life
[64]. This subject is covered in greater detail in Section 25.5.

Agile Communication and Meetings for Large-Sized Projects
Improve interpersonal communications “ Good interpersonal communication is
vital to the success of large agile projects; but with increasing project size, larger
teams, and the potential for some of the stakeholders to not be co-located for part
or all of the project, maintaining effective communications becomes increasingly
challenging.
Agile Testing: How to Succeed in an Extreme Testing Environment
244


In meetings and other project communication, techniques to encourage all
members of the team to participate must be used; all members of the project
have a valuable perspective and must be given the opportunity to contribute,
and instances of where dominant personalities may hijack meetings or where
knowledgeable but less extroverted individuals are prevented from contributing
must be avoided. Similarly, when important issues need to be discussed, tech-
niques for keeping all members of the team focused on the task at hand are key
in facilitating larger meetings.
Make good use of the communications techniques described in Chapter 19,
as well as in [72, 99], to signi¬cantly improve the quality and effectiveness of
communications in large-sized agile projects.
Interim iteration meetings “ For large-sized agile projects involving relatively
long iterations, there is likely to be value in holding a meeting at the midpoint
(or more frequently if the iteration is particularly long) to discuss progress,
review risks and dependencies, and review outstanding tasks. This is particularly
appropriate where one or more key stakeholders are not co-located but are able to
make time to be present for such a meeting. One of the case studies also describes
the bene¬ts of scheduling such a meeting to coincide with another regular event
or meeting so that particular staff can attend both events (Chapter 20).
On a Scrum-oriented project, this goal might be achieved through a Scrum of
Scrums (in effect, a higher-level meeting where the attendees are representatives
drawn from other Scrums). In particularly large Scrum projects, this might even
be achieved using a Scrum of Scrum of Scrums [104].
Agile workshop meetings “ As the project size expands, as team numbers grow,
and as the complexity and duration of the project increase, there is likely to be
a requirement for longer and increasingly formal meetings that involve more
stakeholders. In RAD projects [18], joint application design (JAD) workshops are
attended by customer representatives, developers, and project administrators and
can last up to ¬ve days for particularly demanding tasks in large and complex
projects.
Where such meetings are necessary, the proposed duration of the event must
be challenged to ensure it does not adversely impact project progress. Addi-
tionally, make good use of techniques to control the duration and focus of the
meetings (such as the 15-minute and 80/20 rules [102]) as well as using the
communications best practices described in Chapter 19 and in [72, 99].
Agile project closedown meetings “ For large projects with extended timescales
and a large number of stakeholders, there may be signi¬cant value in holding a
project closedown meeting above and beyond the end of iteration retrospective
meetings. Such meetings can be of value by allowing the lessons learned from
earlier iteration retrospectives to be collated, providing an overall project per-
spective on what worked well and what could be improved (Chapter 15). From a
Scrum perspective, such a meeting could be supported using a Scrum of Scrums
My Agile Process
245


approach, with one or more representative from each retrospective providing
their input on the key lessons learned.

Agile Automation for Large-Sized Projects
Tool selection process “ On large-sized agile projects where signi¬cant use of
automation is likely, or where an automation architecture is planned or in the
process of being set up, it may be necessary to follow a formal tool selection
process. This process should be as lightweight as possible, and could rely heavily
on the previous experience of the team members, who may already have had
exposure to particular tools.
Any tool review, evaluation, and selection process must take into account the
integration and interoperation needs of other tools already selected, or in the
process of being selected, for use on the project, as well as company guidelines
on tool selection. Also, investigate what tool skills are available within the team;
decisions involving the selection of a product that nobody knows about, in pref-
erence to one that members of the project team already have experience of using,
should be challenged.
Also challenge the use of open-source products; although their use may appear
to offer cost savings, issues such as availability of support, experienced users,
training, upgrade, maintenance, and scalability should all be considered in mak-
ing any decisions about a particular tool. Similarly, the need for a particular
candidate tool to integrate and interoperate with existing or planned products
needs to be carefully considered.
To save time and effort in evaluating products, and to adopt a formal rigorous
approach, consider using the tools evaluation framework presented in [4].
Static analysis tools “ Strongly consider the use of a commercial static analysis
tool to check the quality of the code generated in the project. For large projects
with many developers, these tools can be used to ensure that a consistent approach
to development is maintained. Examples of these tools exist that can also be used
to verify that code conforms to project or company coding standards.
Test harness tools “ In a large project there is likely to be a signi¬cant need to test
software components that interact with other components that have not yet been
developed. Under such circumstances, the use of test harness tools should be
strongly considered. Although individual developers may generate their own test
harnesses, to encourage standardization and reuse, you should strongly consider
the use of the open-source test harness tools available ([50], for example) to save
the project time and effort and reduce cost.
Functional test tools “ For larger agile projects involving many developers, with
substantial code generation, and a complex application involving frequent builds
and releases, there is likely to be signi¬cant value in using a capture“replay style
functional and regression testing tool. Such tools are able to record the navigation
and veri¬cations employed in a manual test in the form of an automated test script
Agile Testing: How to Succeed in an Extreme Testing Environment
246


that can be replayed against later builds and releases of the software to perform
regression testing (see [62], for example).
Although they are of value to many projects, capture“replay tools are not
the panacea for all software quality issues; they work best with reasonably stable
software systems (where the user interfaces are not subject to frequent change),
they should be used in combination with integrated requirements management
and defect tracking solutions, and you should consider the value of using an
integrated code coverage tool.
Although such tools are valuable in allowing testing to be conducted in
signi¬cantly shorter timescales than equivalent manual testing, and to support
additional unattended testing of the software (overnight or over the weekend,
for example), it is still possible on a large project for the test suite to grow
suf¬ciently large to cause test execution performance problems. One of the case
studies reported the use of an automated test suite that in the worst case could
take up to ten days to execute! Under such circumstances, consider implementing
the “identify targets of test practice” to reduce the size of the regression testing
suite and to allow automated testing to ¬t into the required project timescales.
Requirements management tools “ For large agile projects, with increasing team
size, high complexity of the system being developed, and potential lack of co-
location of key stakeholders, the use of an automated requirements management
solution is essential. Bottom line “ without formal tool support that provides
rigorous traceability, it will become very dif¬cult to ensure that the delivered
system does what the customer actually requires of it.
From a development and testing perspective, distributed requirements man-
agement tools allow the developers to understand what functionality needs to be
implemented to support the customer requirements, and the tester to know what
needs to be tested to ensure the implemented code meets the customers needs.
In selecting a requirements management solution, ensure the tool integrates
and interoperates with existing or planned products (and particularly change
management and test management solutions).
Where important stakeholders are unable to be co-located with the devel-
opment team, ensure that the selected tool provides support for distributed
requirements management. A number of such tools provide a web-based inter-
face to allow distributed teams to inspect, use, and modify the requirements (such
as [95]); such a facility will be essential for offshore projects.
Build management tools “ Such tools automate the process of generating a
build, bringing together the units and components required to assemble the
application under development. A number of the case studies describe the use of
build management tools in combination with a continuous integration approach
that incorporates automated unit test and functional test, and generates code
coverage and complexity metrics. Used as a key component of an integrated tool
architecture, automated build management tools can save projects signi¬cant
time and effort and improve the quality of the delivered software.
My Agile Process
247


Change management tools “ With large and complex projects of extended dura-
tion involving large numbers of stakeholders (many of whom may not be co-
located), the role and use of change management tools becomes critical. The role
of robust and reliable commercial change management products, and particularly
those with a distributed capability, must be considered for use in such projects
(such as [95]). As with other tools, any candidates selected must integrate with
other tools used on, or in, the process of being acquired for the project and should
be selected with regard to possible company guidelines on tool acquisition.
Defect tracking tools “ For large projects where substantial time, effort, and
money are spent on software quality, it is essential that any defects that are found
are ¬xed. Whatever defect tracking solution is employed, it should support a
rigorous work¬‚ow that ensures that defects are highlighted to the appropriate
stakeholders, that defects are ¬xed in a timely manner and retested, and that the
¬x is introduced back into the code base only after formal sign-off of the change
by an authorized member of the team.
Similarly, it is critical that defects reported by the customer following delivery
of the software should be treated in the same manner, ensuring that they are
¬xed and that they do not appear in the next release (or later releases) of the
software.
Investigate the acquisition of a reliable and robust commercial defect tracking
solution that integrates with the functional testing, regression testing, and test
management tools already being used or planned for use on the project. Ensure
that any such tool provides access for any stakeholder that is not co-located with
the project team (this is a particularly valuable facility to provide to the customer “
allowing defect reports to be submitted by the users, or a user representative, from
their own site). Where a change management tool has already been introduced
into a project, investigate whether it also supports a defect tracking solution.
Process enactment tools “ For larger agile projects, particularly those involving
new or inexperienced agile practitioners or where project stakeholders are dis-
tributed across a number of separate sites or even offshore, it may be a challenge
to ensure that all stakeholders follow the same process.
Process enactment tools provide an integrated development environment that
embodies and enforces the use of a particular agile best practice (good examples
of such tools can be customized to support a particular agile method). In effect,
the users are constrained to follow the speci¬c agile practice selected for use on
the project.
Such environments typically incorporate sophisticated facilities to promote
effective communications across the project (which can include the offsite and
offshore team members). Such tools can provide comprehensive support for met-
rics collection and use, with, for example, developer estimates and actuals that are
captured in real time and which can enable the project leader to accurately report
on project progress. Consider using process enactment tools in conjunction with
solutions for making process advice and guidance available.
Agile Testing: How to Succeed in an Extreme Testing Environment
248



25.6 Agile Best Practices for Offsite and Offshore Projects
A key agile best practice is that of co-locating all the project stakeholders. While this
may be a realistic option on smaller agile projects, what happens to agile projects
where key staff cannot be co-located and spend much of their time at a different site,
or where signi¬cant numbers of the team have to work in different geographies?
Under these circumstances, how can effective communication be achieved?
The best practices documented in this section have been shown to have particular
value when used on agile projects where a signi¬cant number of project stakeholders
are offsite or even offshore. The following practices should be used in conjunction
with the agile practices documented in the preceding sections to provide a combined
set of best practices for use in offsite or offshore agile projects.

Agile Process and Project Management for Offshore Projects
Progress measurement “ Due to differences in time zones, combined with poten-
tial language and cultural differences, obtaining accurate and timely project
data is a particularly challenging aspect of offshore projects. Offshore projects
signi¬cantly increase the likelihood that important project information will be
misinterpreted or misunderstood. Wherever possible, consider tool support for
estimating individual and overall project progress. (This issue is covered under
the section on agile automation for offshore projects.)
Progress visualization “ Solutions for delivering essential project data to all mem-
bers of a team that includes offshore assets must address both technical issues
(ensuring that the solution can be distributed) and cultural/language issues
(the information must be easily understood by the offshore team). Although an
Internet-based project portal [103] satis¬es the former issue, the latter is more
challenging, perhaps involving the assistance of a co-located offshore representa-
tive (see the section on non-co-located teams), considering the role of semiotics
[105], or through a pilot scheme to explore and address these issues before the
full project is launched.
Availability of process guidance “ It is critical in an offshore project that all team
members follow the same process in developing the software, and it is essential
that all members have access to guidance on the process that has been adopted
for the project. Process guidance should be made available to all members of the
team via a distributed solution, such as an Internet-based project portal [103].
Be sensitive to language or cultural differences when setting up such a solution,
and ensure that the content and its presentation can be easily understood and
used by the offshore team.
Metrics “ Effective solutions must be found to ensure important project data can
be collected, analyzed, and distributed in an accurate and timely manner. The role
of distributed process enactment products in offshored projects, which provide a
development environment that enforces the use of the agile process selected for
the project, and which also automatically collect data on estimates and actuals,
My Agile Process
249


provide an effective solution to this requirement. (Further information on this
solution is provided in the section that follows on agile automation for offshore
projects).

Agile Communication and Meetings for Offshore Projects
Non-co-located teams “ The analysis of the case studies provides some excellent
best practice guidance on speci¬c solutions for improving the quality of offshore
communications.
Set up and use an effective project communications infrastructure. The facil-
ities could include instant messaging, Internet telephony (potentially involv-
ing a webcam for face-to-face communications), and/or full videoconferencing
tools. Web 2.0 technologies, such as virtual team meetings in Second Life,
may also be of value.
Harmonize offshore time zones. Consider adjusting the working day start and
end times between the different time zones to improve the working overlap
between the zones. Even starting work half an hour earlier in one time zone,
while ¬nishing work a half hour later in the other zone, will make an overall
difference of one hour. Each week or every other week, consider varying the
start and stop time on one particular day by one hour. This overall two-hour
difference could be used for the purposes of holding an extended combined
project meeting.
Co-location of an offshore representative “ On an extended or high-value
project, consider having one of the members of the offshore team co-located
at the primary project site. This can be of signi¬cant value in helping to
address cultural and language issues associated with offshoring.
Agile meetings “ Although the more traditional solution of videoconferencing
can be used to provide “face-to-face” meeting support, also be receptive to the
use of innovative technologies to support meetings with offshore teams. Web 2.0
tools, such as Second Life [64], may provide one practical solution to holding
distributed virtual meetings.
Improving interpersonal communications “ For the success of those agile
projects where key stakeholders or larger parts of the team are offsite or off-
shore, effective good-quality communication becomes critical. In project meet-
ings and other opportunities for contact, all members of the project must be
given the opportunity to communicate clearly as well as have an equal oppor-
tunity to contribute to the discussion. Be sensitive to cultural differences and
language issues, and ¬nd opportunities to establish a good rapport with offshore
colleagues. Chapters 15 and 19 provide excellent advice on improving the quality
of communications under these circumstances.

Agile Automation for Offshore Projects
General “ Any productivity tool that is planned to be employed by all team
members on an offsite and/or agile offshore project must be robust and reliable,
Agile Testing: How to Succeed in an Extreme Testing Environment
250


and speci¬cally intended for distributed, cross-geography operation. Integration
and interoperation issues have to be carefully considered to ensure that any
selected products that need to work together can do so successfully in a distributed
manner. A formal tool evaluation and selection process (such as that documented
in [4]) is essential in selecting the correct products.
Requirements management tools “ To ensure everyone on the project has up-
to-date and accurate requirements information, consider the use of a distributed
requirements management tool. Any candidate product should support dis-
tributed access for all stakeholders, such as via a web interface, to enable all
team members to submit, review, discuss, and use requirements information.
Build management tools “ Strongly consider using distributed build manage-
ment tool support to facilitate continuous integration across the whole team,
including offsite or offshored resources. Integrate testing tools (including unit
test, code analysis, functional and regression test, and code complexity analysis
products) within the build management process to automatically execute the
tests and ¬‚ag those builds containing defects or areas of concern.
Process enactment tools “ The use of such products appears to be of particular
value in offsite or offshore projects. A distributed process enactment tool that
is capable of operating in a cross-geography manner can be used to ensure
all members of the project are following the same agile process in the same
way and that the project manager is able to obtain up-to-date and accurate
information on the progress of the project (see [63], for example). Consider using
a process enactment tool in conjunction with a process guidance tool to ensure
all practitioners have access to information on the agile best practices the project
has adopted.


25.7 Summary
This chapter has provided guidance and advice on how the agile practices identi¬ed
in Chapter 24 can be used as the basis of generating your own agile process. For
additional information describing the details of the best practices and their context,
you can also refer to the case studies.
To assist you in the selection of agile practices to populate your own agile process,
Appendix H contains a checklist of agile practices, showing graphically which prac-
tices are particularly appropriate for different styles and sizes of agile project that
can be used as a visual summary and an aide memoire of the information provided
in this chapter.
Finally, the best process in the world will only ever be of academic interest unless
it is adopted and used by other agile practitioners. Chapter 26 provides you with
guidance on how you can manage the roll-out and adoption of your agile process in
your own organization.
26 The Roll-out and Adoption of My Agile Process
As is so often the case, improving quality turns out not to be a
technology problem, but a people issue and a leadership challenge.
Nick Denning, MD of Diegesis




26.1 Introduction
Having selected and documented your agile best practices, even the best agile devel-
opment and testing process is nothing without the acceptance and consistent use by
a community of testers within an organization. Technical issues aside, the introduc-
tion and adoption of your agile testing process is likely to be the most dif¬cult step
in delivering effective and ef¬cient agile testing within your organization.
In some organizations, the roll-out and adoption of agile best practices may not
be an issue. For example, if you work in an enlightened company that is already
familiar with the bene¬ts agile can provide, it is likely that agile practices are already
in the process of being used. Similarly, if your company already has a pressing need
to adopt an agile means of working “ perhaps to satisfy the terms and conditions of
another organization that has asked them to bid for some development work, and
who are looking for responses from companies that will employ agile practices “ you
may not need to evangelize about agile.
If none of the aforementioned applies to your organization, the following sections
describe a tried and tested strategy for the introduction, adoption, and ongoing use
of your agile process. This chapter also addresses the issues associated with the
subsequent maintenance and update of your testing process.
The strategy is structured under the following phases with their associated tasks:

Pre-agile Process Roll-out Phase:
Establish the need for agile process.
Establish the likely return on investment of the proposed agile test process.
Gain management commitment and approval.
Gain agreement to run an agile proof of concept (POC)/pilot project.
Agile Process Roll-out Phase:
Baseline current development and testing practices.
Resource the POC/pilot project (staff, equipment, software, rooms, etc.).
Provide training, mentoring, and process guidance.


251
Agile Testing: How to Succeed in an Extreme Testing Environment
252


Execute agile POC/pilot project.
Review results of POC/pilot project.
Post-Agile Roll-out Phases:
Report on success of POC/pilot project.
Monitor and improve agile test process.
Advertise results.
Continue to evangelize.



26.2 Roll-out and Adoption
The phases and their associated tasks involved in the roll-out and adoption of your
agile process include the following.

Pre-Agile Process Roll-out Phase
During this phase, a number of tasks must be completed in preparation for the
roll-out and use of your agile process. The tasks in this phase include the following:

Establish the need for agile process. To make a major change to the way it
currently develops software, your organization needs a compelling need to act;
for example:
If your organization is currently developing software using traditional tech-
niques, have there been issues with projects missing their delivery dates,
with projects completing over budget, and/or with delivery of software that
contains too many defects?
Do your customers complain that the development approach used by your
organization is too rigid, that they do not have the opportunity to in¬‚uence
what gets developed, and that delivered software does not match their needs
or expectations?
Are your competitors adopting agile practices to make their software devel-
opment and delivery process more effective and ef¬cient?
Is your organization under signi¬cant commercial pressures to deliver soft-
ware faster, with less effort and cost, and with higher quality?

If one or more of these compelling needs to act apply to your organization, you
can use this information to make a compelling business case for agile to your own
senior management.

Establish the likely return on investment of the proposed agile test process.
Before approaching management with your proposal for the adoption of your
agile process, spend some time collecting information that you will need to make
a compelling business case. Consider the time and effort involved in rolling out
and adopting an agile process and the likely returns to your company compared
with continuing to use the current approach to developing and testing software.
The Roll-out and Adoption of My Agile Process
253


Gain management commitment and approval. In making a business case to
management for the adoption and use of your agile process, there are a number
of things you might do:
Identify an agile champion “ this should be someone in your organization who
can communicate at all levels in the company (from management to practi-
tioner level), who is enthusiastic about agile, and who has the communication
skills to make the case for agile.
Consider delivering internal agile educational/lunch and learn/workshop/
external speaker sessions to your colleagues and to management to raise
the pro¬le of agile methods and their potential bene¬ts to your organi-
zation.
Highlight the compelling need(s) to act, which were identi¬ed earlier, to
appropriate managers (perhaps making use of your agile champion) and dis-
cuss the effort and likely bene¬ts of rolling out your agile process.
Look to gain (and document) management support for an agile POC or pilot
project to demonstrate the quantitative bene¬ts of your agile process. Ensure
that management support for the POC/pilot includes adequate time, budget,
and resources to complete it successfully.


Agile Process Roll-out Phase
During this phase, a number of tasks are performed in preparation for the roll-out
of your agile process. The tasks in this phase include the following:
Baseline current development and testing practices. If you are to convince man-
agement of the bene¬ts of adopting an agile approach, you will need to be able
to demonstrate quantitatively that agile will save time, effort, and money. To do
this, you need to obtain data on how long it typically takes your organization
to write software, how much effort is expended, and what the overall cost is.
It may also be necessary to show improvements in software quality, in which
case you may need to be able to obtain data on defect detection rates as well.
If your organization already collects these metrics, that will make the process
easier. If not, you may have to rely on estimates obtained in discussion with your
colleagues.
Resource the POC/pilot project. Ensure you give the agile pilot project the best
chance for success; using your documented management approval, recruit the
best staff you can, ensure you have dedicated hardware and software, and appro-
priate rooms to ensure co-location of the team.
Provide training, mentoring, and process guidance. Where necessary, make sure
that training and mentoring are available to ensure the team members have
the right skills and knowledge to be successful, and make agile process guidance
available to the team (this could be as simple as the ready reference cards described
in Chapter 5).
Agile Testing: How to Succeed in an Extreme Testing Environment
254


Execute agile POC/pilot project. In setting up and running the POC/pilot project
you need to consider the following:
Select an appropriate project. Ensure you pick a project that is appropriate to
an agile approach and that is achievable; you must ensure that you show off
your use of an agile approach to its best advantage. Don™t be overcon¬dent
and bite off more than you can chew on the POC.
Underpromise and overachieve. Look for early wins to increase team morale
and con¬dence, try to deliver working software as quickly as possible, involve
the agile champion, and demonstrate early progress to your management
sponsor and customer.
Collect accurate metrics. At the end of the POC, enthusiastic qualitative
claims about the success of the project are unlikely to in¬‚uence hard-nosed
managers. However, if you have collected metrics that show quantitative
improvements over the incumbent development process, this can sway even
the most stoic of accountants.
Collect useful collateral. Be receptive to, and make notes on, particular high-
lights from the POC, particularly successful events, or memorable quotes
from team members, management, or the customer; these can be reused
after the pilot to help promote the success of the project.
Review results of POC/pilot project. Take some time at the end of the POC to
review the result. Get together with the team and the agile champion to discuss
the highlights of the project, to review project metrics, and compare these with
the baseline ¬gures for the existing development and testing approach. Begin
to document the results and consider how to report this information to the
interested parties.

Post-Agile Roll-out Phases
Report on success of POC/pilot project. After analyzing the results of the POC,
work with your agile champion to determine the best means of reporting the
success of the project. The deliverables from this task could include the following:
A management summary document that highlights the key results of the
project. This summary will save senior managers the effort of working their
way through a larger and more comprehensive report document (which may
make the difference in some cases between the results getting read or not).
A full report that formally documents the results of the project in detail,
and which could contain an analysis of the project metrics as well as formal
return on investment calculations. This document can be used by anyone
in the organization who needs a deeper understanding of the results of the
project.
A formal presentation that highlights the results of the POC, and which can
be delivered to senior management to provide them with a succinct brie¬ng
on the results of the project. This presentation could be used as the vehicle for
the delivery of the management summary and/or full report to management.
The Roll-out and Adoption of My Agile Process
255


An informal presentation (perhaps reusing a customized version of the formal
presentation) on the highlights and results of the project, that can be delivered
to interested colleagues and other practitioners.

Following a successful pilot project or POC, seek management approval to intro-
duce your agile process on other projects, perhaps as a follow-on from the POC or
on new projects that are in the pipeline.

Monitor and improve your agile test process. In the best traditions of agile
projects, continue to improve your agile process. Learn from projects that were
particularly successful, or from agile projects that didn™t do so well, and document
the lessons learned for use on future projects. Investigate how the process might
be customized for different-size or special needs projects (such as offsite and
offshore projects).
Advertise results. Nothing succeeds like success; ensure agile win-stories are pub-
licized to management, colleagues, and other practitioners in the organization:
Consider using “lunch and learn” sessions, email newsletters, and/or items
on the company intranet to promote agile successes.
Continue to make use of your agile champion to promote the case for agile
throughout the organization.
Consider passing on your agile best practice knowledge to others in the
industry by writing technical papers and looking for opportunities to speak
about your agile experiences.
Continue to evangelize. Don™t make the assumption that agile will simply take
off on its own; it may, but continued agile evangelism by you, your colleagues,
and your agile champion will all help to promote the cause of agile develop-
ment and testing. Consider setting up an agile special interest group within the
organization, investigate holding monthly meetings (and ensure management
representatives are invited), and look to invite interesting external speakers. The
key thing to remember is to have stamina!


26.3 Maintenance of Your Agile Process
Almost as soon as it has been rolled out and adopted, your agile process is likely to
be out of date; implementation technology changes at a very rapid pace, new agile
practices continue to be developed and re¬ned, and new and improved tools become
available to be used on agile projects. At the very least, the process will need to
change to take account of lessons learned during its use.
You should establish some key points where your agile process should be reviewed
and proposals for changes discussed. These could include the following:

At the end of individual projects to identify any lessons learned during the project.
If agile retrospectives are a feature of your agile process, de¬ne a formal mech-
anism for documenting the lessons learned from all agile projects, and put a
Agile Testing: How to Succeed in an Extreme Testing Environment
256


change management process in place to process proposals for change to the
process. Consider the value of using tools to support the process of accepting
requests for change and for documenting and maintaining the requirements
(ensure these are distributed products if you have a number of sites using your
agile method).
Following changes in the wider agile community, such as the availability of new
or re¬ned agile best practices. You should be receptive to such changes and new
agile practices by being aware of the agile literature, registering on agile special
interest Web sites and visiting agile blogs, and attending appropriate special
interest group meetings.
As a prelude to the adoption/introduction of new or enhanced testing tools or
techniques.
Following changes in the management approach of the organization, such as the
adoption of a new project management methodology, or some formal process
improvement scheme, that impacts your agile process.
At regular dates, such as an annual or biannual review.
As and when requested by the company quality assurance manager (or equi-
valent).

Following such a review and where changes need to be made to the process, the
following actions should be performed:

Update whatever mechanism you have selected to distribute your agile process to
your colleagues (ensuring that any changes do not adversely impact agile projects
that are still active).
Actively inform key stakeholders in your company that changes have been made
and tell them where they can ¬nd more information.
If the change is a major one, consider the need for training and mentoring to
make sure that your colleagues are able to employ the changes correctly and
bene¬t from them.
Review the success of the change at some point following its introduction and
after suf¬cient time has elapsed to allow an accurate appraisal to be made of its
bene¬ts.


26.4 Summary
As fantastic and as valuable as you believe it to be, your agile process will not roll
itself out to your colleagues and your organization.
If you want your process to be anything more than just an academic exercise in
assembling a collection of agile best practices, then you need to ¬nd some means
of getting your colleagues and your organization to use them in earnest on projects
that have some value to your company.
This chapter has described an effective approach that you can use to drive the
successful roll-out and adoption of your agile process and to continue to manage its
The Roll-out and Adoption of My Agile Process
257


ongoing success. Use it to spread the agile word and to gain some agile brownie
points in your organization.
And as a ¬nal thought: as Jon Tilt and David Evans say in their agile case studies “
it™s okay to be agile about the roll-out and adoption of your agile process in your
organization.
APPENDIX A




The Principles of Rapid Application Development
In certain situations, a usable 80% solution can be produced in 20% of the
time that would have been required to produce a total solution.
Walter Maner




A.1 Introduction
This appendix reviews the essential rules and practices of the Rapid Application
Development (RAD) agile method, focusing in more detail on the testing aspects of
the method.
The guiding principles of RAD are to deliver usable high-quality systems quickly
and with low costs. The RAD rules and practices are discussed under the following
sections:

Overview of the Characteristics of RAD Projects,
Joint Application Design,
Rapidity of Development, and
Incremental Prototyping.


A.2 Overview of the Characteristics of RAD Projects
RAD seeks to avoid the traditional problems associated with large, monolithic
waterfall-style projects by breaking the organization and management of the project
tasks into smaller, more easily managed increments.
Historically, RAD projects have been most successful when employed to develop
relatively small and simple applications, with low complexity and risk. For larger,
more complex applications, such as the implementation of a company-wide customer
relationship management (CRM) system, for example, organizations typically looked
to more mature software project management and development approaches (such
as PRINCE [22]). A number of organizations have had success combining both
approaches, with PRINCE used for the overall management of the project and RAD
employed to develop the comparatively simple and low-complexity applications that
run against (using our preceding example) the CRM system.
RAD appears to be particularly well suited to projects where the user community
and its use of the delivered software is well de¬ned and highly interactive. Within
such projects, the key RAD technique of rapid prototyping can be used to produce

259
Appendix A. The Principles of Rapid Application Development
260


simpli¬ed models of elements of the system to obtain feedback from the users and
to re¬ne the developers™ understanding of the requirements of the ¬nal system.
RAD projects commonly fall into one of two types; the intensive and the phased
RAD project. In an intensive RAD project, a combined team of users and developers
work together:

for relatively short periods (perhaps a couple of weeks, for example),
in a controlled environment where external distractions are kept to a minimum
(a so-called clean-room environment; see Section A.3), and
to deliver a complete working element of the ¬nal system.

In a phased RAD project,

the timescales are slightly more relaxed, and the project can be of several months™
duration;
there is less emphasis on continuous co-location between users and developers,
with requirements and design information being elicited and re¬ned through
intensive workshops involving users and developers (so-called joint application
development workshops; see Section A.3); and
the project is planned and managed in terms of the development, demonstration
(to the users), and re¬nement of a series of incremental prototypes, which build
toward the ¬nal deliverable.


A.3 Joint Application Design
The principles of Joint Application Design (JAD) include

Dedicated teams “ Developers and users work together in small teams (typically
four to eight people) that are empowered to make design decisions. The most
successful teams are made up of members with good communications skills, com-
bined with speci¬c domain knowledge; users with detailed business knowledge;
and experienced developers with appropriate techniques and tools skills.
JAD workshops “ A series of JAD workshops are used throughout the development
process to elicit and re¬ne requirements and design details. These workshops are
attended by customer representatives (including key users); developer represen-
tatives (with good requirements elicitation skills); plus a secretary or “scribe,”

<<

. 8
( 10)



>>