January 31st, 2014 by Adam Sandman
Boil
the ocean
eat
the elephant
climb
the mountain
and
now - Automate the tests for the application!!
All
things we must do bit by bit, bite by bite, step by step, and case by case.
The
task looks like it is just too huge. The average test plan for a
commercial grade application will have between 2,000 and 10,000 test cases. You have a team of five. Let’s
see, that works out to a thousand a piece or more... Sure, we will be done by next Thursday!
This
is the quandary of an applications development shop and test group, thinking
about automation for the first time, or even a few more iterations down the
road than that.
Let’s
lay out some of the issues:
·
Our
team knows the application and can run through it in a few days to see if there
are any defects.
·
The
automation tools can’t be as smart as my team.
·
The
automation tools cannot deal with the complexity of my application
·
We
failed at automation before
·
We
cannot afford failure.
All
of this is about setting expectations. I will say it - automation cannot do it all.
Heresy, from an automation expert. Yes I said it and I believe it.
You should not automate everything. I will also say that humans are
smarter than machines (forgive me o robotic overlords).
Let’s
set the expectations at a reasonable level. Oh say 20% automation.
Too small the managers yell! Well let’s quiet them by describing
what 20% means.
20%
means 20% of your test cases. which 20% you may ask? how about the
20% that are used most often, have the most impact, and that chew up around 70%
of the test teams time. Yeah, that 20%.
These
are the cases that you spend hours doing, every day, every release, every
build. these are the cases that you dread, it is like slamming your head
into a brick wall, the outcome never seems to change, it is monotonous, it is
boring, but, yes, it is necessary. it is necessary because most of your
clients use these cases as paths to get their task done, and therefore they are
the tasks that pay the company, and the test team, to exist. they are
boring, but important.
I
would like to refine the 20%. and redefine it. 20% represents what
in a 40 hour workweek (who has those anymore)? it is 8 hours; 8
hours is a full day; at the average testers salary of $60k, it is $250
per week, or $13,000 per year. if using an automation tool could save
just $13,000 per year, per seat, for a team of 5, that is $65,000. More that the salary for 1 additional
staff.
This
doesn't mean we need one less person on staff, this means that we now
have the equivalent of one extra staff member to test deeper, and more often,
those things that cannot be tested with automation.
So,
even if your application is very complex, even if you have failed before, you
cannot fail. You cannot fail because you set the goal as achievable; it
is one bite of the elephant; it is one pail of the ocean. it is a first step up
the mountain; it is an achievable goal!
Imagine what happens
if you hit 30%.