Scripted testing… There are so many things wrong with it, its hard to know where to begin. Many test experts and prolific bloggers have written about this theme, therefore there is an abundance of literature about topics such as scripted testing vs. exploratory testing out there.
The highest form of ignorance is when you reject something you don’t know anything about. Wayne Dyer
ST is like an old wives’ tale, an urban legend, something that gets passed down from generation of testers to generation, and gets propagated and perpetuated due to the ignorance of testers everywhere. I was once an ignorant tester myself, who like many others learned from another unenlightened tester that the way to test is to write countless numbers of test scripts, with detailed steps, which will be diligently reported upon daily and you will get a gold star if you execute the pre-set quota of test cases for the day.
In this day and age however, there is no excuse to be uninformed about anything, which is why it shocks me how many testers still don’t know there are alternatives to scripted testing, especially here in Sydney.
This week I had coffee with a ‘seasoned’ tester, and the conversation about exploratory testing came up. This tester had heard about it, but like many others, had assumed that it was just ad-hoc-free-for-all testing. So I started asking questions about the validity and pitfalls of scripted testing with them. I find that this topic comes up very often in conversations with other testers, and I feel I’m repeating myself constantly, as I never pass up the opportunity to talk about context-driven testing, exploratory testing, and other fun topics like that!
Like I said in the beginning, there is plenty of information on the web about why scripted testing is not the way to better testing. Below is a list I had on Evernote, which I usually send on an email to testers that are interested in learning more about ET:
- What is Exploratory Testing, and how it differs from Scripted Testing by James Bach – a great place to start and get familiar with the real definition of ET.
- Exploring Exploratory Testing by Cem Kaner and Andy Tinkham – a paper presented at the 2003 StarWest conference, which goes into the detail about what is ET, ET strategies, the role of heuristics and how to train explorers.
- What Exploratory Testing Is Not by Michael Bolton – a fantastic five part blog series which touches on the major fallacies about ET. It is not touring, it is not after-everything-else-testing, it is not tool-free testing, it is not quick testing and it is (most definitely) not undocumented testing.
- Scripted Testing is Not Better for Novice Testers by James Bach – this is an argument I hear again and again, that scripted testing is good for training new testers. No its not!
- Exploratory Software Testing by James Whitaker – a classic book about ET
- Explore It! by Elisabeth Hendrickson – a brand new book about ET by an experienced practitioner I’m still reading it and will post a review soon.
Every time I have an opportunity, I attempt to stop ignorance in testing. I chat and challenge testers that are still on the dark side. The list proposed above is just a starting point for testers interested in learning the craft of testing, for those looking for alternatives to what they know, deep down is fake testing!
Road Less Tested 
I realize that this post is somewhat older, yet I feel inclined to comment about it. I am a software professional, meaning I test software, and I perform a myriad of other tasks relating to software, such as programming, designing, writing about it, etc. I think that your comments about scripted testing are provocative, and they make one ponder. And as such I have pondered, and have come to a similar, though disconnected conclusion: scripted testing can keep a software tester from digging in deeply into the software they are testing (especially if there are qutoas of any sort involved). There is a thought that new eyes bring a new perspective to how to test a software product, and thus expose more and different defects. And if scripting is playing center stage for new software testers, then it is probably that we will quickly numb those new eyes to anything that they might otherwise have found. Having said that, I think that scripted testing probably has its place, and should not be completely avoided. It’s like a naiive developer saying never use Java, always use C++ — in the end there has to be some personal preference combined with what makes sense to use. If scripted testing is all that you use, then you should get outside of your comfort zone and try something new, but for regression testing, scripted testing might just be the type of testing to ensure that nothing has regressed since your last deployment.
Hi Nathan
Thank you for your comments. I’m glad the post made you ponder, its great when we stop to reflect.
You say: “scripted testing can keep a software tester from digging in deeply into the software they are testing”, which is very true, we tend to switch our brains off when there are detailed tests to be followed (which is the whole debate about checking vs. testing). However there are other problems with highly scripted testing such as inattentional blindness and maintenance.
And lets not forget that there are other ways to catalog and report our test cases – other than scripting them all. Testers would do well to investigate other options out there. Often, when they do, they never go back to scripting again.
As Nathan, I feel forced to comment ignoring the timeline…
I agree to stop with ignorance testing. I always say that a monkey can do clicks on a page following a script. I prefer let those clicks and assertions to automation and let machines do this job!
I believe in let our brains free to really get deeper on situations that we never think when writing these scripts.
But I don’t believe in one living without other… A exploratory tester will never do the same again, and regression tests will fail. A exploratory tester have to always update the scripted test (for automation, ok?)