Wednesday, March 11, 2009

Automation is not easy, nor should it be

If you've ever been in an organization that starts to get some process around itself you will discover in testing that someone, at some point, wrote a script.  That script was the beginning of other scripts, which eventually became under the control of maybe a master script or were all bundled into a suite.  When that bundle, or suite, was run time and time again and the results were known and used, you suddenly have the good beginnings of a Test Harness.

It can take time to develop, but that's what those scripts are, development.  Script Development, with all the process and care that goes into the code that comes under test, so to should those scripts being written be vetted and reviewed by someone else.  It's not "just for QA" when the scripts are doing the work and validating someone elses code, this is something that has become a tool which is being used for a purpose, that purpose is validation.  When you validate something there needs to be a story to back it up, someone has to be able to stand up in front of the crowd and proudly say "this is my script and here is what it does!".  This invites all kinds of questions and discussion and debate, it does happen, but you know this is not a bad thing.  Discussion and debate is good for not just the Requirements and the Product, but for those little things that help the product get out the door, because that script is code and code needs reviewing, and commenting, and that is work that needs to be done.

Automating work takes time, dedication, patience and practice.  It's not easy to generate a framework that will allow you to run tests, especially a framework that you can run and say that when a test case fails its because the product has a problem and not the automation.  Automation is hard work because you need to develop something that will aid you in your goal of running tests and validating the product, but its also a project on its own and needs its own schedule, set of tasks, care and review.  Just going out and writing a script is fine, if all you want or need is a script that does X and never Y, but sometimes there is a point where there is a need to do X, Y, Z and all the other letters before them.  It's not easy, and it should not be because a badly written and implemented Automation system does no one any good, but it doesn't all need to be out at once either, it can be done in stages, just like a real project because after all that is what it is.

I've done this already, and this is the background for what I have here.  When I came to my current place of employment there were a lot of scripts written, there was even a beginning stab at automation, done by someone else who was here.  We worked together and extended the automation scripts and built a nice little test harness, it didn't test everything, but it tested a good piece of functionality consistently and routinely and gave results.  So we knew where things were and we had discussions about where we wanted the automation to go, saw what was lacking and talked about how to fill in the gaps and brought Developers into the discussion to get their feedback.  The other person left and I took over, I kept up with what we had discussed but now that I was driving I was talking about the automation but fewer and fewer people were talking back, eventually it was just me.  Then the product took a new direction and I said to myself "hey me!  Let's get back on board and do this right and fill in those gaps!"  Because I had been patching the system we had for awhile, working around it when the scripts we had were breaking because they were starting to be used in ways we never intended, in between testing and writing the automation the automation became a part time job.  During the break while the new direction was being created, I came up with a full blown plan on what we needed, how we were going to get there, brought people into the conversation who were excited about it and we made a staged release plan.

Now we are building some cool stuff, people are talking about it, other groups in the company find it cool and think its a nice idea, and they want to talk about it, and I am feeling reenergized in what I am doing because we will have some really nice stuff when this is all done.

About a year from now.

No comments: