You're never too old to Mentor

One thing I did in my last position was bring in a new programming language to the group in order to generate a test harness that would help us out with some of the sore points we had, but while I did that I brought other people into the test harness project to help out.  Adding in new people and a language, Ruby in this case, meant that there were going to be things people did not understand, no matter how much I could document in all the scripts we had there were some that had little information in them.  So how to handle this?

Well I could have documented it on a wiki, and I had.  Of course the first place most of the documentation on the scripts should be is in the scripts themselves, but I inherited about 95% of the code that had built up over years and as I said, I tried to document when I could.

I could give people books and web sites to learn what they needed to know, this was really done without my interaction as everyone took it upon themselves to learn what they needed and I also made my library open.

The third way I saw was to help people learn, so I needed to mentor people.  I took time on occasion to show people how to write certain scripts, I did monthly talks about some parts of coding (but that was already a long standing practice in this company).  Sitting down and taking time to go over small scripts with my co-workers, I sometimes gave them easy projects to do to get them used to the coding, giving advice and answering questions about how things happened.  I learned a few things in the process as well, which is always a bonus, but it worked out well for my colleagues who were able to come up to speed quick and make an impact to the coding needed to transition to a new test harness.

So why are you never too old?  Because the people joining me on the project were Senior and Principal level QA staff who were coming from testing the GUI to writing scripts that they could use to make their own testing better.  None of us were Junior level people, or interns, so no matter what level you may be there is always something new to learn.

Entering Defect Reports

Its something we do all the time, and I've been doing it in various incarnations of software for years now, but I rarely just give detailed thought to it until I went into an interview recently and was asked - "What do you put into a Defect Report?"

At first I was thinking, sure I know this and could easily list off many items that should be in there, but then I started wondering what is the purpose of some of these things I do and write down.  Sure I do them because I know it gets the right detail on the problems listed, it also makes it easy for Developers or anyone else to read the reports, I have been doing them for a long time, and I have evolved certain things over time.  So in order to put it down, and to have some place to also answer people who seem to ask this time and time again in the forums I give you my List Of Things To Be Included In A Defect Report.

1. Title - the title should be 20 or 25 words, in some cases its about the length of the title field in certain software, but its also good to be concise.  Make the title useful in case you are searching for defects and the title is a field that is returned, you want to be able to find old defects easy and know by the title what they are about.  It's also useful to other people who may be searching old or open defects, if they can tell what the defect is for then so much the better, it avoids much opening of defects to review.  Usually I put in error messages, if short, or a small description of what the error is.
2. Description - this can be a lengthy overview of what the issue is, sometimes its useful for detailed defects so there is something to check over before getting into the nitty gritty, but the description also contains many other things which shall be noted shortly.
3. Steps to Reproduce - this is vital.  If you want someone to be able to fix the defect they may need to reproduce it, without knowing how you'll get visits by people who want to know what you did.  Or it may just be closed as unreproducible, or sent back to you for more information.  Make sure you run through the steps a few times, make sure nothing is missed and that other people can do it on the same machine, or a different one if that is your company's environment.  This should be included in the description.
4. External Files - screenshots, core files, dumps or whatever else may help someone triage the issue should be included in the report, if possible, or note a location on a publicly accessible area, or at least an area accessible by your co-workers, where the files can be found.  These are necessary to determine where issues have come from, or may be needed for someone to see the error messages, of UI issues, you report.  This should be included in the description if not attached to the report, its often nice to note this information is attached.
5. Platform, machine type, operating system - anything else specific about where the issue occurred.  Many defect systems have this as pull down information available when entering the defect, if not put this in the description, and sometimes its useful in the Title as well.
6. Why its not a duplicate - if this is similar to an existing issue but not the same, note why in the Description.  If its related to an existing one, make sure to link the new one to the exisitng defect so there is a trail for people to follow, many defect systems allow this otherwise add it to the Description.

There is usually more, but the core information is what is really need to make the report complete, easy to understand and most importantly searchable within the defect system.  Searchability is a very important part that gets overlooked, if the issue ever comes up again you want to make sure the information is easy to find so you avoid duplicates, and most importantly if its been fixed you can check and see if the fix is still valid, in which case you have something entirely new.  Probably.

Review: How We Test Software at Microsoft

Although I will never work at microsoft, although definitely not at the main campus since I have no intention of moving off the East Coast of the US, it was interesting to get a glimpse at how the company looks at Testing and Testers.  Considering that they are often used as an example of how not to make software and release, considering the bugginess of some of their releases, I found the evolution of the QA positions in the company to be illuminating.  I never thought they put this much thought into it.

The examples were nice, sort of like Lessons Learned From Software Testing, another of my favorites, and in some cases were pretty funny.  Seeing how software is made behind the scenes is not as messy as sausages, but it can be as interesting.

One of my favorite chapters was Chapter 5 - Functional Testing Techniques, not just because I saw Bj Rollison give part of the chapter as a talk to the STPCon in 2008, but because I like the detail of examples and clear definitions of Equivalence Partitioning and Combinatorial Analysis; subjects I sometimes I forget about.  I'll be rereading this chapter now and again to catch up on those testing techniques, as they are not ones I have done a lot of during system testing in the past.

It's not often I find a test book that gives me material I can look at now and in the future, this book is one that has found a fast place on my bookshelf at work.