What is it You are Testing?

Months ago we had a discussion between the Dev and QA teams as to what was going on with our project and make some plans going forward, strangely even though we talk and documentation is given for review a comment came up in the discussion that said "I don't know what QA is testing?"

To me and the Manager it was a WTH (what the hell??) moment.  We had never heard this before, and even when we discuss bugs and defect fixes with the Developers this never came up, neither did this when we would go over reproducing bugs.  To me, I was testing and they were seeing results, but maybe someone was just not seeing things the same way I was.  We got over that, went into a project and did a lot of work over the past few months, adding some tests, discussing a few and verifying many fixes.  Yet, from those earlier discussions a document was made that still had that comment in it, so now that we are between versions I am going back to this and starting to think out, what is it that someone was missing and we never picked up on.

One of the projects for the next release is to update our test harness, adding some wrappers to it so that the scripts are more integrated and a database to store results, yes I know we still record some results in Excel and if not for that - not much historical tracking.  Well, that and the output documents I have from my tests, that I save for each release and archive away in case we ever need to go through old data.  I am also taking time to go to the Developers and show them what we are doing, get some input and putting the question to them "what should we be testing?".

Getting people involved is a good way to eliminate any confusion, since a follow-up to that is going to be "what would you like to test with this?" and offer our Test Harness to those who really do not have one.  Some parts of our product don't have many Unit Tests and its a shame, or they have them and they are kept secret, that's a future task  I'm not going to be giving them a fish, I intend to give them a rod, boat, ocean of fish and pilot and let them see what they can catch.