Testing With Sun Tzu - Chapter 3

We come to Chapter 3 Attack By Stratagem, I like that translation.  Maybe one day my Chinese will get good enough so I can read the original, but that’s a long time coming, I have enough trouble reading printed characters never mind calligraphy.  I may wander my thoughts a bit here, my apologies, maybe I’ll clean it up later.  While QA doesn't really need to attack, its usually good diplomacy to not go on the offensive in a project, remember War is the last resort to Diplomacy so always try to work things out before going on the offensive.

First in this chapter comes some very good advice “the best thing of all is to take the enemy's country whole and intact; to shatter and destroy it is not so good.”  So much for a scorched earth policy.  But the important thing is to take from the enemy what was theirs and subsist on it.  Not that Development should ever be considered the enemy, but going to Dev-Land and seeing their resources, and what could be useful in testing, is a good strategy.  In some respects consider the area of a bug that has been fixed has been left alone, we are reclaiming or recycling what the Developer left and used to do his work.  Reusing tools is not only good policy but is good for the environment too!  Just as “it is better to recapture an army entire than to destroy it” you do not want to waste resources of any kind, including intellectual resources, make the Developers you work with your friends, get them on your side and in your army and they will help you against the Bug Enemy who comes to frustrate your efforts to complete your task.

I would like to point out, in this section is one of the lines Bruce Lee utters in Return of the Dragon, “my style is fighting without fighting”, originally written here as “Hence to fight and conquer in all your battles is not supreme excellence; supreme excellence consists in breaking the enemy's resistance without fighting.”  Cool stuff.

Part of the thinking here is to deal with the enemy without engaging forces, in planning “the highest form of generalship is to balk the enemy's plans”.  This can be done early by marshaling your forces and preparing for as many contingencies as possible in the project.  In the footnote to this quote Ho Shih puts this very clearly:  "When the enemy has made a plan of attack against us, we must anticipate him by delivering our own attack first."  Don’t wait for the bugs to appear, anticipate them, prepare for them, plan for them and defeat them first.  Know which areas of code have been rewritten, touched, added, deleted and or modified and focus on them; when I have worked on code common to multiple platforms I focus on that area first, before the individual platforms, because I know the changes affect more areas.  If the code does not work properly on both, that is going to be evident very quickly, if there is a need to delve deeper that can be done individually as well but it will be a judgment call on what is more important, something that comes easier with experience.

To recap some in the original “Therefore the skillful leader subdues the enemy's troops without any fighting; he captures their cities without laying siege to them; he overthrows their kingdom without lengthy operations in the field.”  Do not directly engage the faults of a project, anticipate them, and do so in a way that is economical to you; do not have QA resources testing some area that will not work if it’s known, just to see how much is unaffected.  Do not “siege” bugs by using multiple people to try and approach the bug to see where it might be, and if there is a way to get more data on it (there may be cases you need to do it, but this should be a rarity IMO).  Try and find the issues quickly, simply and once documented move on to the next areas under test, using resources in one area and one area only causes delays the project may not have.  Attack with one or two resources, don't pour everything you have into the "siege", using Pairwise Testing works, but not the whole department.  It may be useful in some cases but not all.

Now this gets me to something new, in laying siege to “faults” there are resources, and these resources are finite (if not, I’d like to work for your company) so just as there is a strategy to use your own personal resources on a bug so to are there resources that need to be effective on a project.  Part of being a Manager, Lead or whatever titles the person in charge has means knowing how to allocate resources, hopefully effectively.  The numbers are Sun Tzu’s not mine, so take them with a grain of salt.

• “It is the rule in war, if our forces are ten to the enemy's one, to surround him”.  If your project is small and you have the resources then do everything you can, attack the project from multiple points at once and test everything by overwhelming it; though I admit this is probably a rarity.
• “..if five to one, to attack him…”  This is getting closer in a ratio of resources to project points, not as many people to work with, but still very manageable.
• “…if twice as numerous, to divide our army into two…”  When dividing resources make sure to do it properly, in this respect one part of an army can meet the enemy head on, the other can be used for special tactics, so many focusing one group on Functional or Regression Testing and the other for specialized testing like Unit Tests, Automation, Stress or Load or whatever else needs to be done.
• “…If equally matched, we can offer battle…”  This is probably the reality for many, where there is enough to get the job done, so plan it out and get it done, you won’t have much to gain by creative allocation of resources, stick with the plan you get from your team and what they can do, reprioritize only when necessary and make those judgment calls.
• “…if quite unequal in every way, we can flee from him…”  Well you can’t really flee, unless you want to quit, but you may want to raise those red flags now on how late things will probably be by the time you are done.

Now there are also some ways to make sure your project can fail, I will only adjust these when necessary and most are pretty understandable as they are.

• “By commanding the army to advance or to retreat, being ignorant of the fact that it cannot obey.  This is called hobbling the army.”  Don’t keep moving resources around to meet day to day issues unless you are set up for it, if you are then you have the planning necessary and you are not reacting to situations, you are dealing with them before they arrive.
• “By attempting to govern an army in the same way as he administers a kingdom, being ignorant of the conditions which obtain in an army.  This causes restlessness in the soldier's minds.”  Don't overstress your resources either, setting impossible deadlines or work on your team, listen to them and know when the grumblings are real and not just a sound off of something else.  If you don't know the mood of your team, you are not in sync with them.
• “By employing the officers of his army without discrimination”.  This is a warning to put the right person on the right job, match skills to the task at hand.

One last thing that is mentioned here and I can bring this to two words “Know Thyself”.  Finishing a task comes down to a couple of things;

• “If you know the enemy and know yourself, you need not fear the result of a hundred battles.”  Know your abilities and what it may take to deal with the bugs you find and you will succeed.
• “If you know yourself but not the enemy, for every victory gained you will also suffer a defeat.”  Knowing your abilities but not what it will take to determine what a bug is and you can not only waste time, but yourself, in trying to determine something without reward.  True there are learning opportunities but these should be done when a schedule allows, when you have 2 days left in a project and you need to learn Java to discern a problem in the application this is not the time to do this yourself.
• “If you know neither the enemy nor yourself, you will succumb in every battle.”  If you are unsure of your abilities and what it takes to determine what the bug is this is what I call being over your head, ask for help or come clean with your manager on what you know or don’t know.  Don’t be the person to bring the project down, that will not earn you any friends.

I do like this quote here as well attributed to Chang Yu:  "Knowing the enemy enables you to take the offensive, knowing yourself enables you to stand on the Defensive... Attack is the secret of defense; defense is the planning of an attack."

Finally, 3 down!

Testing With Sun Tzu - Chapter 2

Now that you have enjoyed Chapter 1 – Laying Plans, we come to Chapter 2 – Waging War; which strangely is not really about war itself, but what it takes to do it.  I may make some allusions to certain current political situations, but if so and they offend you my apologies, I am just trying to draw parallels and any judgments made are my own.

As this part starts out there is a distinct drawing of men per chariot, this is done to give a method by which to structure the army, the structure was determined within Chapter 1.  What this gives is a method by which you know the breakdown, and how it fits into the framework that was determined early on.  In a project you have a strategy and a framework that will be what you work with; then you flesh it out with numbers and actual details, basically going from the general to the specific.  In writing out a general SWAG (simple wild assed guess) on how much time a certain component may take to test, to then determining the steps taken to do the work, and how much time each of those steps will take.  There is also a requirement that the soldiers must have enough provisions to carry them a thousand LI; a LI is generally 2.78 per mile today, so you have to not only make sure of your details but that you also have the stuff to get you there.  Sun Zi went into detail on what it takes to get an army somewhere, including expenditures on the home front, he was nothing if not detailed.

That stuff will vary by culture, whether its coffee, tea, Mountain Dew, Lucozade, chai or whatever Code Monkeys like these days.

Sun Zi also realized that a long term battle can cause equipment failures as well, “if victory is long in coming, then men's weapons will grow dull and their ardor will be damped.”  Just as in a project, the longer it goes on the more tiring it can be and oftentimes the more people will just want it to be over.  I’ve been on long projects, some extending immediately after release to a maintenance mode, right when you also need to be preparing for the next project, and this can be difficult.  Planned right it can go well, but again this is where you either plan properly and you can succeed or you plan badly and you fail, having the milestones in the project to know where you are going, and whether or not you are on your way to victory (a good release) or defeat (you’ll be spending the next few weeks working late nights and weekends).  To stay sharp and on focus with the project means making sure you know your own limits, as well as keep on top of anything that will push you off track, don’t rush through the project but don’t drag it on either, it’s a delicate balance that takes experience to realize and understand. 

 “Again, if the campaign is protracted, the resources of the State will not be equal to the strain.”  Here is one I am going to make a dangerous parallel on.  War is an exhaustion of resources, of all kinds, so you want to use them effectively.  In a project if there are to be Unit Tests and Automated Tests make them effective, not just to have them, but make sure you are going to get use out of the Tests not have them in your bag of tricks and use them as resume fodder.  If there is a GUI that needs tests get it done, no need for fancy stuff just because some new tool came out and it would look cool to know, unless doing that cool thing will also save you time on the project, if you get no efficiency from the work it may not be worth doing.  Now being American I can look at the current situation in Iraq, notwithstanding my own views, taking a look at the timeline there were definitely things that went wrong with planning there.  But just as important, as recent election results have shown, not only was short range planning off, and generated a larger mess than it should have been, but the long term planning in keeping the people who should be supporting this was also lacking.  People need to believe in the project and be on board with all of it, get buy-in, even from those who may not agree with you, the best way to look at a project and a plan is from many different sides, and then make a decision using all the information on hand.  My last comment for now on the whole Iraq situation is “There is no instance of a country having benefited from prolonged warfare.”

There is also a lot in this chapter about foraging in the enemy’s lands, using his equipment captured in battle to augment your own forces and rewarding your own forces with spoils of war.  Hard to make a good parallel here, but I’m going to try anyway, what can be used are tools developed by Developers.  Yes, they actually do test their stuff, well a lot of the ones I know do, and they may have a tool or script that they used to make sure their stuff worked.  Use it.  Keep chatting with them about the work they did and how they did something, it could be useful for you.  At least Sun Zi was reusing chariots, not that I have room in my cube for one, but it would be a heck of a conversation piece.

Just as is said - “In war, then, let your great object be victory, not lengthy campaigns.”  Don’t let the projects extend more than they have to, the reason for making the project plan is to have a way to measure progress, but also to keep the project constrained into a time that will give it time to be completed.  While it’s often been said that “if I just had a little more time to test I could find all the bugs” well in reality that’s just not possible.  Again avoid the length campaign, sure there may be one or two bugs that get left in, that is the unfortunate part of the work that is done by Quality Professionals, what is best to avoid is the serious bug that will completely ruin the experience of all Customers when using the product.  It’s risk mitigation, if a test suite is run on a component 20 times and in the last 5 times no new bugs have been found, with no changes in the code, then its unlikely that running it one more time will raise that bug that will stop the product going out the door.  It may, but it’s like gambling, if you have confidence in your tests and test suites that you know nothing will be found.  Risk Mitigation is an important part of the business, but it also takes a lot of experience in being able to do it properly and efficiently, I know I am still learning how to do it and I’ve released a lot of software.

Just remember that the QA Manager or Project Manager or Lead or whoever is in charge is the one who will eventually make the call.  “Thus it may be known that the leader of armies is the Arbiter of the people's fate, the man on whom it depends whether the nation shall be in peace or in peril.”

2 down, 11 to go!