theologians, nothing is innocent; if you believe the military, nothing is safe."
Lord Salisbury
To paraphrase, "if you think a software deployment never has problems, nothing is shipped".
Originally I started this post thinking about deployments but with the release of the ACA web site, portal, whatever it really is I saw a very public forum airing something on which I deal with on a consistent basis. Anyone, and I mean ANYONE, who has worked in software has been involved with or seen a botched deployment on code either being shipped to customers, or as is more the case these days, on code released to a live site. Things generally go well, and the scale of problems varies, but it is always there. To mitigate we test, check, use the code in an environment that will simulate the production environment. There are too many test types to mention that can be used, but in any of them actually USING the code or application will show up issues of some kind, at the very least. I will say something that anyone working in tech knows, No Software is Perfect.
Listening to On Point on NPR, I support Public Radio and typically I am agreement with most of what Tom Ashbrook says but this time I sadly think he was out of his depth. Sure it looks bad, but this is a political hot potato and anything that could have gone bad would be a candidate for hyperbole. Was the rollout of the government web site bad? Oh yes, just from the comments I wonder what sort of process they ran it through, but doing this for a living I know how hard it is and what it takes. My older relatives who barely use computers would probably get a different perspective. Yet I also know people who have worked on government projects, even looking back at many of the governments weapon's procurement and development programs, none of this should be a surprise. The US Government still does a large amount of work on a waterfall process where they just start down a road and keep driving, filling up the tank as it gets empty so the work continues and many times no one is navigating, so the project just keeps driving along.
Still, after hashing through many of the reports over the past two weeks here is what I see as the major issues, and yet I see this on many projects, some even today, and no one has died yet.
- No Testing, or whatever kind there was seems to be minimal. If the site was crashing as people said I would have to ask what sort of Load Testing there was. I don't even think I want to bring up Security testing.
- Coding done by Government Contract, seems like a big waterfall of a project with a process to match, seems like there was little Acceptance testing done until the end when it was deployed.
- Outsourcing, not a bad thing, but was the company that really doing this the best choice? From some of what I had seen few traditional software companies wanted in on this, and the company that did do it seems to have a spotty track record at best.
- No Communication or Transparency, this was a big project (I am using that sarcastically since more Web Applications and sites don't have millions of lines of code, but it's big in the scale of visibility) and with the size it seemed much of the status was held close to the leaders. Sure its a political liability, but when you aren't transparent people make up their own rationales.
- Tech Surge, oh yeah we all know throwing more people and money at something gets it done faster! Right?
- Were the people really the "best and brightest"? Or if they are only coming in now why would they want to refactor code? Seems a waste of a skill set to me. And if they are only coming in now then who was working on this?
- Bureaucracy. 'nuff said.
Rocky roll outs are a norm, but maybe this time people who actually run projects, or those on them that they feel are going wrong can point out to the ACA web site rollout and say, "let's not be like them."
Maybe, just maybe, then, someone will listen.
