« Other Nice Spots on the Microsoft Campus | Main | The Legacy of Y2K »
January 05, 2005
Specing Errors
One of the things I do as a program manager is write specifications, or specs.Everyone knows the order you are supposed to do this in: first define the requirements, then write a functional spec, which eventually results in a developer writing a design spec, and a tester writing a test spec. The code and tests are written, and we are off to the races.
The main thing is to make sure you pause at each step. Have all your requirements before you write the functional spec, and finish the functional spec before anyone writes a design/test spec (or god forbid starts writing code).
This works well in general. It is good to really think through all your scenarios and address them in a functional spec before you start designing the software.
Except for one thing. Errors.
Every spec has a section for errors (and exceptions), and in most specs this is marked as "TBD" in the first draft. Then eventually someone does a bit of thinking about the ways that the program could fail, and fills in some stuff. The user-visible error messages are defined for each error. But then the code gets written, and the errors may or may not match what was speced.People sometimes complain about the thinking that has to be done ahead of time when writing a functional spec. Wouldn't it be easier, the argument goes, to just write the code and then spec it afterwards (like in the "good old days")? But these days there is too much that needs to be known ahead of time. It's not just the developer; the tester has to plan the tests, UA has to write help, etc. So most people who complain about having to think through scenarios for a functional spec should just deal with it. But errors, I think, is the one area where this reasoning is correct. It is inefficient to spend the time to spec errors up front.
The problem with specing the errors beforehand are:
- You may not think of every possible error, so that the developer discovers an un-speced error condition.
- You may think of errors that can't happen, meaning that you spec what to do when something fails in a certain way and then discover that your code can't distinguish that error from another one.
- You may spec discrete failures as the same user-visible error, which means you are hiding information that could be valuable when debugging the program later.
Technically, in all these cases the developer can't just do "the right thing", because the functional spec doesn't say to do that. So you have to go back and update the spec, which takes time, on top of the time you already spent thinking about errors. And maybe updating the spec requires a DCR or signoff or one of those things.
So what I propose is that errors be treated separately from the rest of the functional spec. What you do is write the functional spec without the errors. Then the developer writes the code and sees the actual errors that can be generated, making sure that each one is uniquely identifiable from the user perspective (even if it is just a unique number that is printed out, that the user can read over the phone to a tech support person). Then the functional spec is updated to match what the code does, the error message text is written and added to the code (hopefully it is in a resource file, not compiled in, so easy for a non-programmer to edit), and we are done.
The result is you get all your errors documented as before, they are all unique, and you don't waste any unncessary time.
I doubt any managers at Microsoft would agree, but it's nice to dream.
Posted by AdamBa at January 5, 2005 08:27 PM
Trackback Pings
TrackBack URL for this entry:
http://proudlyserving.com/cgi-bin/mt-tb.cgi/129
Comments
Generally I agree, except in the instance (which im currently in) is when the feature IS error handling and how errors are thrown :)
Posted by: Steve at January 5, 2005 09:39 PM
Yes, I should have mentioned that (and was going to, but forgot). You should define your overall error ARCHITECTURE and all that ahead of time -- it's the specific errors that can wait.
- adam
Posted by: Adam Barr at January 5, 2005 09:55 PM
I disagree. Here's the reason why:
People sometimes complain about the thinking that has to be done ahead of time when writing a functional spec including errors. Wouldn't it be easier, the argument goes, to just leave the errors out of the spec, write the code, and then spec it afterwards (like in the "good old days")? But these days there is too much that needs to be known ahead of time. It's not just the developer; the tester has to plan the tests, UA has to write help, etc. So most people who complain about having to think through error handling scenarios for a functional spec should just deal with it.
With the list of reasons for not including specs that you have I can do the exact same thing I just did - except switch errors to APIs (you may not think of every necessary bit flag or option, you may think of APIs that are unnecssary, you may overload APIs in a way that doesn't allow the end user to build on them the way they need to).
The truth is that development is an iterative process. You write the spec, the developer uses the spec to write the code, the tester uses the spec to write the tests, you find places where things aren't write and you revise the code, specs, and tests.
You're just picking one arbitrary area and throwing it out as being less important. While this is the case for many classes of software for others it is extremely important. Errors are things that have to be tested for. And generally they're the HARDEST thing to test for. Not only do they need to be tested for, the program also needs to remain consistent in the face of errors, so there's also an unplanned dev cost hiding here.
Writing solid software is very much requires getting things right when everything goes wrong. And if you leave this portion of your product until the end you're merely backloading work that you'll need to do later. And what happens to that work during crunch time? It doesn't get done.
(what, I'm not allowed to use paragraphs? If you like paragraphs view->source, it looks like they're preserved - at least in preview <p> and <br> don't even work)
Posted by: Anonymous at January 28, 2005 08:51 PM
