The Testing Blog

Hyperbole in Testing! (Shame on us)

Posted on December 1, 2010. Filed under: Uncategorized |

It’s funny how it’s always easier to point out the speck in someone else’s eye and ignore the giant plank in your own as a great 1st century religious figure once pointed out.  (I’m talking about Jesus!!  Matthew 7:3.  It’s Christmas time!!)

And no other group on the whole face of the earth is better at pointing out flaws than software testers.  But when does testing go too far?  When are our scenarios become too outlandish to reasonably fit into reality?

Here’s an actual scenario:

For the past couple weeks, we’ve been trying to get some new hire testers up to speed on our product.  They are zealous and making good progress.  One of these testers found an address control in our UI that’s doing no validation between the City, State, and Zip.  i.e you can put in “CA” for the state and “10100″ for the zip.  Clearly without looking, you can tell that a California zip would have a lot higher number than that.  And if you match it up for the city, you can be even more specific.  Good find.  It’s perfectly reasonable to assume that those values should be validated.  That’s a very basic thing now.  There are services out there to assist with that.  Not hard to develop; we did lots of that kind of stuff at Edfinancial.  The tester submitted the “Bug” to development, and they denied it on the grounds that it works as coded and no requirements to validate the City, State, and Zip.  That’s a reasonable response too.  Though it’s a defect that might cause a problem given a certain data set, it’s really management’s call on whether it needs to be fixed.

This zealous new tester decided to write a response to development, arguing that they must fix it because it could put our company in legal troubles, violating compliance and privacy agreements.  Though my senior in age and experience, he wisely asked me advice on whether to send it or not.  Given my knowledge of the culture,  I told him there’s no way in Hell that I would commit that response to the ticket.  He did his job.  He reported the defect, explaining the scenario nicely.  End of story.  Just drop it.  He countered with other passionate arguments that the mail from us might end up in a competitor’s hand, given a Zip Code mix up, and it might cost our company millions of dollars to straighten out the problem.  I told him that his arguments weren’t really sound because the probability of that actually happening is slim to none.  Out of the millions of addresses in the US, what is the chance that the letter, because of a bad Zip Code, could end up in a competitor’s hand?  Even if it did, what are the chances that the competitor would open mail not directed to them?  Even if they did, what kind of ruthless havoc could they inflict on our company?  They certainly wouldn’t want to advertise that they broke federal law by opening someone else’s mail.  In my mind, it’s a moot point.  Maybe it could happen, but there’s no realistic chance.

Epiphany:

Then it hit me!  As developers underplay the chances of their bugs causing a problem, we testers often overplay the effects of leaving bugs in! We traffic in hyperbole as much if not more than developers.  We predict doom and gloom when the truth is everything will probably be fine.  So, why do we do it?

• The Business Loves Emergencies: Maybe we believe that if we don’t paint a picture of Armageddon, it won’t ever get fixed.  At my company, there’s nothing like a good fire to get the coders up and running, especially when it happens at odd hours!
• Hedging Against Disaster: Maybe we can’t represent the bug in proper context because we’re afraid that it might be bigger than we expected.  We say it’s no problem, then it’s the BP oil spill.  It’s better to be on the “I told you so” end of a disaster.
• Self-Importance: Maybe we have to make the bugs look bigger so that we can feel better about the contribution we’re making.  Maybe we’re a bunch of ego maniacs starved for accolades.

Final Thought:

As a tester, credibility is king.  If you’re a credible, the business will heed just your word.  If you’re a chicken little tester, clucking at every little thing, they will take everything you say with a grain of salt.  Be honest as you can with yourself.  ”To thine own self be true, ” as The Bard said.  Don’t focus on the trivial.  Focus on the big picture.  Focus on testing!

Postscript:

Here’s something interesting.  According to one of our subject matter experts, the suspect address control isn’t being used by ANY of our current customers.  So, the lack of validation is all really “Much ado about nothing.”  Two Shakespeare references in one post!  Score!

Make a Comment

3 Responses to “Hyperbole in Testing! (Shame on us)”

Comments RSS Feed

Where's The Comment Form?