The accepted, overplayed archetypes present in the Tourist made me think about testing.  Just like the movie clichés, we’re so stupid in freely accepting bullshit testing patterns and practices.  MY definition of a bullshit testing pattern or practice: one that doesn’t work for your situation. Testing methods that might be pure gold to your project might be pure garbage to another.  (We can’t all be Charlie Sheen, converting “tin cans into pure gold”).  I’ve heard it said that there are no best practices in testing, only good ones.  The worst thing you can do as a QA group is do work that doesn’t somehow help the quality of your software.  When doing anything in testing, you need to ask yourself, what’s my return on investment (ROI)?  Is the testing that we’re doing telling us anything about the code?

One area that seems to be rife with bullshit is test automation.  YES, I SAID IT!!!  In fact, if you’re using off-the-shelf test automation tools (like Mercury products), then there’s a 90% chance (with a +-10% margin of error) that your test automation is complete and utter bullshit!  Before you get mad at me, ask yourself, how many bugs have you found by running that behemoth suite of tests?  How has it helped your product actually get better?  Are you getting any assurance that your product is up and running?  Was it worth the oil sheik fortune that you paid for it?

Let me give you a real world example on automation bullshit.  There’s this product team that I have intimate knowledge of.  This product team has both unit tests (JUnit) and Selenium UI tests being written against their code base.  The unit tests are being run with each build, and the build won’t complete if any tests break.  The Selenium tests are scheduled to run nightly in Hudson at various intervals so they don’t overlap.  Except for a one hour reserved maintenance window, the Selenium tests keep the servers busy all night.  Yet, with all this “testing” that is being done, the product still often arrives to the QA team egregiously broken.  Why is that?  Don’t the unit tests find problems?  The offshore team writing the Selenium tests told me that they have 700+ tests running at regular intervals.  He went on to say that they have no bugs to report from running those tests, but when they do, they will let me know.  Hmm, so you’ve spent months creating these suites, and you aren’t finding any bugs?  On the other hand, you’ve paid another team to write unit tests on code they didn’t write, and major problems are still getting through?  You have these Cobetura reports show that you’re getting close to complete coverage on the code base.  Complete coverage and there’s still problems.  YOU GOT TO BE KIDDING!!!  WHAT A STEAMING LOAD OF BULLSHIT!!!  One team is writing Selenium tests.  One team is writing JUnit tests.  And there are no bugs to show for it.  There are no red flags showing us potential defects.  There is no confident assurance that anything is working or not working.  I think it’s safe to say that the company is getting a negative return on their testing investment.  And that, in this case, automation is a complete waste of time.

Obviously, JUnit, Cobertura, and Selenium aren’t bad tools.  They just happen to be bad tools in the wrong hands–in the wrong situations.  They are used to create all these “tests”.  Instead of just accepting these “tests”, teams should be testing that the tests do something meaningful.  They should be looking to the yield of the fruits of their labor, not how much work they’ve done.   For every automated test written, you should prove how it can fail by making it fail. Think about it!  Test it!  Have others think about it too!

Let’s get away from automation because my point isn’t to trash automation.  Here’s another example.  Given Agile’s popularity, I think the full manual regression of a product is very passé.  The full regression of a big product could take weeks or months.  If your dev team is adding code everyday, then it’s not pragmatic to take on a full regression.  You need to be more surgical with your testing, more precise.  Focus on testing the parts that are actually effected by change.  The full regression is one of those archetypes that’s accepted because it’s familiar, but it’s unrealistic, antiquated, and pretty stupid in a continuous integration world.  However, you may be in an industry that requires that kind of attention to every detail.  And in your case, that might not be a bullshit testing practice.

My point is not to tell you what you’re doing to test your product is bullshit.  My point is to wake you up to practices that aren’t helping your product.  I understand that not all tests are created to find problems, but some testing is there to assure you don’t have problems (regression testing).  However, are you sure that your tests will reveal a problem when there is a problem?  My point is to be skeptical.  Don’t just do something because you saw it done like this when you worked at IBM or AT&T.  Don’t just do something because that’s the only thing you know to do.  Do something because it’s going to make the code better.  Start with a goal for the testing in mind.  If your goal is to tell you X, then come up with scenarios where that X would be revealed.  Then execute those scenarios!  Just like we don’t trust the developer code handed to us.  Don’t let your left hand trust the test case that your right hand created.  Remember, at the end of the day, it’s all about the return on investment.  Just like developers deliver value by coding something useful, testers need to deliver value by uncovering problems or telling the true story of the product.

And no other group on the whole face of the earth is better at pointing out flaws than software testers.  But when does testing go too far?  When are our scenarios become too outlandish to reasonably fit into reality?

Here’s an actual scenario:

For the past couple weeks, we’ve been trying to get some new hire testers up to speed on our product.  They are zealous and making good progress.  One of these testers found an address control in our UI that’s doing no validation between the City, State, and Zip.  i.e you can put in “CA” for the state and “10100″ for the zip.  Clearly without looking, you can tell that a California zip would have a lot higher number than that.  And if you match it up for the city, you can be even more specific.  Good find.  It’s perfectly reasonable to assume that those values should be validated.  That’s a very basic thing now.  There are services out there to assist with that.  Not hard to develop; we did lots of that kind of stuff at Edfinancial.  The tester submitted the “Bug” to development, and they denied it on the grounds that it works as coded and no requirements to validate the City, State, and Zip.  That’s a reasonable response too.  Though it’s a defect that might cause a problem given a certain data set, it’s really management’s call on whether it needs to be fixed.

This zealous new tester decided to write a response to development, arguing that they must fix it because it could put our company in legal troubles, violating compliance and privacy agreements.  Though my senior in age and experience, he wisely asked me advice on whether to send it or not.  Given my knowledge of the culture,  I told him there’s no way in Hell that I would commit that response to the ticket.  He did his job.  He reported the defect, explaining the scenario nicely.  End of story.  Just drop it.  He countered with other passionate arguments that the mail from us might end up in a competitor’s hand, given a Zip Code mix up, and it might cost our company millions of dollars to straighten out the problem.  I told him that his arguments weren’t really sound because the probability of that actually happening is slim to none.  Out of the millions of addresses in the US, what is the chance that the letter, because of a bad Zip Code, could end up in a competitor’s hand?  Even if it did, what are the chances that the competitor would open mail not directed to them?  Even if they did, what kind of ruthless havoc could they inflict on our company?  They certainly wouldn’t want to advertise that they broke federal law by opening someone else’s mail.  In my mind, it’s a moot point.  Maybe it could happen, but there’s no realistic chance.

Epiphany:

Then it hit me!  As developers underplay the chances of their bugs causing a problem, we testers often overplay the effects of leaving bugs in! We traffic in hyperbole as much if not more than developers.  We predict doom and gloom when the truth is everything will probably be fine.  So, why do we do it?

• The Business Loves Emergencies: Maybe we believe that if we don’t paint a picture of Armageddon, it won’t ever get fixed.  At my company, there’s nothing like a good fire to get the coders up and running, especially when it happens at odd hours!
• Hedging Against Disaster: Maybe we can’t represent the bug in proper context because we’re afraid that it might be bigger than we expected.  We say it’s no problem, then it’s the BP oil spill.  It’s better to be on the “I told you so” end of a disaster.
• Self-Importance: Maybe we have to make the bugs look bigger so that we can feel better about the contribution we’re making.  Maybe we’re a bunch of ego maniacs starved for accolades.

Final Thought:

As a tester, credibility is king.  If you’re a credible, the business will heed just your word.  If you’re a chicken little tester, clucking at every little thing, they will take everything you say with a grain of salt.  Be honest as you can with yourself.  ”To thine own self be true, ” as The Bard said.  Don’t focus on the trivial.  Focus on the big picture.  Focus on testing!

Postscript:

Here’s something interesting.  According to one of our subject matter experts, the suspect address control isn’t being used by ANY of our current customers.  So, the lack of validation is all really “Much ado about nothing.”  Two Shakespeare references in one post!  Score!

Okay, so, have you heard about NUnit attributes, and how they’re awesome?
Sure you’ve heard of Setup, Test, TestFixture, and TearDown? But how about Values? How about Range?

The Values attribute allows you to run the same test several times with different values. It makes it where you’re not doing something crazy like copy and pasting the same test, over and over again! I was close to that situation, but it hurt my soul too much to do that. So, in order to save my soul, I had to seek a path for redemption. Here’s what I found:

Okay, to try this, you may need to set up a sample test. If you need to start from scratch, check out my End-to-End Example: http://thetestingblog.com/2009/09/10/selenium-rc-in-c-using-nunit-an-end-to-end-example/

My example test verifies that the text “Daniel Brown” is present on the main testing blog page–a simple pass or fail. But imagine we wanted to run the test 3 times, each time verifying the presence of a different Testing Blog author. Here’s how we would do it using the Values attribute:

        [Test]
        public void ValuesTest([Values("Daniel Brown", "Marisa Seal", "Josh Carroll")] string textPresent)
        {
            selenium.Open("http://www.google.com");
            selenium.Type("q", "thetestingblog.com");
            selenium.Click("btnI");
            selenium.WaitForPageToLoad("60000");
            Assert.IsTrue(selenium.IsTextPresent(textPresent));
        }

As you can see, we’re basically passing the argument “textPresent” into the method. The values are what you want the argument to be. For each value, run the test once with “textPresent” = to that value:
[Values("Daniel Brown", "Marisa Seal", "Josh Carroll")] string stringPresent

So, in the case of this test; here’s how it would run:
SetupTest();
ValuesTest(“Daniel Brown”);
TeardownTest();
SetupTest();
ValuesTest(“Marisa Seal”);
TeardownTest();
SetupTest();
ValuesTest(“Josh Carroll”);
TeardownTest();

This is how it looks in the test runner:

Okay, that wasn’t too bad.  Now, let’s go a crazy.  Let’s get nuts.  Potentially throwing best practices to the wind.  We’re going make this example where it pulls the values from a text file instead of hard coding them.  To do that, let’s add a text file to our project:

Right click on your project (“TestingBlogSeleniumRc”) in the Solution Explorer -> Add -> New Item…

Select Text File and call it ValueList.txt

Open the file in your editor and type the three values into the text file.  One per line:

Next we want to make sure that the text file is copied over into the bin folder on build.

Right click on your project (“TestingBlogSeleniumRc”) in the Solution Explorer and select Properties.  Then select Build Events.  Under the Pre-build event command line: (or post-build; it doesn’t matter) paste in:

copy "$(ProjectDir)ValueList.txt" "$(TargetDir)"

Our test is going to be looking for ValueList.txt so that copy command assures that it’s in the same folder as the test dll.

(FULL CODE EXAMPLE BELOW)

• We need to add a reference to System.IO
• Next we’re going to add a global string array to our class called textPresentList
• We are going to read the ValueList.txt file into an array in our setup.  As coded, it should only be read in the first time the SetupTest() method is run.  If you’re a Unit Test purist, you could easily change it to read the file in every time, but I thought it seemed like a waste.
• Next we need to add the Range attribute to a copied and renamed version of our first test.  The range should be hard coded from 0 to 2.  (I tried to do something clever like put it from 0 to textPresentList.Length-1, but the compile wouldn’t have it.  These NUnit attribute values, it appears have to be somewhat static.  This kludge that I’m showing is way to get around the static nature if only a little bit.  This way the values can be dynamic even if the number of values can’t be.  By the way, if anyone has a better idea to make it more dynamic, please post it.)
• Change the selenium.IsTextPresent(textPresent) to selenium.IsTextPresent(textPresentList[i]).
• Then Compile and Run.

You won’t be able to see the values from the file in the Test Runner, but you could write the value translations to the console so that you could see which index corresponds to which value:
Console.WriteLine("Index " + i +": " + textPresentList[i]);

In the Test Runner, select the Text Output tab to view the output:

The whole point of this post is not to show you what to do, but instead to open up your mind to possibilities.  NUnit has some cool functionality, and most of us never scrape the surface of it’s potential.  I challenge you to go and explore the various attributes available for yourself.  Here’s the link: http://nunit.org/index.php?p=attributes&r=2.5.7

Remember upgrade your NUnit to the newest version.  Chances are that you’re not current.

FULL CODE:

using System;
using System.Text;
using System.IO;
using System.Text.RegularExpressions;
using System.Threading;
using NUnit.Framework;
using Selenium;

namespace TestingBlogSeleniumRc
{
	[TestFixture]
	public class FirstTest
	{
		private ISelenium selenium;
        private string[] textPresentList;

		[SetUp]
		public void SetupTest()
		{
            if (textPresentList == null)
                textPresentList = File.ReadAllLines("ValueList.txt");
			selenium = new DefaultSelenium("localhost", 4444, "*iexplore", "http://www.google.com/");
			selenium.Start();
		}

		[TearDown]
		public void TeardownTest()
		{
		    selenium.Stop();
		}

        [Test]
        public void DynamicValuesTest([Range(0,2)] int i)
		{
			selenium.Open("http://www.google.com");
			selenium.Type("q", "thetestingblog.com");
			selenium.Click("btnI");
			selenium.WaitForPageToLoad("60000");
            Console.WriteLine("Index " + i +": " + textPresentList[i]);
            Assert.IsTrue(selenium.IsTextPresent(textPresentList[i]));
		}
        [Test]
        public void ValuesTest([Values("Daniel Brown", "Marisa Seal", "Josh Carroll")] string textPresent)
        {
            selenium.Open("http://www.google.com");
            selenium.Type("q", "thetestingblog.com");
            selenium.Click("btnI");
            selenium.WaitForPageToLoad("60000");
            Assert.IsTrue(selenium.IsTextPresent(textPresent));
        }
	}
}

Here was the spark that finally spurred me to action this morning.  And I’ve seen this before but it just hit me particularly wrong this morning:

RESTART!!!!  You got to be kidding!!!  Why the Hell do I have to restart?  All I did was update a viewer, really just a plugin for a browser.  Adobe, do you have any idea how long it takes my work computer to restart?  What could be so important that you’re making me reboot?  If it’s such a pain for me, it’s most likely a pain for other people too.  Have you considered the irritation you’re causing everyone?

The Solution: Foxit

Okay, I’ve had enough.  I uninstalled Adobe Reader, and I installed a replacement product that my friend Brandon recommended called Foxit.  http://www.foxitsoftware.com/pdf/reader/

• My first worry was that it doesn’t work with my browsers.  However, that worry was unfounded.  I tested pulling up a PDF from a URL in IE8, Firefox, and even Chrome (my browser of choice).
• My first observation was the speed.  It was so much faster than Adobe.  (Didn’t Adobe invent PDF’s?)
• Plus, the Interface looks pretty refined and easy to use.
• Foxit install is 10.44 MB versus Adobe Reader 9.3 at 143 MB

I don’t completely endorse the product because I haven’t had substantial time using it, but so far I like it.   The main thing is that I got a chance to pwn Adobe!  Take that you bastards!

I was in the middle of making a rather lengthy comment on Daniel’s most recent blog post when I decided it was long enough to warrant a full posting.

I wanted to get across the point that the situation Daniel finds himself in is not solely the plight of the software tester. Rather, anyone who esteems software quality runs the risk of being criticized, dismissed, or made fun of. Why is it that organizations and groups are so resistant to something that will improve their process, their product, and their bottom line. The idea is one I have been thinking a lot about, and I believe I have had an epiphany.

Sometimes, people are more afraid of success than failure!

Wait… what did you say?!? You heard right, and I’ll say it again. Sometimes, people are more afraid of success than failure.

I recently gave a presentation where I cited the Microsoft study on TDD that found:

The results of the case studies indicate that the pre-release defect density of the four products decreased between 40% and 90% relative to similar projects that did not use the TDD practice. Subjectively, the teams experienced a 15–35% increase in initial development time after adopting TDD.

The idea that up to 90% of your test and fix cycles can be eliminated represents an enormous cost savings even if you factor in the high end estimate of 35% increase in initial development time. You will more than make that time up by not having to fix the 90% of bugs that are now gone from the system. With numbers like that you would think that management would be breaking people’s knuckles for not using TDD. Developer’s would be falling over themselves to TDD even the smallest spec of code just to avoid doing maintenance (curse the ground that software maintenance walks on).

But alas, there was no stampede. No rush to go forth and test. No proclamation from management that it must be so. So… what gives?

What gives is that it might just work! Developers might just write cleaner code, and be more productive. Testers might be able to confirm code quickly and have the freedom to explore automated acceptance testing. Deadlines might be met, and the company might save money. And… and… we might have to admit that we have been doing it all wrong for a very long time.

And there it is; the big white elephant in the room of software quality. Not fear of failure, but fear of success. Because success means having to admit things can and should be better. Even worse, that they could have been better all along. And we don’t like that. Most of us like to think we are pretty smart, that we are pretty damn good at what we do. When someone shows us a better way, our little bubble bursts, and we are forced to admit that we aren’t as special as we’d like to think we are.

The mediocre will always defend the status quo because their sense of worth is derived from being the smartest, funniest, sexiest, whateverist person around. They are a big fish in a little pond, and they like it that way.

The exceptional are never satisfied with the quality of their own work. They assume that there must be a better way, and seek out the best and brightest in the world to learn from. They draw their sense of worth from knowing they are better today than they were yesterday.

Exceptionalism is the arch-enemy of mediocrity, and success is it’s kryptonite!

I was very impressed by the people that attended the talk.  They were all software developers and highly technical. Plus, they all seemed interested in testing.  (Of course, when I pulled out the candy, people got even more interested.)  I was glad to have an audience that challenged, asked questions, and made terrific observations and points.  People mentioned technologies they used.  They talked about patterns and approaches, citing real world implementations.  As I speaker, I couldn’t ask for more.

Not one person from my current company came to the presentation, though I informed everyone through email with notice.  That seems reasonable.  The presentation was downtown KC (our company is 25 mins away in the suburbs); someone would have had to be very interested in coming to want to go.  Either that or really owe me a favor (i.e. I went to one of their presentations or baby sat for their kids).  I wasn’t shocked that no one came.  What I was shocked about was how few people replied to wish me luck or asked me about it.  This is after I sent out an email with the phrase, “I’m presenting on Exploratory Testing tomorrow, April 27^th at 7:00 pm at the DotNext meeting.  Wish me luck.”  Three people out of 30 wished me luck.  And one guy asked me about it the next day.  That tells me that people don’t care.  And they don’t even care about faking to care.  What does that tell you?  Two groups of software developers: one group wanted to listen (total strangers) and one didn’t want to condescend interest (people I know).

Jesus said to them, “A prophet is not without honor except in his hometown and among his own relatives and in his own household.” (Mark 6:4)

Most of the time, QA people don’t get the respect they deserve.  Working with my colleague Marisa  at Edfinancial, QA was well respected and not seen as the pariahs of the group.  Most organizations don’t respect their QA prophets.  We, the prophets, tell them that their shortcuts are going to bite them in the butt.  We tell them that standards and sound process will help things, and they don’t believe us.  We tell them that bugs need to be fixed and defects are dangerous, and they don’t listen.  No only will they not listen, but they also want to stone us and run us out of town.

My question is where do you, as a QA prophet, draw your purpose and satisfaction?

Here’s the part that I give the answer, right?

Wrong.  I don’t have the answer.  All I know is that it’s probably something intrinsic.  I think that will and desire have to come from within.

If you are a little lost by the title, it will all make sense in a moment… um… sort of.

Until I started my career as a software developer I worked a lot of other jobs. I have had a job since I was 16 years old, and that was just an excuse to get out of doing manual work for my parents around the house. I’ve worked as a bagger at a grocery store, cashier at a big-box-mart, usher, concessions, and box clerk for a movie theater, tech support for a DSL provider, general laborer for a masonry outfit, and the last job I worked before slinging code full-time was waiting tables.

All of these jobs really have one thing in common: I was the person who always caught the tail end of the customer’s experience: whether it was good or bad… especially when it was bad!

Waiting tables is an interesting experience, and one I think everyone should have to do at least once in their life. As a waiter you are the face of the dining experience for the customer. Before the customer even gets there, you are preparing the store to make their time enjoyable. You clean the kitchen, wash dishes, stock pantry items, napkins, silverware. You go over your section to make sure that everything is set up correctly and clean.

Once a table in your section has been seated, as far as the customer is concerned, you are the restaurant. You size up the customer to figure out what kind of approach you should take (believe me it makes a difference). You give them an overview of the menu, get their drinks, throw out a couple of suggestions, and meticulously write down the order  making double sure to repeat back the order so there are no misunderstandings. Without delaying, you ensure that the order is put into the computer accurately and check to see that the kitchen staff gets the ticket. If needed you even explain any nuances of the order to the cooks: “‘SUB BAKER ADD CHEESE.’ Hey Joe, they want a baked potato instead of mashed, and they want to add cheese to the sandwich, NOT the baker!”

While waiting for the food you are feverishly busing, washing, and resetting tables for the next set of patrons that walk into your section; you make sure they think they are the first people to sit there. A single spec of food left on the table or the floor can ruin the whole meal! Every time you walk into the kitchen you are checking the status of your food.

“307 IN THE WINDOW!”

Your order is up and now you are checking every detail to make sure the food you have matches the ticket. You tidy up the presentation a little and make sure the plates are picture perfect. Drop the food off to the table and make sure they have everything they need: drink, condiments, extra napkins, steak knives, etc… You let the table dig in and about two minutes later you come back to make sure everything was up to par. You never take them at their word either! You have to actually look at their plates, and their faces to see if they are telling the truth. Most people are too sheepish to tell you everything sucks, and they will simply not come back to the restaurant if the experience is bad. It is up to you to see this and recover their patronage.

Ok. So what? Why waste your time with this?

Because QA and waiting tables have a lot in common. As QA, you are the standard bearer for a quality user experience much the same way that a waiter is the standard bearer for a quality dining experience. In order to really be effective you have to be involved in the process from start to finish. You need to be there when the user story is first conceptualized in order to effectively develop a testing strategy. You need to determine the set of criteria by which a story can be considered done (acceptance criteria). You also need a repeatable way to ensure the story as coded meets those criteria (Fit/FitNesse, StoryTeller, Selenium).

As the story is being developed you need to be in constant communication with the developer (cook) to make sure that any misunderstandings are cleared up immediately. Developers make assumptions if the acceptance criteria are vague. So find out what those assumptions are, and refine your acceptance tests accordingly. Be proactive about communicating your testing strategy to the developer(s) working on the story so they won’t be tempted to just throw it over the wall. A good developer will make your job easier here, but good acceptance tests do NOT care how well written the code is. Remember that your main goal is not to find bugs, but to verify that the feature/product will meet the customers needs. An acceptance test that fails is not a bug, it is simply an incomplete feature, and we don’t ship incomplete features.

Being a QA professional takes a lot of discipline and involves a lot of responsibility, but it is your job as a professional to demand software quality above all else. A professional will stand on principle in this area while an amateur will make excuses for the level of quality… “I don’t have enough time to do automated testing! The developers write bad code! My boss doesn’t appreciate QA!”… All those things may be true, and may make it harder to do your job, but it doesn’t change your job!

QA can be a tiring and thankless job, but the professional derives pleasure from a job well done. Be committed to the project/story/user, and not your own hide. This is what separates “testers” from Software Quality Professionals. This is the difference between a great user experience and a mediocre one.

The meal is over, and the customer is satisfied. On their way out the table says “The meal was excellent! Be sure and thank the cook for me,” as they hand you a three dollar tip.

“I’ll do that sir!” You say with a smile, knowing that you did your job. You made sure that everything was perfect. You took care of all the grimy little details so they could concentrate on what they came there for… a good meal.

Déjà vu all over again! Another basic Selenium example. What the heck!? Can’t this guy do anything else?

Well, I agree this is somewhat repetitive, but I hope that this sequel will be in the league of Empire Strikes Back or Star Trek: The Wrath of Khan. Both movies, though they were sequels, were arguably greater than the originals. These sequels introduce us to the likes of Ricardo Montalban playing “Khan” and Billy Dee Williams playing “Lando Calrissian”.  And of course,  a muppet playing “Yoda”.

Of course, many sequels start off with the desire to be Empire, and instead they end up being:

• Caddyshack II
• Beastmaster 2: Through the Portal of Time
• Speed 2: Cruise Control
• Batman and Robin (No, we don’t need nipples on the Batman suit, Joel Schumacher.  But thanks for asking.)
• Mannequin: On the Move.
• Or the biggest sacrileges of all time Godfather III
• And Episode I (Empire, it is not!).

My hope is that this post doesn’t suck like those movies.  At the very least it’s nipple and Clooney free.

• Selenium RC
• Firefox
• Selenium IDE
• Java Runtime Environment (You should already have this)
• Visual C# 2008 Express Edition (Only get this if you don’t have a full version of Visual Studio)

Selenium RC
-No installation necessary. Just find a permanent location and unzip it there. I unzipped mine at C:\ just to make things simple

Firefox, Visual Studio Express, and Java
-Just run the respective installs with all defaults selected.

Selenium IDE
-To install, right click on the file and select “Open With…”
-Select Firefox in the programs list. If it’s not in the immediate menu, browse to it. (default location should be “C:\Program Files\Mozilla Firefox\firefox.exe“
-When Firefox launches, click button in the dialog box that says “Install Now”.
-After the install of the plug-in, hit the “Restart Firefox” button when you see it.

Okay, so, now that you have the tools, we’re going to record a simple test in Selenium IDE. Even though we want to eventually run the test in Selenium RC, Selenium IDE is a great tool for getting the basic test recorded since it’s all UI point and click.

• Start up Firefox.
• Click on the “Tools” menu and click on “Selenium IDE”
• (Take your time to explore the Selenium IDE form)
• Notice that it’s already recording.  The red circle button on the right side is pressed in:
• Type “www.google.com” the main Firefox browser address bar and navigate to it.
• In the search box, type in “thetestingblog.com” and hit the “I’m Feeling Lucky” button.
• Notice in your Selenium IDE window that steps are being recorded.
• With any luck, you hopefully landed on this blog’s home page.
• Highlight the text “Daniel Brown” on the page, right click.  You should select the option that says “VerifyTextPresent Daniel Brown”.  See below:

• Okay, we got our test recorded.  Now let’s run it to make sure it passes.
• Unclick the red circle button to turn off recording.
• Click on the green play triangle with the single bar to run the test

• All green, it passed.  Good.
• Now we want to convert this test to C#.
• Select “Options ->Format -> C# – Selenium RC” from the Selenium IDE window.

• Next select all the C# code and copy it.

• Open up Visual Studio
• Select “File ->New ->Project”
• In the New Project window under project types, select “Visual C# ->Windows”
• Select the “Console Application” Template and name it something like “TestingBlogSeleniumTest” and click “Ok”.
• It will create the project along with a file called “Program.cs”. Paste your SeleniumIDE test into Notepad. You’re going to take pieces out of the test and put it in “Program.cs”.

1. Add the using references from the SeleniumIDE test.  Replace any that are there.
2. Add the “ISelenium” and “StringBuilder” object references inside the Main() method.  As you see, I removed the “private” access control prefixes.  We don’t need them.
3. Add the body of the SetupTest() method after the object references.  (I put a comment to designate that it was part of the setup.)
4. Change “*chrome” to “*iexplore” for Internet Explorer.  Use “*firefox” for Firefox.  (There are other browser options, but those work best.)
5. Change the URL to the site you start your test on.  In this case, change that URL to “http://www.google.com/” since we’re starting from Google.

6. Paste body of the TheUntitledTest() method into the Main() method.

7. Paste body of the TeardownTest() method into the Main() method.

8.  Get rid of the Try/Catch blocks on Teardown.

9. Get rid of the StringBuilder references.

10.  Get rid of the Try/Catch block in the test body, replacing it with an if statement that writes to the console window if true (See my code below).  Notice that I have to cast the Selenium.IsTextPresent statement as a boolean.  I don’t understand why it doesn’t work without it because the return type for that method should be boolean.  But whatever.  I’m just dummy tester; what do I know?

• Next we need to add all the references so this thing will run.
• Go to the solution explorer, and right click on the “References” and select “Add Reference…”
• Hit the “Browse” tab and navigate to where you unzipped your SeleniumRC files.  Look in the dotnet client driver folder.  Here’s the path to mine:

C:\selenium-remote-control-1.0.1\selenium-dotnet-client-driver-1.0.1\

• Once you navigate to where the proper *.dlls are, select all of them and click “OK” to add them to the project.  (I know you can probably get away with adding less, but let’s keep it simple.)

• Build the project to see if it works.  (Mine worked.  Hopefully yours did too.)

• In order to run the Selenium RC test, you first have to have the Selenium RC server running.  The Selenium RC server folder should be in the files you unzipped.  Normally, I have a batch file on my desktop created to run the Selenium Server, but for our purposes, we’ll just run this from the “Start -> Run” commandline, using this command:

java -jar C:\selenium-remote-control-1.0.1\selenium-server-1.0.1\selenium-server.jar

• You should see some a dos window popup, and you will see something like the screenshot below.  DON’T CLOSE IT WHILE TESTING; IT NEEDS TO STAY RUNNING.  If you don’t have Java installed, this won’t work.  But most people should already have Java.

Next you need to select Debug from your Visual Studio menu and select “Start Without Debugging”. (I’m choosing this option because it doesn’t close the console window after the end of execution.  The debug option does.)

You should see the Selenium window(s) pop-up.  And it should do it’s thing.  When it’s finished, you should see something like this:

So, where is my test executable?  Well, it’s in the bin\debug folder of the project.  For me, the path to it looks like this:

C:\Documents and Settings\dbrown\My Documents\Visual Studio 2008\Projects\TestingBlogSeleniumTest\TestingBlogSeleniumTest\bin\Debug\

So, what do I need to run the program?  Just take the executable file and the dll’s.  Make sure the dll’s are in the same folder as the test when you execute it, otherwise you will get a big fat error.  If you send the executable to someone, make sure you send the dll’s too.

Remember if you port that executable to another computer, it needs to be a Windows box with the .NET framework installed and Selenium server needs to be running.  I hope this actually helps someone.  I’m going to post some tricks to “sex up” your console app.  Be looking for them shortly.

Sorry, I still haven’t figured out Selenium GRID good enough to post on it.  Maybe Marisa could post on it?  She’s awesome like that.

Then something happened one day!  A fob became available.  As soon as I got the email, I replied to all claiming my stake over the available fob.  Much to the chagrin of the office admin, who seems to have a problem with me, I ended up with the fob!  But she wouldn’t let me physically have it until the registration is changed over in the system.  That should take a day.  I was fine with that, but I shouldn’t have been.  The whole reason for having electronic systems is so that things can happen instantly.  Well, maybe it’s not the system itself.  Maybe it’s the human keying it into the system that’s the bottleneck.

Once I received the fob from our admin, I went to use it.  It didn’t work.  Damn.  I wondered if the office admin was vindictive enough to run a magnet across my fob.  Then I checked myself for thinking so highly of myself.  (Like she really cares enough to sabotage my fob!) I asked a maintenance man, who just happen to be passing by what I can do.  He asked me if I’m suppose to have access.  Did I pay for the fob?  Am I using someone else’s fob?  I was a little annoyed by the questions, but I kept a smile on my face and assured him that I’m suppose to have access.  He then directed me to the 3rd floor office of the building managers.

The lady in there was very cordial, and she told me that it’s possible I wasn’t pointing the fob correctly at the access point.  I knew that the fob should work, but I was willing to condescend her.  We walked down together.  She pointed the fob at the access panel, and sure enough, it didn’t work.  She told me that she didn’t understand; it should work.  Then, as if by magic, she remembered that they are working on the door; they are doing a software upgrade so the system may be down.  Obviously that wasn’t the case because there were people in the gym working out.  If the door was inaccessible, then they wouldn’t have been able to get in.  We went back up to her office.  She told me to call her in the afternoon, and she would attempt to re-add me to the system.  I told her okay, and I took a business card off here desk (she’s on the hook.).

In the afternoon, I called her, and she said that I was already in the system so my fob must be bad.  Come to her office and get another fob.  I get another fob from her, and she said that I’ll probably be good to go.  I told her that if it doesn’t work I’m coming back up stairs to see her.  I went down, and tried it.  No, it didn’t work.  So I marched back up stairs.  I’m very perturbed, but you have to keep on a good face.  She tells me that the software upgrade may have had trouble.  she’ll send one of the maintenance guys to give the fob to me once they get it working.  Later in the day, she calls me and tells me that the problem will be resolved early next week.

On Monday (today), my good friend, the office admin, came by and gave me the fob.  Hmm, I assume they got it working.  Being a tester, I got to go try it.  So I walk over to the workout facility and try it.  It doesn’t work.  I wanted to be mad, but I couldn’t bring myself to be angry.  It’s typical.  I thought they were going to try it before they sent it back over?  That’s what the lady promised.

Immediately, I go upstairs to the property management office.  Another lady greets me, and I tell her who I am.  She apologizes to me.  Apparently the other lady set me up wrong.  She explained that there’s two databases that needed to be updated with the information.  The first lady had just updated one.  Everything should be fine tomorrow once the system updates are in place.  The lady, to her credit, asked me if I wanted to use her fob to go workout.  I told her no; I just would like mine going eventually.  She told me that it should be fine tomorrow.  I told her that I didn’t believe her, but I will try it tomorrow.

WHAT’S THE POINT OF THIS WHOLE STORY?

I’m sick and tired of software excuses!  Don’t tell me the system takes a whole day to update!  I don’t give a crap that the system is down.  I don’t give a crap that two databases need to be updated.  It ought to just work!  As the user, I should be abstracted from all those details.  Here’s my user story: USER MUST BE ABLE TO USE KEY FOB TO OPEN THE DOOR.  Is that complicated?  Why can’t it be simple?  If it was just a regular lock, I  could have went down to Walmart, made a copy of the key, and got back in time to start achieving my fitness goals.  Why are there two databases that need to be updated?  It’s just one flipping door!  Software excuses are so ingrained in society that they are legitimate as “my diabetes is acting up”.  Nobody questions the bullshit of them.  And why is that?  I think it’s because most people don’t understand software.  As someone that understands software better than most of the populous, I say enough is enough.  I don’t care that systems are complex!  Stop the insanity!  This is a work in progress, but here’s my new software manifesto:

Daniel’s New Software Manifesto:

1) Software should be Simple to Use.  If your target user can’t *easily* use it (intuitive), it’s poorly designed, and it should chunked into the fire.

2) Software should Work.  Your software should work 99% of the time.  If your users are constantly complaining about problems with your software, then you suck and need to be chunked into the fire.  Don’t compromise on quality; it’s a slippery slope.  If you evangelize quality (and really mean it), you software will get better.

3) Software should be designed for the User.  I have an infinite amount of respect for Steve Jobs because he gets this.  He doesn’t have software developers (0r testers) designing the UI.  He actually pays people to design the UI so that Apple’s software looks slick and appealing.  I hate that companies are too cheap to invest in a good designer or user experience expert.  Design for the user!

4) Software downtime is Unacceptable.  Downtime is unacceptable for your software.  Make sure it’s up and running.  Regular maintenance is okay, but it’s not great.  The more you make this acceptable; the more downtime you’ll have.  Start with the premise that it’s unacceptable, and you’ll do better with it.

5) Software should fulfill the User’s Need. If you have software to control the access of a door, it better do it’s job.  If it doesn’t let the user in, it’s a utter failure and should be chunked into the fire.

EPILOGUE:

Probably when (if) they resolve my access issue, there will probably be no retrospective to say that things need to be different.  I sure the ladies in that office might feel bad for me, but nothing is going to change in their process.  I’m sure the second lady just thinks that it was a training issue with the first lady (i.e. if she knew that you have to do this and this and update two databases then that would have solved it).  But I really think that’s a wrong way of looking at it.  Maybe the software needs to be simplified.  Maybe they need some quality control in place (i.e. try the fobs on the door before they hand them off).  Maybe they need to restructure access systems.

But here’s the real food for thought: how many times do we as software professionals put the users through Hell and then not care?  I want you to empathize.  Every time you throw out some bullshit code, you are ruining someone’s life.  Do you really want to be the deliverer of misery in the world?  Do you want to blame their woes on your diabetes acting up?  Or do you want to say, “There’s no excuse.  Our servers sucks!  This code sucks!  This system sucks!  It’s imperative that we make it better.  To bring about a better world.”

As of today, I am officially a contributor to FitNesse. I contributed a new feature idea and its implementation (the initial idea was borrowed from FitLibrary’s SequenceFixture but morphed a bit based on suggestions from Bob Martin), and also worked on Slim version detection. I am not a developer by training nor trade, but armed with basic coding skills and fueled by the guidance and what I can only describe as the graciousness of Uncle Bob, I was successful. The experience was trying, exciting, and actually pretty fun. While I learned an awful lot more than I’m going to detail here, I think I had three really important realizations while working on the updates.

TDD Works!

Given that I am a tester, I have read/learned about, discussed, and even advocated TDD. Given that I do not (usually) write production code, I had never actually tried it. I decided I’d give it a go since I needed to write unit tests for my updates anyway.

The existing unit tests for the class I updated served as my guide. I wrote a test to verify the simplest case of the new feature. I ran it – here’s the interesting thing: it passed. This surprised me, but also informed me that I did not need to make any changes to the code for the simplest case. The update involved implementing a special character to invoke a different processing scheme for function calls in Slim Script tables, so I had figured that I’d at least need to update the code to ignore the special character in the simplest case…but as the unit test proved, I did not.

I wrote a second unit test to verify another simple (yet different) case – this one failed as expected. I added code to the class in question, ran the test, and saw it fail again. But aha – I also saw why it failed, and that information helped me find my mistake. At this point I’d like to say that I made the fix, ran the test, and saw it pass – but it didn’t happen that way. I went through a few iterations of running the test and fixing my code.

The test finally passed, and I ran the entire suite of tests for the class. They all passed. The most appropriate word I can find to describe that moment is epiphanic. It was then that I truly understood and appreciated two of the benefits of TDD I have often heard:

1. Practicing TDD can help ensure that only necessary code is implemented, since the goal of writing/updating the code is to make one test pass. Remember, I didn’t even need to write any code to make the first test pass.
2. Having a suite of unit tests – a byproduct of practicing TDD – can provide confidence that updates have not unintentionally changed existing behavior. The bonus is that unit tests are fast, so the feedback can be almost instantaneous. For example, the entire suite of 208 unit tests for the class I updated runs in under 1.5 seconds on my machine.

Having an understanding of a practice is one thing; having experience implementing the practice is another. Upon reflection, I now find it odd that I could have advocated TDD without having tried it myself.

Refactoring is more than just “Changing”

I’m a bit embarrassed to admit this, but up until recently, I used the word “refactoring” as a substitute for “changing,” since I thought that’s what refactoring meant. I thought it was just a dev-ified word for “changing” or “improving.”

Once I submitted the first update to Bob Martin, I asked him for feedback. I was quite proud that the new feature took only about a dozen new and updated lines of code, but I wondered what I could have done better. Uncle Bob told me that since I had added a couple lines of code to an existing function, the function’s length was getting to the point where he’d want to extract some of the functionality – he suggested extracting a particular loop to a “method object.” I immediately Googled “extract to method object” and found information about it on refactoring.com. I quickly realized that refactoring actually encompasses specific techniques that address specific scenarios – refactoring is not just changing code.

I’m not as embarrassed to admit that I really had no clue how to execute “extract to method object.” I ruminated about it for a few days, and finally just told Uncle Bob that I didn’t know how to get started. He sent me the code, explained step-by-step how he got to the end result, but still allowed me to commit the change myself…which leads me to my last point.

I think I know one reason why he’s called “Uncle” Bob

There were many times throughout the last few weeks that I wondered when Uncle Bob would say to me “I’ll just do it myself – it’ll be faster that way.” That certainly would have been a true statement. Being that I am not – as I stated before – a developer by training or trade, I was surprised that he even supported the idea of me actually committing code changes when I first contacted him to suggest the new feature. It turns out that my fear and surprise were completely unfounded. Uncle Bob continually offered guidance and encouragement, and always ended his messages with “Let me know if you need any help.” Here’s the kicker: he meant it, too.

I really did feel like I was the lucky recipient of mentoring by a person who has literally “written the book” (multiple books, actually) about software craftsmanship.

I hope that some FitNesse users find the new feature valuable, and that I will be able to contribute to FitNesse again in the future. It was a great experience.