Okay, so, have you heard about NUnit attributes, and how they’re awesome?
Sure you’ve heard of Setup, Test, TestFixture, and TearDown? But how about Values? How about Range?

The Values attribute allows you to run the same test several times with different values. It makes it where you’re not doing something crazy like copy and pasting the same test, over and over again! I was close to that situation, but it hurt my soul too much to do that. So, in order to save my soul, I had to seek a path for redemption. Here’s what I found:

Okay, to try this, you may need to set up a sample test. If you need to start from scratch, check out my End-to-End Example: http://thetestingblog.com/2009/09/10/selenium-rc-in-c-using-nunit-an-end-to-end-example/

My example test verifies that the text “Daniel Brown” is present on the main testing blog page–a simple pass or fail. But imagine we wanted to run the test 3 times, each time verifying the presence of a different Testing Blog author. Here’s how we would do it using the Values attribute:

        [Test]
        public void ValuesTest([Values("Daniel Brown", "Marisa Seal", "Josh Carroll")] string textPresent)
        {
            selenium.Open("http://www.google.com");
            selenium.Type("q", "thetestingblog.com");
            selenium.Click("btnI");
            selenium.WaitForPageToLoad("60000");
            Assert.IsTrue(selenium.IsTextPresent(textPresent));
        }

As you can see, we’re basically passing the argument “textPresent” into the method. The values are what you want the argument to be. For each value, run the test once with “textPresent” = to that value:
[Values("Daniel Brown", "Marisa Seal", "Josh Carroll")] string stringPresent

So, in the case of this test; here’s how it would run:
SetupTest();
ValuesTest(“Daniel Brown”);
TeardownTest();
SetupTest();
ValuesTest(“Marisa Seal”);
TeardownTest();
SetupTest();
ValuesTest(“Josh Carroll”);
TeardownTest();

This is how it looks in the test runner:

Okay, that wasn’t too bad.  Now, let’s go a crazy.  Let’s get nuts.  Potentially throwing best practices to the wind.  We’re going make this example where it pulls the values from a text file instead of hard coding them.  To do that, let’s add a text file to our project:

Right click on your project (“TestingBlogSeleniumRc”) in the Solution Explorer -> Add -> New Item…

Select Text File and call it ValueList.txt

Open the file in your editor and type the three values into the text file.  One per line:

Next we want to make sure that the text file is copied over into the bin folder on build.

Right click on your project (“TestingBlogSeleniumRc”) in the Solution Explorer and select Properties.  Then select Build Events.  Under the Pre-build event command line: (or post-build; it doesn’t matter) paste in:

copy "$(ProjectDir)ValueList.txt" "$(TargetDir)"

Our test is going to be looking for ValueList.txt so that copy command assures that it’s in the same folder as the test dll.

(FULL CODE EXAMPLE BELOW)

• We need to add a reference to System.IO
• Next we’re going to add a global string array to our class called textPresentList
• We are going to read the ValueList.txt file into an array in our setup.  As coded, it should only be read in the first time the SetupTest() method is run.  If you’re a Unit Test purist, you could easily change it to read the file in every time, but I thought it seemed like a waste.
• Next we need to add the Range attribute to a copied and renamed version of our first test.  The range should be hard coded from 0 to 2.  (I tried to do something clever like put it from 0 to textPresentList.Length-1, but the compile wouldn’t have it.  These NUnit attribute values, it appears have to be somewhat static.  This kludge that I’m showing is way to get around the static nature if only a little bit.  This way the values can be dynamic even if the number of values can’t be.  By the way, if anyone has a better idea to make it more dynamic, please post it.)
• Change the selenium.IsTextPresent(textPresent) to selenium.IsTextPresent(textPresentList[i]).
• Then Compile and Run.

You won’t be able to see the values from the file in the Test Runner, but you could write the value translations to the console so that you could see which index corresponds to which value:
Console.WriteLine("Index " + i +": " + textPresentList[i]);

In the Test Runner, select the Text Output tab to view the output:

The whole point of this post is not to show you what to do, but instead to open up your mind to possibilities.  NUnit has some cool functionality, and most of us never scrape the surface of it’s potential.  I challenge you to go and explore the various attributes available for yourself.  Here’s the link: http://nunit.org/index.php?p=attributes&r=2.5.7

Remember upgrade your NUnit to the newest version.  Chances are that you’re not current.

FULL CODE:

using System;
using System.Text;
using System.IO;
using System.Text.RegularExpressions;
using System.Threading;
using NUnit.Framework;
using Selenium;

namespace TestingBlogSeleniumRc
{
	[TestFixture]
	public class FirstTest
	{
		private ISelenium selenium;
        private string[] textPresentList;

		[SetUp]
		public void SetupTest()
		{
            if (textPresentList == null)
                textPresentList = File.ReadAllLines("ValueList.txt");
			selenium = new DefaultSelenium("localhost", 4444, "*iexplore", "http://www.google.com/");
			selenium.Start();
		}

		[TearDown]
		public void TeardownTest()
		{
		    selenium.Stop();
		}

        [Test]
        public void DynamicValuesTest([Range(0,2)] int i)
		{
			selenium.Open("http://www.google.com");
			selenium.Type("q", "thetestingblog.com");
			selenium.Click("btnI");
			selenium.WaitForPageToLoad("60000");
            Console.WriteLine("Index " + i +": " + textPresentList[i]);
            Assert.IsTrue(selenium.IsTextPresent(textPresentList[i]));
		}
        [Test]
        public void ValuesTest([Values("Daniel Brown", "Marisa Seal", "Josh Carroll")] string textPresent)
        {
            selenium.Open("http://www.google.com");
            selenium.Type("q", "thetestingblog.com");
            selenium.Click("btnI");
            selenium.WaitForPageToLoad("60000");
            Assert.IsTrue(selenium.IsTextPresent(textPresent));
        }
	}
}

Here was the spark that finally spurred me to action this morning.  And I’ve seen this before but it just hit me particularly wrong this morning:

RESTART!!!!  You got to be kidding!!!  Why the Hell do I have to restart?  All I did was update a viewer, really just a plugin for a browser.  Adobe, do you have any idea how long it takes my work computer to restart?  What could be so important that you’re making me reboot?  If it’s such a pain for me, it’s most likely a pain for other people too.  Have you considered the irritation you’re causing everyone?

The Solution: Foxit

Okay, I’ve had enough.  I uninstalled Adobe Reader, and I installed a replacement product that my friend Brandon recommended called Foxit.  http://www.foxitsoftware.com/pdf/reader/

• My first worry was that it doesn’t work with my browsers.  However, that worry was unfounded.  I tested pulling up a PDF from a URL in IE8, Firefox, and even Chrome (my browser of choice).
• My first observation was the speed.  It was so much faster than Adobe.  (Didn’t Adobe invent PDF’s?)
• Plus, the Interface looks pretty refined and easy to use.
• Foxit install is 10.44 MB versus Adobe Reader 9.3 at 143 MB

I don’t completely endorse the product because I haven’t had substantial time using it, but so far I like it.   The main thing is that I got a chance to pwn Adobe!  Take that you bastards!

I was very impressed by the people that attended the talk.  They were all software developers and highly technical. Plus, they all seemed interested in testing.  (Of course, when I pulled out the candy, people got even more interested.)  I was glad to have an audience that challenged, asked questions, and made terrific observations and points.  People mentioned technologies they used.  They talked about patterns and approaches, citing real world implementations.  As I speaker, I couldn’t ask for more.

Not one person from my current company came to the presentation, though I informed everyone through email with notice.  That seems reasonable.  The presentation was downtown KC (our company is 25 mins away in the suburbs); someone would have had to be very interested in coming to want to go.  Either that or really owe me a favor (i.e. I went to one of their presentations or baby sat for their kids).  I wasn’t shocked that no one came.  What I was shocked about was how few people replied to wish me luck or asked me about it.  This is after I sent out an email with the phrase, “I’m presenting on Exploratory Testing tomorrow, April 27^th at 7:00 pm at the DotNext meeting.  Wish me luck.”  Three people out of 30 wished me luck.  And one guy asked me about it the next day.  That tells me that people don’t care.  And they don’t even care about faking to care.  What does that tell you?  Two groups of software developers: one group wanted to listen (total strangers) and one didn’t want to condescend interest (people I know).

Jesus said to them, “A prophet is not without honor except in his hometown and among his own relatives and in his own household.” (Mark 6:4)

Most of the time, QA people don’t get the respect they deserve.  Working with my colleague Marisa  at Edfinancial, QA was well respected and not seen as the pariahs of the group.  Most organizations don’t respect their QA prophets.  We, the prophets, tell them that their shortcuts are going to bite them in the butt.  We tell them that standards and sound process will help things, and they don’t believe us.  We tell them that bugs need to be fixed and defects are dangerous, and they don’t listen.  No only will they not listen, but they also want to stone us and run us out of town.

My question is where do you, as a QA prophet, draw your purpose and satisfaction?

Here’s the part that I give the answer, right?

Wrong.  I don’t have the answer.  All I know is that it’s probably something intrinsic.  I think that will and desire have to come from within.

Déjà vu all over again! Another basic Selenium example. What the heck!? Can’t this guy do anything else?

Well, I agree this is somewhat repetitive, but I hope that this sequel will be in the league of Empire Strikes Back or Star Trek: The Wrath of Khan. Both movies, though they were sequels, were arguably greater than the originals. These sequels introduce us to the likes of Ricardo Montalban playing “Khan” and Billy Dee Williams playing “Lando Calrissian”.  And of course,  a muppet playing “Yoda”.

Of course, many sequels start off with the desire to be Empire, and instead they end up being:

• Caddyshack II
• Beastmaster 2: Through the Portal of Time
• Speed 2: Cruise Control
• Batman and Robin (No, we don’t need nipples on the Batman suit, Joel Schumacher.  But thanks for asking.)
• Mannequin: On the Move.
• Or the biggest sacrileges of all time Godfather III
• And Episode I (Empire, it is not!).

My hope is that this post doesn’t suck like those movies.  At the very least it’s nipple and Clooney free.

• Selenium RC
• Firefox
• Selenium IDE
• Java Runtime Environment (You should already have this)
• Visual C# 2008 Express Edition (Only get this if you don’t have a full version of Visual Studio)

Selenium RC
-No installation necessary. Just find a permanent location and unzip it there. I unzipped mine at C:\ just to make things simple

Firefox, Visual Studio Express, and Java
-Just run the respective installs with all defaults selected.

Selenium IDE
-To install, right click on the file and select “Open With…”
-Select Firefox in the programs list. If it’s not in the immediate menu, browse to it. (default location should be “C:\Program Files\Mozilla Firefox\firefox.exe“
-When Firefox launches, click button in the dialog box that says “Install Now”.
-After the install of the plug-in, hit the “Restart Firefox” button when you see it.

Okay, so, now that you have the tools, we’re going to record a simple test in Selenium IDE. Even though we want to eventually run the test in Selenium RC, Selenium IDE is a great tool for getting the basic test recorded since it’s all UI point and click.

• Start up Firefox.
• Click on the “Tools” menu and click on “Selenium IDE”
• (Take your time to explore the Selenium IDE form)
• Notice that it’s already recording.  The red circle button on the right side is pressed in:
• Type “www.google.com” the main Firefox browser address bar and navigate to it.
• In the search box, type in “thetestingblog.com” and hit the “I’m Feeling Lucky” button.
• Notice in your Selenium IDE window that steps are being recorded.
• With any luck, you hopefully landed on this blog’s home page.
• Highlight the text “Daniel Brown” on the page, right click.  You should select the option that says “VerifyTextPresent Daniel Brown”.  See below:

• Okay, we got our test recorded.  Now let’s run it to make sure it passes.
• Unclick the red circle button to turn off recording.
• Click on the green play triangle with the single bar to run the test

• All green, it passed.  Good.
• Now we want to convert this test to C#.
• Select “Options ->Format -> C# – Selenium RC” from the Selenium IDE window.

• Next select all the C# code and copy it.

• Open up Visual Studio
• Select “File ->New ->Project”
• In the New Project window under project types, select “Visual C# ->Windows”
• Select the “Console Application” Template and name it something like “TestingBlogSeleniumTest” and click “Ok”.
• It will create the project along with a file called “Program.cs”. Paste your SeleniumIDE test into Notepad. You’re going to take pieces out of the test and put it in “Program.cs”.

1. Add the using references from the SeleniumIDE test.  Replace any that are there.
2. Add the “ISelenium” and “StringBuilder” object references inside the Main() method.  As you see, I removed the “private” access control prefixes.  We don’t need them.
3. Add the body of the SetupTest() method after the object references.  (I put a comment to designate that it was part of the setup.)
4. Change “*chrome” to “*iexplore” for Internet Explorer.  Use “*firefox” for Firefox.  (There are other browser options, but those work best.)
5. Change the URL to the site you start your test on.  In this case, change that URL to “http://www.google.com/” since we’re starting from Google.

6. Paste body of the TheUntitledTest() method into the Main() method.

7. Paste body of the TeardownTest() method into the Main() method.

8.  Get rid of the Try/Catch blocks on Teardown.

9. Get rid of the StringBuilder references.

10.  Get rid of the Try/Catch block in the test body, replacing it with an if statement that writes to the console window if true (See my code below).  Notice that I have to cast the Selenium.IsTextPresent statement as a boolean.  I don’t understand why it doesn’t work without it because the return type for that method should be boolean.  But whatever.  I’m just dummy tester; what do I know?

• Next we need to add all the references so this thing will run.
• Go to the solution explorer, and right click on the “References” and select “Add Reference…”
• Hit the “Browse” tab and navigate to where you unzipped your SeleniumRC files.  Look in the dotnet client driver folder.  Here’s the path to mine:

C:\selenium-remote-control-1.0.1\selenium-dotnet-client-driver-1.0.1\

• Once you navigate to where the proper *.dlls are, select all of them and click “OK” to add them to the project.  (I know you can probably get away with adding less, but let’s keep it simple.)

• Build the project to see if it works.  (Mine worked.  Hopefully yours did too.)

• In order to run the Selenium RC test, you first have to have the Selenium RC server running.  The Selenium RC server folder should be in the files you unzipped.  Normally, I have a batch file on my desktop created to run the Selenium Server, but for our purposes, we’ll just run this from the “Start -> Run” commandline, using this command:

java -jar C:\selenium-remote-control-1.0.1\selenium-server-1.0.1\selenium-server.jar

• You should see some a dos window popup, and you will see something like the screenshot below.  DON’T CLOSE IT WHILE TESTING; IT NEEDS TO STAY RUNNING.  If you don’t have Java installed, this won’t work.  But most people should already have Java.

Next you need to select Debug from your Visual Studio menu and select “Start Without Debugging”. (I’m choosing this option because it doesn’t close the console window after the end of execution.  The debug option does.)

You should see the Selenium window(s) pop-up.  And it should do it’s thing.  When it’s finished, you should see something like this:

So, where is my test executable?  Well, it’s in the bin\debug folder of the project.  For me, the path to it looks like this:

C:\Documents and Settings\dbrown\My Documents\Visual Studio 2008\Projects\TestingBlogSeleniumTest\TestingBlogSeleniumTest\bin\Debug\

So, what do I need to run the program?  Just take the executable file and the dll’s.  Make sure the dll’s are in the same folder as the test when you execute it, otherwise you will get a big fat error.  If you send the executable to someone, make sure you send the dll’s too.

Remember if you port that executable to another computer, it needs to be a Windows box with the .NET framework installed and Selenium server needs to be running.  I hope this actually helps someone.  I’m going to post some tricks to “sex up” your console app.  Be looking for them shortly.

Sorry, I still haven’t figured out Selenium GRID good enough to post on it.  Maybe Marisa could post on it?  She’s awesome like that.

Then something happened one day!  A fob became available.  As soon as I got the email, I replied to all claiming my stake over the available fob.  Much to the chagrin of the office admin, who seems to have a problem with me, I ended up with the fob!  But she wouldn’t let me physically have it until the registration is changed over in the system.  That should take a day.  I was fine with that, but I shouldn’t have been.  The whole reason for having electronic systems is so that things can happen instantly.  Well, maybe it’s not the system itself.  Maybe it’s the human keying it into the system that’s the bottleneck.

Once I received the fob from our admin, I went to use it.  It didn’t work.  Damn.  I wondered if the office admin was vindictive enough to run a magnet across my fob.  Then I checked myself for thinking so highly of myself.  (Like she really cares enough to sabotage my fob!) I asked a maintenance man, who just happen to be passing by what I can do.  He asked me if I’m suppose to have access.  Did I pay for the fob?  Am I using someone else’s fob?  I was a little annoyed by the questions, but I kept a smile on my face and assured him that I’m suppose to have access.  He then directed me to the 3rd floor office of the building managers.

The lady in there was very cordial, and she told me that it’s possible I wasn’t pointing the fob correctly at the access point.  I knew that the fob should work, but I was willing to condescend her.  We walked down together.  She pointed the fob at the access panel, and sure enough, it didn’t work.  She told me that she didn’t understand; it should work.  Then, as if by magic, she remembered that they are working on the door; they are doing a software upgrade so the system may be down.  Obviously that wasn’t the case because there were people in the gym working out.  If the door was inaccessible, then they wouldn’t have been able to get in.  We went back up to her office.  She told me to call her in the afternoon, and she would attempt to re-add me to the system.  I told her okay, and I took a business card off here desk (she’s on the hook.).

In the afternoon, I called her, and she said that I was already in the system so my fob must be bad.  Come to her office and get another fob.  I get another fob from her, and she said that I’ll probably be good to go.  I told her that if it doesn’t work I’m coming back up stairs to see her.  I went down, and tried it.  No, it didn’t work.  So I marched back up stairs.  I’m very perturbed, but you have to keep on a good face.  She tells me that the software upgrade may have had trouble.  she’ll send one of the maintenance guys to give the fob to me once they get it working.  Later in the day, she calls me and tells me that the problem will be resolved early next week.

On Monday (today), my good friend, the office admin, came by and gave me the fob.  Hmm, I assume they got it working.  Being a tester, I got to go try it.  So I walk over to the workout facility and try it.  It doesn’t work.  I wanted to be mad, but I couldn’t bring myself to be angry.  It’s typical.  I thought they were going to try it before they sent it back over?  That’s what the lady promised.

Immediately, I go upstairs to the property management office.  Another lady greets me, and I tell her who I am.  She apologizes to me.  Apparently the other lady set me up wrong.  She explained that there’s two databases that needed to be updated with the information.  The first lady had just updated one.  Everything should be fine tomorrow once the system updates are in place.  The lady, to her credit, asked me if I wanted to use her fob to go workout.  I told her no; I just would like mine going eventually.  She told me that it should be fine tomorrow.  I told her that I didn’t believe her, but I will try it tomorrow.

WHAT’S THE POINT OF THIS WHOLE STORY?

I’m sick and tired of software excuses!  Don’t tell me the system takes a whole day to update!  I don’t give a crap that the system is down.  I don’t give a crap that two databases need to be updated.  It ought to just work!  As the user, I should be abstracted from all those details.  Here’s my user story: USER MUST BE ABLE TO USE KEY FOB TO OPEN THE DOOR.  Is that complicated?  Why can’t it be simple?  If it was just a regular lock, I  could have went down to Walmart, made a copy of the key, and got back in time to start achieving my fitness goals.  Why are there two databases that need to be updated?  It’s just one flipping door!  Software excuses are so ingrained in society that they are legitimate as “my diabetes is acting up”.  Nobody questions the bullshit of them.  And why is that?  I think it’s because most people don’t understand software.  As someone that understands software better than most of the populous, I say enough is enough.  I don’t care that systems are complex!  Stop the insanity!  This is a work in progress, but here’s my new software manifesto:

Daniel’s New Software Manifesto:

1) Software should be Simple to Use.  If your target user can’t *easily* use it (intuitive), it’s poorly designed, and it should chunked into the fire.

2) Software should Work.  Your software should work 99% of the time.  If your users are constantly complaining about problems with your software, then you suck and need to be chunked into the fire.  Don’t compromise on quality; it’s a slippery slope.  If you evangelize quality (and really mean it), you software will get better.

3) Software should be designed for the User.  I have an infinite amount of respect for Steve Jobs because he gets this.  He doesn’t have software developers (0r testers) designing the UI.  He actually pays people to design the UI so that Apple’s software looks slick and appealing.  I hate that companies are too cheap to invest in a good designer or user experience expert.  Design for the user!

4) Software downtime is Unacceptable.  Downtime is unacceptable for your software.  Make sure it’s up and running.  Regular maintenance is okay, but it’s not great.  The more you make this acceptable; the more downtime you’ll have.  Start with the premise that it’s unacceptable, and you’ll do better with it.

5) Software should fulfill the User’s Need. If you have software to control the access of a door, it better do it’s job.  If it doesn’t let the user in, it’s a utter failure and should be chunked into the fire.

EPILOGUE:

Probably when (if) they resolve my access issue, there will probably be no retrospective to say that things need to be different.  I sure the ladies in that office might feel bad for me, but nothing is going to change in their process.  I’m sure the second lady just thinks that it was a training issue with the first lady (i.e. if she knew that you have to do this and this and update two databases then that would have solved it).  But I really think that’s a wrong way of looking at it.  Maybe the software needs to be simplified.  Maybe they need some quality control in place (i.e. try the fobs on the door before they hand them off).  Maybe they need to restructure access systems.

But here’s the real food for thought: how many times do we as software professionals put the users through Hell and then not care?  I want you to empathize.  Every time you throw out some bullshit code, you are ruining someone’s life.  Do you really want to be the deliverer of misery in the world?  Do you want to blame their woes on your diabetes acting up?  Or do you want to say, “There’s no excuse.  Our servers sucks!  This code sucks!  This system sucks!  It’s imperative that we make it better.  To bring about a better world.”

In some companies, software testing is often used to combat bad practices.  Our code is crap and it’s breaking at every turn, maybe we need some people to test it so we can find the problems.  As awesome as testers are, we can’t save you from your bad decisions.  Do not be deceived!  You reap what you sow.

I believe in quality assurance (making sure the process is sound) more than quality control (making sure the product is sound).  If you’re doing things the right way, you have a great chance of success.  Part of the reason I believe in QA over QC is because of Sun Tzu.  In the Art of War, he  says, “Victorious warriors win first and then go to war, while defeated warriors go to war first and then seek to win.”  If you have a good process in place, then you can develop software with confidence that it’s going to be of good quality.  If you have no process or a bad process in place, you’re fighting for quality but only after the bad quality software is complete.

So, what are some things that might make a good software development process:

Code Reviews: One of the biggest things I learned at Edfinancial was the magic of code reviews.  They work on so many levels.  They work:

1. As a device to Share Knowledge among your team.  Are you in a company that has legions of siloed developers working on their own thing?  Put a stop to it!  Implement code reviews.  It forces your team members to see what each other are doing.
2. As a Learning Opportunity.  You want to know how to mentor your young coders?  Have them do code reviews with more experienced coders.  They will learn so much about best practices.  (Personally, I learned a ton from my former colleague Ryan when he did my code reviews.)
3. To Spur Collaboration.  When you have code reviews, this is a great time for spirited debate on the implementation of something.  Also, it prevents someone from completely going off the reservation.  It’s the reality check.  It’s the “What were you smoking?” check.  Many times crazy bad code just gets committed without so much as a once over.  That’s tragic!  Unacceptable!  And easily avoided!
4. To Find Bugs. Besides testers, you know who’s good at finding bugs?  Other developers.  They are great at it!  They are almost always tough on other people’s code, and that’s great for quality.  They will catch the honest mistakes.  Developers often catch their own mistakes as they are showing the code to someone else.  THIS IS BIG!!

Configuration Management: Systems are very complex.  So is code.  So is coordinating several people working on the same code.  You need to make sure that someone is keeping it all straight and enforcing the rules with an iron fist.  You want a process that’s well defined and repeatable.

1. Source Control: You should have all your production code in source control.  If you’re deploying stuff that’s not in source control, SHAME ON YOU!  You source is the life blood of your team.  Without it; you’re dead!
2. Automated Build and Deployment: Nobody should be doing the build and deployment of complex systems completely by hand.  People are too prone to simple errors.  There are countless tools out there that simplify this process.  For shops that do this manually, it’s a gut wrenching experience, but it should actually be as easy as pressing a button.
3. Change Control, Configuration: When you deploy code, do you know exactly what’s going out?  Can you easily tell me what’s been deployed, built, and configured?  If not, then why not?  If you have source control and automated build and deployments, then the change control stuff should be easy.  There ought to be logs documenting that.
4. No Hot Fixes on Production: You should not be making hot fixes on production!  That’s a sign of bigger problems.  And if you do have to make a fix on production it needs to go through a review and approval process.  And it should be the exception, not the rule.  It should never be some lone coder firing new code into production without consulting anyone.  That’s just plain stupid.

Priorities: What the heck are we doing?

1. Requirements: Testers can’t effectively test when there’s no expectation of what the code is suppose to do.  How are we suppose to verify and validate requirements when there are no requirements to look to?  The better question might be how do developers code without requirements?  Companies need to take the time to gather enough requirements to move forward with development.  NO!  YOU’RE NOT BEING AGILE BY HAVING NO REQUIREMENTS OR DIRECTION!  THAT’S CALLED AD HOC CHAOS.
2. Vision: Okay, requirements are a day-to-day thing, but what is the big picture?  What are we trying to accomplish long term?  Note that the less defined the vision is, the less chance that you’ll accomplish something specific.  Oh, and share the vision with the team.  Don’t make it a mystery.
3. Focus: Once you as a company has decided what the development team will work on, don’t constantly change your mind.  A whimsical manager with a big stick can kill productivity more than anything.  One completed piece of software is greater than 10 incomplete pieces.  Keep your eyes on the vision.
4. Realistic Expectations: Some managers think that they can complete software by sheer will.  If I crack the whip hard enough, the developers and testers will work faster.  If the team says 100 hours of development and 100 hours of testing, you as a manager can’t say I need that next Friday.  Sure you delivered it by some arbitrary date, but you will be putting fixes out for the next 6 months.

My assertion is that companies often attack the cosmetic, top level problems instead of attacking the root cause problems.  Testing is often a band aid thrown on the gushing wound.  If they were interested in fixing things, they would attack the root.

I think the big problem is that I am naive.  I assume that companies are indeed interested in fixing those quality issues.  But in truth, they are probably not that interested.  As a top level person in management, it’s more important that I get the product out than it being good quality.  It’s better to sell vaporware than to have a good product and not be able to sell it.  Rather than fix problems, it’s easier to look like you’re fixing problems.  We have quality problems so I hired some testers.  See!  I did what I’m suppose to do.  I didn’t give them power to effect the process, but it’s a nice gesture to have a token tester or two so they can say things like, “That’s whack!”

As long as the money is coming in, does quality really matter anyway?

*Gemba: A Japanese term that’s used in Lean to describe the source of the problem.  Literally translated, it’s “the real place” or “the actual place”.  In the context of manufacturing, it’s the plant manager going down to the assembly line to look at the problems.  I used that particular buzz word because I knew that it would make Bill happy.  Inside joke between us.

It was an excellent open space format conference.  Naresh Jain and Andrew Chen are great men to organize such an enlightening conference.  Also, thank you sponsors (Univ of Pittsburgh, Agile Alliance, Version One, Pillar, Open Information Foundation, Lean Dog, Foreopen, and Westinghouse.) for enabling us to attend the conference for free.  You guys rock!

To get into the conference, you had to write a position paper on a topic you’re interested in discussing.  My position paper was on “Do we, as software testers, really matter?”  You can go read it if you want, but it’s one of the most depressing pieces I have ever written so you’ve been warned.  Basically, the premise of the paper is does it make sense to have software testers anymore or is it a deprecated trade with the evolution of software development?  When I talk about the evolution of software development, I’m talking about things like Test-Driven Development (TDD).  Since developers are being to take a test first, proactive stance toward quality, are we as pure software testers still needed?

Roles?  Where we’re going we don’t need roles:

One of the guys brought forth the position that “roles are evil”.  Basically, his position was something like roles within a software team are damaging because it silos responsibility to one person.  For example, if something breaks in production, then the tester is at fault for not testing that particular part.  There were several people that agreed that the team should take responsibility for all aspects of the development of the software.  Instead of concrete roles, teams should have “specializing generalists”.  What that means is that you should hire exceptional people that are talented enough to fill many roles (writing code, testing, talking to the business, architecture) but maybe specialize in one or two things.  I call this the “Commando Concept”.  In commando teams, you might have a medic, a demolition expert, a linguist, a martial arts expert, but they are all commandos.  Everyone should have should general knowledge in each area.  If a man gets hurt, everyone on the team should be able to give him medical help.  If they need to take down a bridge, everyone should be able to set up the charges and blow up the bridge.  Of course, they all need to know how to handle a hand-to-hand combat situation.  In the same way, software team members need to be able to cover all roles in order to get things done.  I liked the concept.  I thought it was great.

Reality:

All of this sounds good, but in reality, it never works that way.  One woman named Audrey brought up the fact that when everyone is responsible for an item, no one is responsible.  Things that no one wants to do just fall through the cracks and don’t get done.  For instance, there’s many developers out there that really hate testing (I was surprised to find so many developers that love testing at the SDT Conference).  Some loath testing because they could be writing more code instead of testing.  So, instead of testing the code that needs to be  tested, they might take another coding task, and the testing never gets done.  Maybe it’s touch points with the business, where nobody wants to go talk to the business to hear about their technical pain points.  Roles are in place to make sure that someone takes up the responsibility for what needs to be done.  The tester position is in place so that someone on the team worries about quality control and quality assurance.  Testers think differently, and it takes a little time to switch between a developer hat and a tester hat.  Sure developers can write unit tests and check basic logic and functionality, but who’s going to find the strange bugs that can only be found through exploratory testing?  (And yes, these bugs exist.)  Can you find people that have the talent to fulfill all the roles and have enough motivation to do them well?  Even if you have exceptional people that can be a tester, a developer, a business person, and a user, can you get enough of them to fill a whole team?  In reality, most businesses can’t afford to retain that many exceptional people.  It comes down to pure economics.  For a business, it’s cheaper to have one rock star that can do anything and the rest of the people functional in their one role.  If you have a team of rock stars, then the company will break up that team and put one rock star on every team.  Or those exceptional people will leave the company in search of better opportunities.  Or worse, the company sees their greatness and makes them into a manager, giving them a raise and throwing all their talent to the curb to be eaten by rats, birds, squirrels, ants, and raccoons.

Conclusion:

There may be enough exceptional people out there that can form a team, take on all roles, but trust me, the company will be unwilling to pay for those people.  Even if they get them, they won’t be able to keep them.  Since we may not be able to get the pure, roleless team, let’s look at what we can do.  I think that you assemble teams, and you give them the expectation that everyone is responsible for everything, especially quality.  To borrow from the band U2, I’ll call this the “One“ Philosophy.  ”When it’s one need…We have to carry each other.”   As a team you have ONE goal: release high quality software.  It’s not your part and my part.  It’s the summation of all our parts.  It’s all or nothing!  One!  We have help each other out and fight through our collective short comings.  ”We’re one but we’re not the same…We have to carry each other.”  Your role is developer.  My role is tester.  We are not in the same role, but regardless we have to carry each other to reach our goal: release high quality software.  As you embrace this philosophy of One, you will see the roles become fuzzy.  A developer might do testing.  A tester might gather additional requirements.  And a business analyst might fix the code.  OKAY, THE LAST ONE MIGHT BE A STRETCH!  But the point is once you carry (care) for each other, those roles won’t matter.  One will matter.

Watch the Video – U2 “One”

Great Scott! Hear the echoes and screams.
Hide your developers and product owners.
Lock the doors to the server room!
Here comes the Night of the Living Dead.
They are coming to test our software!
Their slow, reanimated bodies march to
The hypnotic spell of predetermined test cases.
They mindlessly push the buttons
On outdated automated test suites.
They write copious, bloated documentation
That no one will read!
If you are a free thinking software tester,
They are coming for you!
God help us! God help us all!

Zombie Testing: Mindlessly  executing a program with the intent of filling spreadsheets with useless data, writing thick boiler plate bug reports, and meeting company metrics.  Find bugs is a secondary pursuit.  Any requirements verifications should be done to the letter; there’s no reason to check the intent of the requirements.

It’s true that zombies walk among us!  Your fellow software testers might be zombie testers!  But how do you know?  I’m going to lay out some indicators that might out your colleague as a zombie:

• They act with mindless disregard toward their job (Passive not Proactive).

Testers that have turned display a mindless disregard for their job.  They do not think; their brains are not buzzing with activity!  They are not analyzing requirements, looking for ambiguities.  They are not listening to conversations about new software implementation.  They are not engaged with investigating bugs to find the root cause.  They don’t communicate their unsubstantiated concerns and hunches to the developers or the project managers.  Zombie testers never pick up on disconnects between technical people and customers, nor would they intervene to increase understanding anyway. They particularly like it when these test cases are written by a QA manager and just executed by them.  Less thinking = better!

• They protect the status quo, rejecting new methods of testing.

Not only are their minds soft, but they will seek to eat the brains of all who challenge the group think.  They stifle creativity.  They have a “not invented here” mentality.  Zombie testers will continue to look for bugs using the same heuristics, and they will continue to believe that they are being effective.  Anyone that questions this status quo will be dealt with severely.  Past successes will be used to give credence to their methods; they will point to the relative stability of the group’s code base.  They will reject radical new testing paradigms like Exploratory Testing in favor of set, predetermined test cases.  Using their super lethargic strength, they will cursh all dissenters until they are assimilated as zombie testers too.  Stay clear!  Their complacency is overpowering!

• They don’t question developers, other testers, managers, authority figures, or process; they just march slowly and aimlessly in stride with everyone else.

A tester that doesn’t challenge anyone or anything displays traits of potential zombification.  Zombie testers will never go against software developers; they will assume that since the developer is an eccentric genius they must know what they are doing.  After all, complicated and over-engineered code is better; right?  When the developer makes statements like “I know what the business wants more than they do”, a zombie tester will agree that this technical person knows the business better than the business people.  Zombie testers won’t disagree with other testers that have more experience than them; after all, these senior testers have written more tests than them.  These senior testers also protect the status quo, which is great!  Zombie testers won’t disagree with their supervisors, even if their supervisor comes in with some crappie half-baked idea that he learned at a conference.  They won’t question the guru testers that have written the books.  Never mind the fact that some of those grey hair gurus haven’t tested since COBOL was en vogue.

• They place too much faith in automated testing.

To a testing zombie, automated tests are the end all be all!  Every test requires a hammer, and our automated test platform is a big ass hammer.  Mindlessly crank these things out.  It’s easy!  Just model each test on the countless examples that are already in the massive suites.  There’s something so therapeutic about not having to think while writing test scripts.  It’s like watching TV.  These zombies love to push the button on the suite and watch as all their tests come up green.  They are all green so that must mean that the tests are all valid, cover the code correctly, and actually test what we want to test.  If the tester starts to question the integrity of the testing harness or decides to manually check something, he or she is probably not a zombie.  He or she is probably a healthy tester doing their job.

• They are more proud of the pretty documentation than the actual bugs they found.

One time, Bill had an interview with a tester candidate that was truly excited about the organization and prettiness of her bug reports.  He told me that in his head, he was thinking, “NO!  I don’t care about that.  I want you to find bugs, not write reports!”  If your testers really start to glow and brag about their documentation, then they might have already turned.  If they are overly critical about the way you write up your bugs or insist that you use boiler plate sheets, then be standoffish.  They might be feasting on your brains at any moment!  Tester zombies are slow, but they can be fast to protect the status quo.  If your tester is  not excited about they bugs they found or the innovative approaches they took to finding them, be very concerned!  Note: zombie testers tend to value metrics, methods, procedures, and systems more than they value making a difference in the quality of the software.  If they constantly quote material from the ISTQB certification test, then analyze how pedantic their tone is.  The more pedantic their tone, the more they’ve turned.

Because I love my readers and aspiring software testers, I’m going to give a dummy tutorial on how to get Selenium RC up and running for C#.NET users using NUnit. I’m going to cover every painful detail. I’m confident once you have this example running, you should be able to do a whole lot of things (assuming you have some basic coding skills).

First off, you need to get all these things (if you don’t have them):

• Selenium RC
• NUnit
• Firefox
• Selenium IDE
• Java Runtime Environment (You should already have this)

Selenium RC
-No installation necessary. Just find a permanent location and unzip it there. I unzipped mine at C:\ just to make things simple

NUnit, Firefox, and Java
-Just run the respective installs with all defaults selected.

Selenium IDE
-To install, right click on the file and select “Open With…”
-Select Firefox in the programs list. If it’s not in the immediate menu, browse to it. (default location should be “C:\Program Files\Mozilla Firefox\firefox.exe“
-When Firefox launches, click button in the dialog box that says “Install Now”.
-After the install of the plug-in, hit the “Restart Firefox” button when you see it.

Okay, so, now that you have the tools, we’re going to record a simple test in Selenium IDE. Even though we want to eventually run the test in Selenium RC, Selenium IDE is a great tool for getting the basic test recorded since it’s all UI point and click.

• Start up Firefox.
• Click on the “Tools” menu and click on “Selenium IDE”
• (Take your time to explore the Selenium IDE form)
• Notice that it’s already recording.  The red circle button on the right side is pressed in:
• Type “www.google.com” the main Firefox browser address bar and navigate to it.
• In the search box, type in “thetestingblog.com” and hit the “I’m Feeling Lucky” button.
• Notice in your Selenium IDE window that steps are being recorded.
• With any luck, you hopefully landed on this blog’s home page.
• Highlight the text “Daniel Brown” on the page, right click.  You should select the option that says “VerifyTextPresent Daniel Brown”.  See below:

• Okay, we got our test recorded.  Now let’s run it to make sure it passes.
• Unclick the red circle button to turn off recording.
• Click on the green play triangle with the single bar to run the test

• All green, it passed.  Good.
• Now we want to convert this test to C#.
• Select “Options ->Format -> C# – Selenium RC” from the Selenium IDE window.

• Next select all the C# code and copy it.

• Open up Visual Studio
• Select “File ->New ->Project”
• In the New Project window under project types, select “Visual C# ->Windows”
• Select the “Class Library” Template and name it something like “TestingBlogSeleniumRc” and click “Ok”.
• It will create the project along with a file called “Class1.cs”.  Paste your SeleniumIDE test into that file.

1. Rename “Class1.cs” to “FirstTest.cs”.
2. In the code, change the namespace from “SeleniumTests” to “TestingBlogSeleniumRc”:
3. Modify the class name from “Untitled” to “FirstTest”.  It’s good to change this so that if you add other tests, there won’t be any duplicated class names in the namespace (This has gotten me before, causing compile errors.).
4. We also have to modify the Selenium instantiation to launch Internet Explorer.  Where it has “*chrome”, change it to “*iehta”.
5. Where it has “http://change-this-to-the-site-you-are-testing/”, change it to “http://www.google.com”.

• Next we need to add all the references so this thing will run.
• Go to the solution explorer, and right click on the “References” and select “Add Reference…”
• Hit the “Browse” tab and navigate to where you unzipped your SeleniumRC files.  Look in the dotnet client driver folder.  Here’s the path to mine:

C:\selenium-remote-control-1.0.1\selenium-dotnet-client-driver-1.0.1\

• Once you navigate to where the proper *.dlls are, select all of them and click “OK” to add them to the project.  (I know you can probably get away with adding less, but let’s keep it simple.)

• Build the project to see if it works.  (Mine worked.  Hopefully yours did too.)

• In order to run the Selenium RC test, you first have to have the Selenium RC server running.  The Selenium RC server folder should be in the files you unzipped.  Normally, I have a batch file on my desktop created to run the Selenium Server, but for our purposes, we’ll just run this from the “Start -> Run” commandline, using this command:

java -jar C:\selenium-remote-control-1.0.1\selenium-server-1.0.1\selenium-server.jar

• You should see some a dos window popup, and you will see something like the screenshot below.  DON”T CLOSE IT WHILE TESTING; IT NEEDS TO STAY RUNNING.  If you don’t have Java installed, this won’t work.  But most people should already have Java.

• Next you need to launch the NUnit GUI to run the test from.  Since you installed NUnit, you should be able to navigate to it “Start -> All Programs -> NUnit 2.4.7 -> NUnit GUI (.NET 2.0)”
• Once you open the NUnit runner, Click “File -> Open Project…” and navigate to the compiled dll from your test project.  For me, the path looks like this:

C:\Documents and Settings\dbrown1\My Documents\Visual Studio 2008\Projects\TestingBlogSeleniumRc\TestingBlogSeleniumRc\bin\Debug\TestingBlogSeleniumRc.dll

• Once it loads, click on “Run” to start the test.

If you got all green, congrats!  You just got your first passing Selenium RC test!  From here we can definitely get more complex. I’m sorry if I was too verbose, but I didn’t want to leave out any important details.  If you have any specific issues, please post a question; I’ll be happy to answer it.  Thanks for reading!

UPDATE  10/14/2009:

Thank you for your comment, Ron.  You pointed out something that I completely spaced on.  There’s one more step you need to complete before your test is ready.  You must get rid of the try-catch exception block.  As long as you have that block in your test, your test will never fail.  Nunit counts on an exception being thrown when an assert statement fails.  If that exception gets caught, then Nunit thinks that the test passed.  Why does Selenium IDE put that try-catch in when coverting the test to C#?  I have no idea.  I just know that it doesn’t need to be in there in our example.

BEFORE Try-Catch is removed:

AFTER Try-Catch is removed:

Just because I want to prove that it works, I will show you an example where I have two tests in the file.  One looks for the text “Jello Pudding”, and one looks for the text “Daniel Brown”.  Here’s code for the tests:

(Helpful Note to Nunit beginners: when Nunit runs, it executes the [SetUp]  method before each [Test] and [TearDown] method after each [Test] .)   After running the two tests, I have very contrasting results (as I figured I would).  The test that looks for “Daniel Brown” passed like a charm, but the test for “Jelly Pudding” failed:

The bar is solid red because one failure at any level of the tree is a failed test run.  THANKS RON!  Sorry to leave out something so important.

Another order of business is I didn’t mention in my article, as granular as it is, that you need Visual Studio!  THANKS DAWN!  I guess I just kind of assumed that you might have it if you are interested in running Selenium in C#.  Well, I hate to assume so if you don’t have Visual Studio, you can download a free copy of Visual C# 2008 Express Edition: http://www.microsoft.com/express/download/ I believe you can run this example using the express edition, but I haven’t tested it.  If you do test it, give me some feedback.  I’d like to hear how it works.  Also, instead of the Visual C# Express Edition, you may also be able to use the Microsoft Visual Web Developer, but again I haven’t tested it.  I don’t know for sure.

So, it happens again.

Here’s another of example of email gone wrong:

One of the ladies at our church is over the kids program that I’m involved in. The program is about to kick off a new year, coinciding with the start of the school year. In an effort to be high tech (For my country church, it’s a big thing.), she’s switched to communicating to the workers through email. That’s very agreeable to me. In years past, the announcements for the kid’s program were buried in the Sunday bulletin with the women’s luncheon and choir practice. The only problem is, if you are going to communicate through email, you need to be effective in your communications.

She hosted a training over this past weekend. I was unable to attend the training because of a wedding out of town. I communicated this point so it was clear. No problem she said.

When I return from the wedding weekend, I see an email in my inbox from this lady. The email talked about how well the training went, and she was excited about upcoming year. It also mentions the upcoming conference for the program. It gives some details about it. She even highlights the details to make sure that everyone knows the price, date, and location. After I read through the whole thing and went on with my week.

Come Wednesday night, I see the email still in my inbox, and I begin to wonder when the program is actually going to start (usually it’s on Wednesday nights at about 6:00). So, I emailed her at about 7:00 pm right before I left work.

And asked her “When does [it] start?”

She apologized (Call me cynical, but it seemed insincere.), saying that she thought she told everyone. It was in the church bulletin, and the “email kind of stated not an exact date but said this week”.

The email she’s referring to was the one that I read on Monday when I returned from the wedding. The rather verbose email consisted of 4 paragraphs and 380 words. At this point I had read the email three times before trying to pull out the part that I had missed. I went back and looked again. I read it, and I read it. Then I found a sentence that referred to the training: “I hope it eased your minds about club starting this week”. That was the big announcement in the email! It was buried in fluff about how proud she was of everyone.

Emails are not a novel! Don’t write them like a novel, trying to build up suspense or hide meanings and obscure action! She eluded but never touched on the most important topic to get across: when the program starts! Everything else in the email doesn’t matter compared to Game Time. The trainings, the conferences, the words of encouragement; none of it mattered compared to actually having the meeting with the kids.

Disclosures: my pride is wounded a little about this because I take my responsibilities really seriously. I don’t want people to think I’m a dead beat volunteer. I don’t “phone it in” on things like this. I feel betrayed because I thought that email would let me know when to show up. I didn’t realize that I needed to follow up with her or read the church bulletin (even though I was out of town). I thought the point of implementing email was to make communication more streamlined. I didn’t know I had to go back to the old sources for information. Up to this point, she has sent us tons of little emails. Why didn’t she send a reminder out the day of? In her epic email, why didn’t she come out and say that we had our first meeting this week, August the 19th. Why wasn’t she cognizant to contact those that didn’t attend the training? Was she counting on the communication getting to me, that I read her email very carefully, or that I looked at the church bulletin? (One funny thing about the email. When writing this blog, I had to read through the email several times again just to find that sentence. I didn’t find it. Instead, I used ctrl-F. Hahaha.)

I am very much a believer in self responsibility. Ask anyone I work with, and they will tell you that Daniel doesn’t dodge blame when he’s responsible. If anything, I want take on all the blame and bring it to the fore front. I get satisfaction out of letting people know that I’m taking responsibility. But my gut told me that I can’t completely take responsibility for this screw up. At least not completely Why?

Doesn’t this sounds like a scenario from a software development team? A tester fails to test something that gets released to production and blows up:

“Why didn’t you test it? “

“I did, but I don’t remember that piece.” OR “No, I didn’t because I didn’t think that it pertained to me.”

Closing Remarks:

Communication is not about one person. If you throw some information out there and the other person doesn’t understand, you don’t get off scot-free because you threw it out there. If the person doesn’t understand you, you are still liable. Also, if you’re the listener or the one on the receiving end, you are not totally unresponsible if they give you an unclear message. You should be asking that important questions to ensure that their message gets to you with all proper meaning and tone attached. Communication break downs happen because of two people. If you know someone will not read your email, go talk to them in person (face-to-face: how novel!). Some people are brilliant but can’t read that well. Don’t be arrogant in thinking that you covered your butt. Think of a baton on a relay team. Does it matter if the hand off is bad or the catch is bad? No! The whole team is pwn’d if that baton gets dropped. When communication breaks down, everyone loses. You. The other person. The team. The company.

One more time because I like to rock.