As of today, I am officially a contributor to FitNesse. I contributed a new feature idea and its implementation (the initial idea was borrowed from FitLibrary’s SequenceFixture but morphed a bit based on suggestions from Bob Martin), and also worked on Slim version detection. I am not a developer by training nor trade, but armed with basic coding skills and fueled by the guidance and what I can only describe as the graciousness of Uncle Bob, I was successful. The experience was trying, exciting, and actually pretty fun. While I learned an awful lot more than I’m going to detail here, I think I had three really important realizations while working on the updates.

TDD Works!

Given that I am a tester, I have read/learned about, discussed, and even advocated TDD. Given that I do not (usually) write production code, I had never actually tried it. I decided I’d give it a go since I needed to write unit tests for my updates anyway.

The existing unit tests for the class I updated served as my guide. I wrote a test to verify the simplest case of the new feature. I ran it – here’s the interesting thing: it passed. This surprised me, but also informed me that I did not need to make any changes to the code for the simplest case. The update involved implementing a special character to invoke a different processing scheme for function calls in Slim Script tables, so I had figured that I’d at least need to update the code to ignore the special character in the simplest case…but as the unit test proved, I did not.

I wrote a second unit test to verify another simple (yet different) case – this one failed as expected. I added code to the class in question, ran the test, and saw it fail again. But aha – I also saw why it failed, and that information helped me find my mistake. At this point I’d like to say that I made the fix, ran the test, and saw it pass – but it didn’t happen that way. I went through a few iterations of running the test and fixing my code.

The test finally passed, and I ran the entire suite of tests for the class. They all passed. The most appropriate word I can find to describe that moment is epiphanic. It was then that I truly understood and appreciated two of the benefits of TDD I have often heard:

1. Practicing TDD can help ensure that only necessary code is implemented, since the goal of writing/updating the code is to make one test pass. Remember, I didn’t even need to write any code to make the first test pass.
2. Having a suite of unit tests – a byproduct of practicing TDD – can provide confidence that updates have not unintentionally changed existing behavior. The bonus is that unit tests are fast, so the feedback can be almost instantaneous. For example, the entire suite of 208 unit tests for the class I updated runs in under 1.5 seconds on my machine.

Having an understanding of a practice is one thing; having experience implementing the practice is another. Upon reflection, I now find it odd that I could have advocated TDD without having tried it myself.

Refactoring is more than just “Changing”

I’m a bit embarrassed to admit this, but up until recently, I used the word “refactoring” as a substitute for “changing,” since I thought that’s what refactoring meant. I thought it was just a dev-ified word for “changing” or “improving.”

Once I submitted the first update to Bob Martin, I asked him for feedback. I was quite proud that the new feature took only about a dozen new and updated lines of code, but I wondered what I could have done better. Uncle Bob told me that since I had added a couple lines of code to an existing function, the function’s length was getting to the point where he’d want to extract some of the functionality – he suggested extracting a particular loop to a “method object.” I immediately Googled “extract to method object” and found information about it on refactoring.com. I quickly realized that refactoring actually encompasses specific techniques that address specific scenarios – refactoring is not just changing code.

I’m not as embarrassed to admit that I really had no clue how to execute “extract to method object.” I ruminated about it for a few days, and finally just told Uncle Bob that I didn’t know how to get started. He sent me the code, explained step-by-step how he got to the end result, but still allowed me to commit the change myself…which leads me to my last point.

I think I know one reason why he’s called “Uncle” Bob

There were many times throughout the last few weeks that I wondered when Uncle Bob would say to me “I’ll just do it myself – it’ll be faster that way.” That certainly would have been a true statement. Being that I am not – as I stated before – a developer by training or trade, I was surprised that he even supported the idea of me actually committing code changes when I first contacted him to suggest the new feature. It turns out that my fear and surprise were completely unfounded. Uncle Bob continually offered guidance and encouragement, and always ended his messages with “Let me know if you need any help.” Here’s the kicker: he meant it, too.

I really did feel like I was the lucky recipient of mentoring by a person who has literally “written the book” (multiple books, actually) about software craftsmanship.

I hope that some FitNesse users find the new feature valuable, and that I will be able to contribute to FitNesse again in the future. It was a great experience.

Aromas

It actually starts with the very first job I ever held – a barista position at the Aromas coffeehouse on the USD campus. My aunt, who worked at the university, told me about the coffeehouse and put me in contact with the manager. I interviewed over the phone, was hired, and found out when to report for my first day on the job. Of course, I called my aunt and thanked her for the introduction. Realizing I had not asked what it is I should wear my first day, I asked my aunt. “Oh, why don’t you wear a nice dress?” she replied.

I showed up to work in a lovely frock, and the very kind manager complimented me on my dress. The sarcastic assistant manager, however, pulled a “get a load of this gal” move with the thumb-pointing and everything. I wasn’t able to train that day because people learning to use an espresso machine ought not to wear short dresses and risk spilling 200-degree espresso on themselves.

Lesson Learned: Ask about dress code if unsure

First Testing Job

Fast forward 4 years – I was an eager young woman fresh out of college, ready to start making some dough to pay off my student loan. I got my first full-time job at Pacific Life Insurance Company as a QA Analyst. After the introductions throughout the cube farm, my boss sat down with me in my cubicle and handed me a CD so that I could install the product I’d be testing. I took the CD and just sort of stared at my computer for a few seconds…I extended my hand out to the thingy under the monitor that had a lot of buttons on it but could not figure out how to open the dang CD drive. My boss said very slowly “Um…the CPU is under your desk.” Forehead smack! I wonder if my boss freaked out just a tiny bit in that moment after realizing that the person he hired to be a tester couldn’t “work the computer.” Now, of course I could work the computer…thanks to my grandpa, I’d been using one since I was about 7…I was just really nervous!

Lesson Learned: Sometimes you just gotta laugh at yourself

First Agile Testing Job

OK, fast forward another few years. I had over 6 years of software testing experience under my belt and was interviewing for a tester position at Edfinancial Services. My potential future boss asked me for my thoughts on bug documentation. I said very confidently, “I document EVERYTHING.”  The rest of the interview went well, and I got the job.

My first day, my new boss sat down with me for a discussion about testing in an agile environment. We got to talking about bug tracking and he said, “I think tracking bugs found during the sprint is a big waste of time.” My mind immediately zipped back to my proud statement about documenting EVERYTHING. Oops…

Lesson Learned: Be prepared to change the way you’ve “always” done things!

My New Job

And now we’re pretty much all caught up…well, wait a second – I did start a new job yesterday. I’m now a software tester for a small group, and I am stoked to be working with Chris McMahon in this gig. Anyway, Chris and I were going through tutorials about the online app we’ll be testing. One of the tutorials provided a link to a “sample” workspace. I started playing around in the sample workspace and had some questions…I posted my questions on the wiki. Chris, our trainer Marcus (he’s a project manager/developer), and I started chatting about the sample space and Marcus asked me for the URL of the sample space. I barely got the sub-domain out of my mouth and Marcus exclaimed “That’s production!”

Turns out that the tutorials need to be updated.

Lesson Learned*: Some day, you WILL accidentally do something on production

*Actually, this was a reminder for me – I’ve already learned the lesson about messing around with production…brought a DB server completely down unknowingly once.

Well, there we have it. The pattern was not broken. The good news is that I really, really like the group of people I’m working with now, and hopefully won’t have to experience another first day on the job for a long while.

Yesterday I started working on an automated test (or, if you prefer, collection of checks) that needed to verify data in a result set, so I started learning about the Slim Query table. The query table example and corresponding fixture code presented in the FitNesse user guide are fairly straightforward. The part that I really had to wrap my head around was the statement “The fixture class must have a query method that returns a list of rows. Each row is a list of fields. Each field is a two-element list composed of the field name and its string value.” Say whaaat?

Now, maybe for those of you who have more of a programming background than I do, you might have read that statement and immediately pictured what the Query results “look” like. I didn’t. I’m used to thinking of query results as a table, not as lists of lists. I had to create a visual.

Given this result set:

Starting at the lowest level, we have “Each field is a two-element list comprised of the field name and its string value.” OK, so for row 0 column 0 we’d have:

For row 0 column 1 we’d have:

etc.

Let’s go up a level. “Each row is a list of fields.” That makes sense now. For row 0 we’d have:

Finally, the Query method “returns a list of rows.” Ohhh now it makes sense.

Hmm…but then I had to figure out how to translate a DataTable result set into the list of this format. It was actually pretty simple; iterate through the rows and columns to create what I call column-value pairs, collect each row’s column-value pairs into a list, then collect the rows into a list.

I’ve provided my solution below (sorry about the formatting – this theme’s code tag sucks).

It’s a method that takes a DataTable and creates the list of lists of column-value pairs that will work with Slim Query tables. So, in each Slim Query Fixture class, I just call off to the method and pass in the DataTable of results, then return the list of lists of column-value pairs from the Query() method.


using System;
using System.Collections.Generic;
using System.Data;
using System.Text;


namespace Your.Namespace.Here
{
public static class SlimQueryResultsHelper
{
// Convert a result set in DataTable format to the format required by the Slim Query test table
public static List ConvertDataTableToObjectList(DataTable theTable)
{
List queryResults = new List();
foreach (DataRow row in theTable.Rows)
{
List rowListOfColumnValuePairs = new List();
foreach (DataColumn col in theTable.Columns)
{
List columnValuePair = new List();
object val = row[col];
columnValuePair.Add(col.ColumnName);
columnValuePair.Add(val == DBNull.Value ? "null" : val);
rowListOfColumnValuePairs.Add(columnValuePair);
}
queryResults.Add(rowListOfColumnValuePairs);
}
return queryResults;
}
}
}


I’m working on automating tests for a web app that has a lot settings configuration features, and many settings are managed in the UI via interactive grid controls. As the details of this app are proprietary, I will of course use a fake example. Since I’m on a Beatles kick, let’s imagine (har  har sorry for the bad pun and yes I know that was Lennon solo) I’m testing an album ranking control that looks like this:

If I record my interactions with the grid using Selenium IDE, the output is something like this:

Notice anything? I notice a couple things.

1. Aside from the fact that I see “grdBeatlesAlbums” in the locator string, there is no context to the rest of it – “//tr[4]” doesn’t appear to correspond to Abbey Road which was the album I was moving up in the ranks.
2. Yikes – am I really going to have to figure out the table row number for the album I want to act on in every test involving this grid?

My first inclination was to write a method that would determine the row position of an album with a given name. Then I realized that would involve parsing HTML. Yuck. I thought to myself, there has to be an easier way.  I went back to the Selenium docs to read about control location strategies a little more closely and saw that xpath would probably do the trick.  As so succinctly put on w3school’s page, “XPath is used to navigate through elements and attributes in an XML document.”

Now my problem was figuring out how to write an xpath expression that would locate a control in a column “next to” a given album name.

I wrote an almost-plain-English sentence to describe what I needed.

Version 1: Locate the “Move Up” link in the column adjacent to “Abbey Road.”

I thought “adjacent” would be a difficult concept but after Googling for a bit, I found information about the following and preceding xpath axes.

Version 2: Locate the “Move Up” link following the cell containing the string “Abbey Road.”

OK, so now to translate this to xpath – the Move Up link is just an HTML link, a.k.a. an “a” element.

a[.='Move Up'] translates to: a element whose inner text exactly matches ‘Move Up‘

And the cell containing “Abbey Road” – well, that’s just a table cell, or “td” element.

td[.='Abbey Road'] translates to: td element whose inner text exactly matches ‘Abbey Road‘

Next step: relate the two.

td[.='Abbey Road']/following::a[.='Move Up'] translates to: a element whose inner text exactly matches ‘Move Up‘, which follows a td element whose inner text exactly matches ‘Abbey Road‘

We just need one more thing: since the td element is not the first element on the page, we need to prepend the statement with // to indicate that the entire document (page source) should be searched to find the element.

//td[.='Abbey Road']/following::a[.='Move Up']

Even though the end result isn’t exactly plain English, it’s a lot more clear to me than “/tr[3]/td[4]/a” which is what the IDE recorded. It could also be reusable, providing the album name is passed via a variable instead of hard-coded.

Using xpath, you can also locate elements whose attributes (e.g. id or name) match certain criteria, and there are some handy functions like contains() to help in situations where a control’s full id or name is not known beforehand.

I’d still prefer the convenience of using control IDs, but there are just some situations where that either not possible or practical – xpath is now my fallback for tricky UI.

Update 8/28/09

Since I didn’t include this originally, the way to write an expression to find an element based on attribute values is

elementtype[@attributename='attribute value']

e.g. //table[@id='someTableID']

An example with contains():

elementtype[contains(@attributename, 'attribute substring')]

Now on to the real topic…I’ve been working with a software development team to guide them in their implementation of agile and automated testing techniques for about a month now. At the first sprint demo (yes – the team practices Scrum), I wanted to show the team a couple of examples of simple automated tests I had created for one of their applications. I demonstrated a GUI-driven test and a sub-GUI test. The demo went off without a hitch and the team seemed eager to start automating their tests in these manners.

The next day, I was to conduct a more detailed meeting with the testers regarding my suggestions for implementing automated tests to cover a particular application. Again, all was going well until I said something (I thought to be) pretty benign – something along the lines of “Well, I’m hoping that the architecture of the app is such that most of these test actions and verifications can be performed sub-UI.”

I heard brakes screech.

All of a sudden, a debate ensued about how driving the application some other way than through the UI was not a “real” acceptance test (after all, “real” users have no choice but to use the application via the UI). I stumbled and fumbled to explain; I was so used to automating 90% of a test suite at the sub-UI level that I actually had to stop and ask myself the question: “Well, why DO we want to automate test actions and verifications primarily at the sub-UI level?”

Here are my primary reasons:

• A verification being performed on the UI may not actually be verifying what the tester intended to verify.

  For example, if I want to verify that a folder monitoring process has swept a test file and read the data from the file correctly, verifying that the file name appears in a “Processed OK” list on a screen only tells me that the file name is appearing in that list – NOT that the file was processed correctly (whatever “correctly” means with respect to the AUT).

  Ask yourself the question “How can I confirm that X action produced Y results?”

• Verification that information and data are presented correctly in the UI doesn’t mean the data stored in the database is correct!

  I actually once found a bug where the UI populated the Last Name text box correctly, but this was because both the insert and retrieve stored procedures were referencing the Middle Name column. In other words, the Last Name data was being stored in and retrieved from the Middle Name column, so it “looked like” the Last Name data was being saved and retrieved correctly.

  Ask yourself the question “Where are all of the ‘outputs’ of action X?”

• (Hopefully) it is not the UI’s job to “do” most of the application processing, but rather to transfer information to and fro the business logic layer.

Ask yourself the question: “Am I testing the code that’s actually responsible for action X?”

There are other reasons commonly posited, such as:

• The UI is fragile and therefore UI tests are more likely to break
• UI tests are much slower than sub-UI tests (and if we have a lot of slow tests, we’re less likely to want to run them)
• If we have to automate a test via the UI, we have to wait until the UI is done, and typically the UI is the last component to be completed (even within a sprint/iteration)

Do I still automate UI tests? Yes, but typically only to confirm that UI behavior is as expected – e.g. buttons are enabled/disabled given certain circumstances, fields or data are hidden for certain users, etc. I also automate 1 or 2 “all the way through” tests – from UI to backend – but leave detailed business logic tests for the sub-UI suite.

Those are most of the reasons why I typically prefer to automate most integration/acceptance tests at the sub-UI level – but don’t take my word for it – the Google Testing blog has a thorough and interesting post on this same topic.

There we have it. My first blog post. Ever. Please, respond to the poll if appropriate and comment as you wish (unless you disagree with me). Just kidding of course.