The Testing Blog

Here it is. My first post. (a.k.a. Why automate sub-UI?)

Posted on August 14, 2009. Filed under: test automation |

As someone who has extremely high standards for oneself, I have majorly procrastinated in writing my first post here on thetestingblog.com. Tonight I’ve decided to go ahead and do it already. If the information is not earth-shattering; if my wit and humor do not make you literally laugh out loud; if (gasp!) you do not agree with me – that’s OK. I have come to terms with that.

Now on to the real topic…I’ve been working with a software development team to guide them in their implementation of agile and automated testing techniques for about a month now. At the first sprint demo (yes – the team practices Scrum), I wanted to show the team a couple of examples of simple automated tests I had created for one of their applications. I demonstrated a GUI-driven test and a sub-GUI test. The demo went off without a hitch and the team seemed eager to start automating their tests in these manners.

The next day, I was to conduct a more detailed meeting with the testers regarding my suggestions for implementing automated tests to cover a particular application. Again, all was going well until I said something (I thought to be) pretty benign – something along the lines of “Well, I’m hoping that the architecture of the app is such that most of these test actions and verifications can be performed sub-UI.”

I heard brakes screech.

All of a sudden, a debate ensued about how driving the application some other way than through the UI was not a “real” acceptance test (after all, “real” users have no choice but to use the application via the UI). I stumbled and fumbled to explain; I was so used to automating 90% of a test suite at the sub-UI level that I actually had to stop and ask myself the question: “Well, why DO we want to automate test actions and verifications primarily at the sub-UI level?”

Here are my primary reasons:

• A verification being performed on the UI may not actually be verifying what the tester intended to verify.

  For example, if I want to verify that a folder monitoring process has swept a test file and read the data from the file correctly, verifying that the file name appears in a “Processed OK” list on a screen only tells me that the file name is appearing in that list – NOT that the file was processed correctly (whatever “correctly” means with respect to the AUT).

  Ask yourself the question “How can I confirm that X action produced Y results?”

• Verification that information and data are presented correctly in the UI doesn’t mean the data stored in the database is correct!

  I actually once found a bug where the UI populated the Last Name text box correctly, but this was because both the insert and retrieve stored procedures were referencing the Middle Name column. In other words, the Last Name data was being stored in and retrieved from the Middle Name column, so it “looked like” the Last Name data was being saved and retrieved correctly.

  Ask yourself the question “Where are all of the ‘outputs’ of action X?”

• (Hopefully) it is not the UI’s job to “do” most of the application processing, but rather to transfer information to and fro the business logic layer.

Ask yourself the question: “Am I testing the code that’s actually responsible for action X?”

There are other reasons commonly posited, such as:

• The UI is fragile and therefore UI tests are more likely to break
• UI tests are much slower than sub-UI tests (and if we have a lot of slow tests, we’re less likely to want to run them)
• If we have to automate a test via the UI, we have to wait until the UI is done, and typically the UI is the last component to be completed (even within a sprint/iteration)

Do I still automate UI tests? Yes, but typically only to confirm that UI behavior is as expected – e.g. buttons are enabled/disabled given certain circumstances, fields or data are hidden for certain users, etc. I also automate 1 or 2 “all the way through” tests – from UI to backend – but leave detailed business logic tests for the sub-UI suite.

Those are most of the reasons why I typically prefer to automate most integration/acceptance tests at the sub-UI level – but don’t take my word for it – the Google Testing blog has a thorough and interesting post on this same topic.

There we have it. My first blog post. Ever. Please, respond to the poll if appropriate and comment as you wish (unless you disagree with me). Just kidding of course.

Make a Comment

9 Responses to “Here it is. My first post. (a.k.a. Why automate sub-UI?)”

Comments RSS Feed

Where's The Comment Form?