Visualize your test results

Me, my team and I use a combination of  NUnit and Selenium for frontend testing and we are running them every night in TeamCity so that every morning we can enjoy fresh report with the latest bugs, errors, and mysteries, of course.

TeamCity knows how to visualize test results for unit tests and it looks just fine:

But how does it looks like for frontend?

Pretty much the same – right?

Yes, for the “green” results. What about “red” results? Do they look similar?

Unittest, pretty straightforward – response.StatusCode should be HttpStatusCode.OK but was HttpStatusCode.UnsupportedMediaType or in other words excepted result was HTTP code OK but was something else. What about front-end test?

Ok, the test failed because Selenium could not locate the element by ID selector. But why it could not find it? Maybe element is missing or it was hidden behind another element? How can I tell?

Screenshots will save you!

Our tests create screenshots after every failure and stores them in the folder that we publish for usage in case of failure (so we can find out the reason for failure, of course).

Okei, so now I must find the right screenshot from a bunch of others and remember test name that failed so I can pick specific screenshot taken and open it in new tab to find the problem and as this sentence grows longer so does the process for finding the problem. Ugh…

Visualize results

To get rid of jumping between screenshots and test results we need to combine it into one nice report.

At first, I was ready to “put together” report by myself, but then I remembered that I am lazy and in a few google searches I found something interesting – ExtentReports (reporting library). Let’s try it.

Spending quite some time trying to get it to work and refactoring my current solution and failing and refactoring and then failing again I finally got something working and my first report:

Okei, now few more fixes (screenshots not showing, tests overwriting each other, tests can’t be run in parallel etc) and we are ready to go. Now we need to put it all together and let’s see how our nightly report looks like now:

Ahh, like a fresh breath of air.

We have our screenshots, we have our console errors (if there is any), we have stacktrace and we can even filter the report to show only “red”/”green” and most importantly – we reduced the time needed to collect necessary information. We have eliminated the need to go to our build server and check for results in a “messy” way. We introduced a much more friendly way to show the information we need so we can focus on fixing the problem, not wasting time collecting the information about it.

Implementation:

This is how the basic test looks like.

Methods:

Red square – ExtentReports(), green – _extent.CreateTest(testSuite), blue – extentTestSuite.CreateNode(testName). Keep in mind, that if you want to create multiple nodes inside same testSuite, you must create them inside same testSuite object.

 

Leave a Reply

Your email address will not be published. Required fields are marked *