Test Automation

I was recently contacted to write an article for a devops online which has been published here but i wanted to post same article on my blog. This is by no means a detailed tutorial but i hope it gives a very quick overview on Continuous Integration and Continous Delivery. Software development is usually undertaken by one software development team or a number of software development teams. Each software development team would usually consist of developers, testers and business analysts.

I have written in my previous blog about how we need to distribute our tests across the testing pyramid for reasons which i have explained in the previous blog. This particular post is focussed on how we ought to design our tests across the higher levels of abstractions which includes the UI test and API tests. For the purpose of the blog, I would be using a case study of team that writes its UI Acceptance tests in protractor (Javascript) and the API tests in Java using RestAssured library.

In this post, I would like to consider how to write automated tests for negative scenarios for a system built using an asynchronous / event driven architecture. In my previous post, i have mentioned, that these systems are peculiar because the effect of the “write” actions are not immediately stored in the persistence layer of the application under test. As an example, in the current system which i have been testing on the write side of the application, the write commands traverses though the Command API layer, the command controller layer and the command Handler layer before it ends up in the Event Store.

In recent months i have been testing a system built using the CQRS pattern. CQRS stands for Command Query Responsibility Segregation and you can read more about it here. As a tester, one of the key takeaway for me is that there is a Read side and a Write side to the application. This is very different to the other applications that i have tested in the past where a write operation would NOT be successfully until the data being sent to the application is successfully persisted.

So I have used quite a number of tools in my time building test automation frameworks over the years; and in the last 2years i have evaluated a number of these tools and i am becoming more and more opinionated in my choice of tools. For someone that has used selenium/webdriver and cucumber a lot in the past, i find myself using Geb and Spock these days and it hasnt been much difficulty in switch to these tools.

This post is spurned by taking a reflection on my journey as a tester and it’s amazing to see how much i have moved from designing large UI test suites to very small UI test suites. For emphasis, the largest test suite had taken over 8 hrs and the smallest has been 3mins of execution time. There are a lot of posts on the internet about how fragile a UI test framework could be and i would expect the next question to be how have you been able to achieve this.

I ran into a problem today where some french and spanish characters were being displayed strangely. And this made my test to fail, After some poking around in google i can across a post that suggested adding require ‘cucumber/formatter/unicode’ to the env.rb and that seem to have sorted the problems i was having. Hopes this saves you some time.

In my last blog post, i described how i have used jmeter-plugins at my current client site, now i am faced with another problem , i need to be able to run these performance test from command line, in other for these performance tests to be executed in a CI environment such as teamcity. A quick search brings to light the ant jmeter task which is an ant task for running jmeter test.

Jmeter is a well known open source performance/load testing tool and to be fair it does a lot of stuff really well. if you wants to do some quick performance testing without a whole lots of infrastructure around it then it is great. I started out my task with jmeter with some objectives: <li> A tool that i could integrate into a CI tool such as teamcity</li> <li> Meaningful graphs that could be easily interpreted by any one in the team</li> <li> Able to integrate the graphs into teamcity</li> <li> Able to monitor the performance of the website under test.

In my the few years, where i have pushed for testers developers to write acceptance test before development or implementation commences, I’ve had testers come up to me say its difficult to achieve this when you dont know what the UI would look like. This post would attempt to decribe how i have done this in the past and i still do this at the moment. NB: Ideally i would write my test using page object pattern, if this was a new project, I would start creating page objects