Running UI tests in parallel mode

At Picscout, we use automated testing.

Running tests is an integral part of our Continuous Integration and Continuous Deployment workflow. 
It allows us to implement new features quickly, since we are always confident that the product still works as we expect.

The automation tool that we are using is Selenium (Selenium is an open source set of tools for the automatic running of browser-based applications).

Despite the many benefits of Selenium, there is one drawback that constitutes a bottleneck: the running time of the tests.

For example, in one of our projects, we need to run 120 UI tests, which takes 75 minutes - a long time to wait for a deployment.

To handle this situation, we developed a tool that runs the tests in parallel mode.

Using this tool, we were able to reduce the run time to 12 minutes of tests.

How it works:

Two tests can be run in parallel mode as long as they are not affected by each other.
We avoid writing tests that use and modify the same data, since they cannot be run in parallel.

The assumption is that two tests can be run in parallel mode if each test has a different id.

When writing a new Selenium test, we add a “Test Case” attribute to the test.

The attribute will be the id of the test.

The tool reads the Selenium project dll file,

Separates the tests into different queues according to their test cases and each queue runs in parallel mode.

13:43:51          Parallel Nunit Console

13:43:52

13:43:52  - Number of parallel queues: 6, number of tests: 29

13:43:52

13:43:52  - Number of serial queues: 1, number of tests: 3

The tool runs 10-15 threads of tests, instead of one thread in serial mode (one by one).

There are two more options in the tool. 

The first option is "update local DB." 

This creates or updates the local DB (a minimized DB, just for running UI tests). 

The second option is for the UI tool to run the tests in parallel mode. 

These two features allow us to run the tests on a developer’s station before the code is pushed, and on our build machine after the code is pushed.

That's how we run UI tests these days.