To make sure their software works well, many companies still rely on manual testing only. Not to mention the worst case when they don't test at all relying on users to find problems. Is it expensive? Yes. Is it slow? Yes. It can block tasks in the testing phase for weeks delaying the release of new features. Especially, if a person responsible for testing is busy with other tasks. And it becomes even worse in the long run. The more features you add, the more time you need to test everything. So it's not scalable at all. What else could go wrong? Let's think about the human factor. We can't guarantee that a person will test everything the same way every time. They could be distracted or just tired. So people invented unit tests, integration tests, and end-to-end tests to automate the process and defined the pyramid of tests. The biggest part of the pyramid is unit tests. And we may falsely think that unit tests are more important ignoring the rest. But the truth is end-to-end tests are the most valuable.

Key Benefits

In that sense, automated acceptance tests could be considered as a subset of E2E tests. Here are the benefits:

  • 🎯 Ensures the app meets user expectations: We define the scenarios followed by the user and expectations.

  • ✅ Increases confidence in software quality: We write tests once and run it many times on a large scope of devices or browsers.

  • 🔄 Facilitates continuous integration and delivery: We trigger test runs on each commit or PR.

  • 🔍 Enhances test coverage and efficiency: We're reusing steps involved in different scenarios. We're also confident regarding regressions.

  • 🏃‍♂️ Supports agile and DevOps practices: This one is self-explanatory. We're applying changes quickly and continuously.

  • 💰 Saves time and money: We don't need expensive resources to run tests. We get quick feedback after changes.

Keeping Things Simple

To get these benefits, it makes sense to start with happy paths for the most critical features. And we could keep scenarios simple and easy to understand following the KISS principle. We probably should leave the most complex and hard to automate scenarios for manual testing.

How could we implement it?

So here is what tools I'm using and why:

  • Node.js/TypeScript: I prefer Node.js for tests because of its learning curve and flexibility.The issue with tests written natively using Swift (XCUITest) or Kotlin (UIAutomator) is that you need to have people who know all these languages and frameworks in order to maintain it. And you need to write it at least twice. It's not as scalable. Why TypeScript if I could use JavaScript? Because of strong typing. It gives us feedback about the code before we run it, during compilation time.

  • Cucumber: If you ever worked with tools like TestRail or Qase, you probably know about test cases. Cucumber allows us to transform these test cases into code, acting as a bridge between business and development teams. It gives more people the chance to see what's being tested and how. This is probably also the main reason why I prefer Node.js over Swift or Kotlin. There are no good tools (at least for Swift) to write tests in Gherkin. We could reuse steps from a Gherkin scenario in other ones. There are even plugins and extensions for various IDEs to autocomplete these steps and when the amount of implemented steps grows, there is a possibility to write scenarios even without knowledge of a programming language. Isn't that cool?

  • WebdriverIO and Appium: When it comes to testing native mobile apps, Appium is probably the only one well-known framework we could use. It's an implementation of the WebDriver protocol. There are various wrappers around it. Currently, I'm using WebdriverIO. Why? Because it's recommended by the Appium team. Would I stick with it in future? I don't know. But it does the job. I'm tempted to try Nightwatch.js in production to compare these two.

  • Mockoon: Often there are some parts of the app which are hard to automate because of external dependencies. For cases with APIs, I'm using Mockoon. It allows quick setup of mock servers. And there is a CLI version which integrates nicely with the TypeScript code. It's possible to edit the response either in the app or in the code. And it can be shared with the team. In the end it's a nice tool to reduce the time spent on asynchronous tasks.

What else?

It makes sense to run such tests on some CI/CD platform in order to unblock the development process. I'll write more about it in the future.

I also plan to write about the process of setting up the environment and writing the first test. So stay tuned!