It is quite normal for things to move quickly in software development. However, the faster you move, the more things break. As a codebase grows in size, its pieces become more and more complex, with every line adding a potential bug. In Wikimedia Foundation, we keep a handle on this through rigorous amounts of testing. Manual testing requires a lot of effort especially when you have a large core repository with a plethora of plugins and extensions that need to be tested. One of the hot frameworks on the scene is , a complete end to end testing solution.
In the past, web-app end-to-end testing has been a tricky beast. Good ol’ has been the main solution for quite some time now and has a huge history. Despite its impressive browser compatibility, having tests that are more consistent and less flakey can be a non-trivial matter mainly because Selenium was not designed for app testing. Which is what makes me super excited about Cypress, which not only promises to fix the old and broken ways of past framework but also make the entire process more developer centric. After spending over two months trying to migrate the tests written in to Cypress, I wanted to share my thoughts on my journey with Cypress thus far.
What Cypress brings to the Party?
End-to-end testing is notoriously known for its fragmented experience. You need to bring a lot of your own tools, for example, you need:
- a test runner
- an assertion library
- some reporters
- mocks, etc
Cypress includes all of these good stuff out of the box, making setup and configuration dead simple. At the same time, Cypress gives developers an immense amount of freedom when it comes to plugging in things of your own preference. In short,
Cypress takes a batteries included but removable approach
Documentation
A good software should be accompanied by a great documentation for it complete.
Cypress has one of the best documentation that I have ever read, with that you are likely to encounter. In fact, I remember when I was just starting out with Cypress, I immediately reached out to , my very close friend and mentor, for some good resources regarding Cypress and he immediately pointed me out to their official documentation. Not only do they do a great job telling you how to use the product, but have in-depth explanations on the , and .
Prototyping
If you have the chance, before adopting anything of this scale, I always think it’s a good idea to test it on a small project first, just to get a feel. Before advocating for it, I added it to my personal blog, just to see how the experience was.
A very simple scenario:
This was really as simple as writing a few lines of Javascript for the test itself, the npm script in package.json, and running it in TravisCI. I was blown away with how fast it took me, under an hour. I had toiled hard trying to . Not only did Cypress perform the assertions but also it was recording videos!
Page Object Model in Cypress
is an object design pattern in Selenium, where web pages are represented as classes, and the various elements on the page are defined as variables on the class. All possible user interactions can then be implemented as methods on the class.
Since well-named methods in classes are easy to read, this works as an elegant way to implement test routines that are both readable and easier to maintain or update in the future. Let us take an example of a page model for testing user account creation in WebdriverIO.
All you need to do is now call the CreateAccountPage.createAccount method with proper parameters in order to create a user account. Let us have a look at the Cypress version for this:
Writing Scenarios with Gherkin
If you have written end-to-end tests before, you may be familiar with Gherkin syntax, used by Cucumber. This is an expressive, English-like way to write test scenarios. It can help with documenting your features and non-developers can contribute to writing test cases. I found .
After writing these commands, the plugin will then go to Cypress to actually run the implementations:
Asserting Elements and Best Practices
End-to-end testing is all about making sure elements on the page have the right content. When writing Cypress tests, you will likely be spending 70% of your time selecting elements and peering inside them. Cypress has a standard command which exposes a jQuery-like selector to you. This should be familiar to those with some experience in Selenium. The problem with this selector is that it can be used incorrectly and there is no way to enforce (with code) it’s usage. Welcome , a wonderful tool maintained by a great testing advocate in the community, .
This plugin exposes a myriad of commands prefixed with find which work similarly to how get() does in native Cypress. All of these commands make for selectors . This can have a dramatic effect on how your tests stay consistent as your application progresses.
Time for Some Debugging
Debugging end-to-end tests with Selenium can be somewhat of a nightmare. With Cypress, this pain is at an all-time low. One of the biggest focuses of the core product is to be able to debug is one of the more pleasant experiences in your Cypress journey. Like for most things, .
Browser Support
Unfortunately, if your organization needs to have support for IE11, you are out of luck. . that I really hope you read through. It goes into why they are rolling this out slowly and didn’t choose WebDriver from the beginning and wrote their own custom driver. This is the only place where Cypress falls behind Selenium. However, it does support most of the Chromium-based browsers and is starting to roll-out support for Firefox too and this should serve the needs for most organisations.
I highly recommend everyone in the end-to-end testing community to give Cypress a shot. The quick setup, smooth documentation, fluid construction of tests and warm and supportive community will likely make you fall in love with it.