Thursday, April 23, 2020

Cypress basics: how do cypress tests work?

Introduction

As I wrote earlier in a blogpost now we have cypress.io test framework integrated into Collabora Online codebase. Since we are testing an online, Javascript-based application that means we are working in an asynchronous environment. So triggering a user event via this framework, does not mean that this event is actually finished at the same time when the related cypress method is called. On the other hand, we try to cover real user scenarios, which are usually a sequence of events, as the user doing things on the UI step-by-step. So the question is, how to write sequential tests in an asynchronous environment. Fortunately, cypress has the answer to that.

The problem of asynchronicity

First, let's see an example of simulating user events in the cypress test framework. The following code triggers a selection of the text by simulating CTRL+A shortcut:

// Select all text
cy.get('textarea.clipboard')
 .type('{ctrl}a')

With calling this method we trigger a keyboard input, which will be handled by the client code (loleaflet). The client JS code sends a message to the server about the keyboard input. On the server-side, we will execute the shortcut, which results in a text selection. Then the server will send back the new text selection to the client-side, which will display it for the user. However the cy.get().type() call won't wait for that, it just triggers the event on the client-side and moves on.

Why is it a problem? Let's extend this code with another event:

// Select all text
cy.get('textarea.clipboard')
 .type('{ctrl}a')

// Apply bold font
cy.get('#tb_editbar_item_bold')
 .click();

Let's assume that this '#tb_editbar_item_bold' item exists. It's a toolbar button which applies bold font on the selected text. The problem here is that the selection is not finished yet when we already triggered the font change. Triggering these events does not mean that they will be executed in the specified order. Both events are handled in an asynchronous way, so if the first event (e.g. typing) takes more time to execute, then it will be finished later. Which means the application won't apply bold font on anything, because text selection is not there yet.

Event-indicator method

What is the solution here? What we need is the event-indicator method. It's very simple. After every event, we trigger in the test code, we should use an indicator that the event is actually processed. Let's see how we can fix the previous scenario:

// Select all text
cy.get('textarea.clipboard')
 .type('{ctrl}a')

// Wait for the text selection to appear
cy.get('.leaflet-marker-icon')
 .should('exist');

// Apply bold font
cy.get('#tb_editbar_item_bold')
 .click();

The '.leaflet-marker-icon' DOM element is part of the text selection markup in the application, so we can use that as an indicator. In cypress, every cy.get().should() call has an implicit retry feature. It tries to get the related DOM element, again and again, until it appears in the DOM (or until a given timeout). Also, it tries to check the assumption described by the should() method, again and again, until the DOM properties meet with this assumption (or until a given timeout). So this specific cy.get().should() call will wait until the selection appears on the client-side. After that, it's safe to apply bold font because we have the selection to apply on. It's a best practice to use an indicator after every event, so you can make sure that the events are processed in the right order.

Bad pracitice

Someone might think of using some kind of waiting after a simulated event, so the application has time to finish the processing. Like bellow:

// Select all text
cy.get('textarea.clipboard')
 .type('{ctrl}a')

// Wait for the text selection to appear (DONT USE THIS)
cy.wait(500);

// Apply bold font
cy.get('#tb_editbar_item_bold')
 .click();

I think it's easy to see why this is not effective. If this constant time is too small, then this test will fail on slower machines or on the same machine when it's overloaded. On the other side, if this constant is too big, then the test will be slow. So if we use a big enough value to make the tests passing on all machines, then these tests will be as slow as they would be on the slowest machine.

Using an indicator makes the test framework more effective. Running these tests will be faster on faster machines since it waits only until the indicator is found in the DOM, which is fast in this case. It also works on slower machines since we have a relatively big timeout for cypress commands (6000 ms), so it's not a problem if it takes time to apply the selection. All in all, we should try to minimize the usage of cy.wait() method in our test code.

One more thing: then() is not an indicator

It's good to know, that not every cypress command can be used as an indicator. For example, we often use then() method, which yields the selected DOM element, so we can work with that item in the test code. An example of this is the following, where we get a specific <p> item and try to check whether it has the specified text content:

cy.get('#copy-paste-container p')
 .then(function(item) {
  expect(item).to.have.lengthOf(1);
  expect(item[0].innerText).to.have.string('\u00a0');
 });

This method call will retry getting the <p> element until it appears in the DOM. However, it won't retry the assumptions specified with expect methods. It will check these expectations only once, when it gets the <p> item first. If we would like to wait for the <p> item to match with the assumptions, then we should use should() method instead.

cy.get('#copy-paste-container p')
 .should('contain.text', '\u00a0');

This method will retry to find a <p> item meet both the selector('#copy-paste-container p') and the assumption. In general, it's always better to use a should() method where it possible. However sometimes then() is also needed. We can use that safely if we use an indicator before it and so we can make sure we check the DOM element at the right time.

In the example below, we have a cy.get('.blinking-cursor') call. If we know that before the text selection we did not have the blinking cursor, then this cy.get() call is a good indicator so we can use then() after that safely.

// Select all text
cy.get('textarea.clipboard')
 .type('{ctrl}a')

// Use blinking cursor position
cy.get('.blinking-cursor')
 .then(function(cursors) {
  var posX = cursors[0].getBoundingClientRect().left;
  ... // Do something with this position
 })

Summary

I hope this short description of the event-indicator method and cypress retry feature is useful to understand how these cypress tests work. This simple rule, to always use indicators, can make things much easier when somebody writes new tests or modifies the existing ones. Sometimes the application does not provide us an obvious indicator and we need to be creative to find one.

No comments:

Post a Comment