Cypress Code Coverage with Svelte or Sapper

It’s been quite a few months since the last post ; right now I wanted to show you and explain the little updates I did on this blog’s code, starting with code coverage

2022 EDIT: code coverage is currently broken, so this content might be deprecated

Today, we’ll see how Code Coverage report was added to this blog, which is based on Sapper / Svelte.

In an earlier post, I showed how TDD is a really great programming practice.

An added benefit, or side effect, of TDD is having high code coverage. Because you start writing test and then code to make them pass, you end up testing almost all of your code, if you can keep the discipline.

What is Code coverage?

Code coverage is a metric that shows how much of your code is being covered by tests.

It does not quantify the quality of your tests, so you might just pass over all your code without really testing anything, so it is not an absolute metric, but it shows that at least you have written some tests. To me, the most important part is to show which code is not tested at all.

In a future post I will talk about mutation testing, but at the moment we’ll focus on code coverage.

results

When I run npm run test, a coverage directory is generated by the Cypress plugin that looks at which code was executed during the tests and correlates it with the lines of codes.

Our code coverage report looks like this:

We are using Istanbul and its command-line interface NYC.

When you click on a line on this report, eventually you end up looking at a file of the repository, such as a Svelte component:

You can see that the highlighting is a bit off, but it still gives you some hints as to which logic path to add test for.

Yes, repeating myself, the purpose of this code coverage report, is to know which code has not been executed in a test.

So you know what new tests to write.

This will happen if you practice TDD at a higher level of abstraction (like this blog), or if you do not practice TDD at all.

How to measure code coverage?

This blog has a lot of functional tests and a few other kinds of tests. Everything is run via Cypress test runner. At the end of each test, the code covered is reported and merged with the existing code coverage report, which means that if we had another test runner for other kinds of tests, we could have merged both reports together.

Let’s review our existing tests:

• links-in-markdown-titles is testing that the marked library works as we expect. It is an essential part of this blog as it compiles markdown content into pages, which makes writing posts a little bit easier to me. It does not call cy.visit so it is not testing the web application.
• uptime-api is a special case because it really calls an API, testing that it answers conforming to our expectations. Remember? this test was added to test the UptimeRobot API. It does not call cy.visit either.
• the other tests are end to end functional tests. When you read the report, it should read like a backlog of requirements. I first wrote them during the migration process from Gatsby to Sapper to compare both blogs. These tests begin with or at some point call cy.visit to test some part of the web application inside a browser.

So we already have different kinds of tests, and thus running Cypress will check all this.

an idea from 2019

Last year, I discovered the code coverage capability for Cypress, and looked for Svelte/Sapper support.

Unfortunately, the timing was not right and the sample repository was in a Work in progress state with an issue describing the problem.

So I put this idea aside for a while — well, one year.

Last week, out of curiosity I remembered the issue and checked if it had been solved.

The maintainer, Gleb Bahmutov, told me he would love a pull request, which gave me a little bit of energy to accept the challenge. I might even say, that thanks to this, I’m back on this blog writing this article.

As it happens it was only a matter of updating a dependency, so it was not as complex as I thought it would be. Sometimes, you just need to dive in, start doing it and discover that you can do it.

So I repaired the Svelte + Cypress code coverage sample repository, made a Pull Request to fix the issue. Until it is merged, you can see the fork here: https://github.com/doppelganger9/coverage-example-svelte

Based on this example, I then forked and created a sample Sapper template: https://github.com/doppelganger9/sapper-template/tree/rollup+cypress+coverage

After this, I thought I could also add it to this blog! I did not plan it at the beginning, I mean, not consciously. And yet here we are

To achieve this, I needed to align the blog to this Sapper + Rollup template:

• updated package.json, cypress plugins, commands, config, dependencies, etc.
• fixed randomly failing blog-post Cypress test (I had to write a cy.wait )
• Added forgotten status component error handling HTTP errors 500 and 404
• replaced Webpack with Rollup
• added data-cy everywhere
• changed the posts list so that the entire post is a link, no just the title
• added TS definitions intelli-sense for Cypress command and comment to enable it in the spec files.

You can check the related commit on this blog repository or the linked sample Svelte or Sapper template repositories. If you are using Webpack, I did not yet manage to make this work, that’s why I switched back to Rollup .

Coveralls GitHub Action

Coveralls is a web service to help you track your code coverage over time, and ensure that all your new code is fully covered.

It is free for Open Source projects such as public GitHub projects. I have been using it for other side-projects such as a Mastermind Java Kata, a CQRS/ES Node + TypeScript API for Games of Table Football/Soccer and a JavaScript Fruit Basket Kata (This one also has mutation testing… spoilers! ).

So now that my build script generates a code coverage report, I can add this blog repository to Coveralls.

There is a Coveralls GitHub Action which I added to my YAML workflow file:

    - name: Coveralls
      uses: coverallsapp/github-action@master
      with:
        github-token: ${{ secrets.GITHUB_TOKEN }}

I also removed the Node version matrix which is kind of irrelevant to have for my blog.

Now, after the app is built and tested, the coverage report is generated, and then sent to Coveralls service.

You can see it here: https://coveralls.io/github/doppelganger9/blog

Conclusion

Thank you for reading, It is a pleasure to share with you and I hope this post will be useful.

It was my first 2020 post as I intend to write a few more, and we saw what is Code Coverage, examples for Svelte or Sapper apps, and of course how I used this for this blog. All this just to get a badge .

I did not go into the details on how to configure this, the Cypress documentation contains lots of samples, articles, anyway, if you are interested in them, tell me, I will do a follow-up post.

I will have a future post about Mutation Testing.

If you have any questions or feedback, or just want to say hi or thank you, my DMs are open on Twitter!