Testing SASS function and mixin errors with True

Recently I wanted to try out some SASS testing, and came across the True framework by Miriam Suzanne. It's a fairly small and straightforward framework to learn, but it's not entirely obvious how to use it to test error output since neither True nor the SASS language support catching errors. Here's a write-up of my approach.

The main reference point for testing errors is the tests for the Susy library (further explained in this issue). By passing all function errors through the _susy-error() function, they can either be thrown as regular errors or returned as strings based on the state of $_susy-error-output-override:

@function _susy-error(
  $message,
  $source,
  $override: $_susy-error-output-override
) {
  @if $override {
    @return 'ERROR [#{$source}] #{$message}';
  }

  @error '[#{$source}] #{$message}';
}

This is then simple to implement in your code, by replacing calls to @error 'message'; in your functions with calls to your error-wrapping function _susy-error('message');.

Mixins present more of a challenge, since they have to return something CSS-like. This isn't something that Susy provides a template for, since it's primarily function-based, so I've come up with two basic approaches. There may be more!

Returning an error value

The limitation of mixins is that they have to return something CSS-like, so you can't use the same function-based approach. Instead, you can use a mixin to return a CSS block containing the error:

$_is-test: false !default;

@mixin _error($error, $override: $_is-test) {
  @if $override {
    error: $error;
  }
  @else {
    @error $error;
  }
}

@mixin my-mixin($colour) {
  @if $colour == 'blue' or $colour == 'orange' {
    color: #{$colour};
  }
  @else {
    @include _error('This is not a valid colour.');
  }
}

Your True test will then look something like this:

$_is-test: true;

@include describe('mixin test') {
  @include it('Checks for an error') {
    @include assert {
      @include output {
        @include my-mixin('borange');
      }

      @include expect {
        error: "This is not a valid colour.";
      }
    }
  }
}

Setting a global error message

Alternatively, instead of returning anything from the mixin you can set a global state variable. In this case, your error mixin will look more like:

$_is-test: false !default;

$global-error: '';

@mixin _global-error($error, $override: $_is-test) {
  @if $override {
    $global-error: $error !global;
  }
  @else {
    @error $error;
  }
}

@mixin my-mixin($colour) {
  @if $colour == 'blue' or $colour == 'orange' {
    color: #{$colour};
  }
  @else {
    @include _global-error('This is not a valid colour.');
  }
}

Your True test will then look something like this:

$_is-test: true;

@include describe('mixin test') {
  @include it('Checks for an error') {
    @include my-mixin('borange');
    @include assert-equal($global-error, 'This is not a valid colour.');
  }
}

The error-checking logic can be refactored into a separate assertion mixin if desired:

@mixin assert-raises-error($error) {
  @include assert-equal($global-error, $error);
}

On balance I prefer this approach to returning error values since there's explicit separation between values returned from the mixin and error messages, and the tests are a little more compact.

The downsides

Because of the way that both of these methods work, you have to have a single point of return for errors. If it's possible to return multiple errors, you'll see incorrect values.

It would be possible to work around this in the global variable scenario by making $global-error append values to a list rather than update a string—then you could instead handle the case where multiple errors are returned, even though this wouldn't strictly be possible in the conventional error flow.

The other way to approach this, as suggested by Miriam Suzanne, would be to make your SASS largely function-based, using mixins sparingly. This is effectively the approach Susy takes, which has worked well for them.

Appendix

For further information on alternative error-handling approaches in SASS see this article by Hugo Giraudel, which I found helpful in playing around with these ideas.