Issue when using unicode in variables

[paroxp]

What

I stumbled upon an issue, where a unicode character that is in a variable, will not be compiled properly, but turned into a square instead.

It's particularly annoying, when working with sprites or something like font-awesome.

Test case

package main

import (
	"bytes"
	"fmt"
	"strings"
	"testing"

	libsass "github.com/wellington/go-libsass"
)

func TestUnicodeCharInVar(t *testing.T) {
	var dst bytes.Buffer

	src := bytes.NewBufferString(`
$test: "\f00d";

p {
	a {
		content: $test;
	}

	b {
		content: "\f00c";
	}
}
`)

	comp, err := libsass.New(&dst, src)
	if err != nil {
		t.Error("Expected to create compiler successfully")
	}

	err = comp.Run()
	if err != nil {
		t.Error("Expected to run the compiler successfully")
	}

	css := dst.String()

	if !strings.Contains(css, "\\f00c") { // This one is passing
		t.Errorf("Expected to find `%s` in compiled css", "\\f00c")
	}

	if !strings.Contains(css, "\\f00d") { // This one isn't
		t.Errorf("Expected to find `%s` in compiled css", "\\f00d")
	}

	fmt.Println(dst.String())
}

Output

→ go test custom_test.go
@charset "UTF-8";
p a {
  content: ""; }

p b {
  content: "\f00c"; }

--- FAIL: TestUnicodeCharInVar (0.00s)
        custom_test.go:46: Expected to find `\f00d` in compiled css
FAIL
FAIL    command-line-arguments  0.005s

[drewwells]

That is annoying. I don't believe go-libsass is causing this issue, but I'm taking a look

[drewwells]

Also thanks for posting a fully working, reproducible unit test. You are the best 💯 🎆

[drewwells]

So interesting, the Ruby Sass maintainers seem to be resistant to address this problem which cropped up in a minor release of 3.4. You can force the expected ascii output by doing this: https://www.sassmeister.com/gist/22743e406eeb3744988c

If possible, I'd recommend moving away from storing escaped unicode in variables. It's unlikely Sass will remedy this issue. There's no work been done to let developers dictate the encoding of Sass output.

Long story here: sass/sass#1395

[drewwells]

This may work as well, content: unquote("\\")+$contentvar;. Either way it requires removing the unicode prefix from all variables.

[paroxp]

Nice research work. Thanks :)

It's weird, as the same code worked totally fine with gulp/grunt. I doubt that the need of modifying an external library (like font-awesome) will be a common thing for people.

I've worked around the issue by not using font-awesome, which I couldn't really justify.

I'm happy for the issue being resolved, as it is clearly not related to the package itself.

Thanks again for investing time in this!

[kimek]

@drewwells
content: unquote("\\")+$contentvar;
works however
content: unquote("\"")+unquote("\\")+$contentvar+unquote("\"");
doesn't. Is there any chance to turn off UTF-8 in sass?
As I see from sass/sass#1395 there is no ASCII 'force mode' yet ;/