Encoding error while calling execjs

[huiyiqun]

Environments:

• docker image based on ruby:2.3
• ruby-babel-transpiler 2.7.0

I'm faced with an encoding issue when using jekyll-babel which depends on ruby-babel-transpiler:

root@1ac6426859a3:/data# ( cd /data/ && jekyll build -t )
Configuration file: /data/_config.yml
Source: /data
Destination: /data/_site
Incremental build: disabled. Enable with --incremental
Generating...
Conversion error: Jekyll::Converters::Babel encountered an error while converting 'static/js/help.es6':
"\xEF" on US-ASCII
/usr/local/bundle/gems/execjs-2.7.0/lib/execjs/encoding.rb:22:in `encode': "\xEF" on US-ASCII (Encoding::InvalidByteSequenceError)
from /usr/local/bundle/gems/execjs-2.7.0/lib/execjs/encoding.rb:22:in `encode'
from /usr/local/bundle/gems/execjs-2.7.0/lib/execjs/external_runtime.rb:8:in `initialize'
from /usr/local/bundle/gems/execjs-2.7.0/lib/execjs/runtime.rb:57:in `new'
from /usr/local/bundle/gems/execjs-2.7.0/lib/execjs/runtime.rb:57:in `compile'
from /usr/local/bundle/gems/execjs-2.7.0/lib/execjs/module.rb:27:in `compile'
from /usr/local/bundle/gems/babel-transpiler-0.7.0/lib/babel/transpiler.rb:24:in `context'
from /usr/local/bundle/gems/babel-transpiler-0.7.0/lib/babel/transpiler.rb:28:in `transform'
from /usr/local/bundle/gems/jekyll-babel-1.1.0/lib/jekyll/converters/babel.rb:24:in `convert'
from /usr/local/bundle/gems/jekyll-3.3.1/lib/jekyll/renderer.rb:109:in `block in convert'
from /usr/local/bundle/gems/jekyll-3.3.1/lib/jekyll/renderer.rb:107:in `each'
from /usr/local/bundle/gems/jekyll-3.3.1/lib/jekyll/renderer.rb:107:in `reduce'
from /usr/local/bundle/gems/jekyll-3.3.1/lib/jekyll/renderer.rb:107:in `convert'
from /usr/local/bundle/gems/jekyll-3.3.1/lib/jekyll/renderer.rb:86:in `run'
from /usr/local/bundle/gems/jekyll-3.3.1/lib/jekyll/site.rb:463:in `block in render_pages'
from /usr/local/bundle/gems/jekyll-3.3.1/lib/jekyll/site.rb:461:in `each'
from /usr/local/bundle/gems/jekyll-3.3.1/lib/jekyll/site.rb:461:in `render_pages'
from /usr/local/bundle/gems/jekyll-3.3.1/lib/jekyll/site.rb:191:in `render'
from /usr/local/bundle/gems/jekyll-3.3.1/lib/jekyll/site.rb:69:in `process'
from /usr/local/bundle/gems/jekyll-3.3.1/lib/jekyll/command.rb:26:in `process_site'
from /usr/local/bundle/gems/jekyll-3.3.1/lib/jekyll/commands/build.rb:63:in `build'
from /usr/local/bundle/gems/jekyll-3.3.1/lib/jekyll/commands/build.rb:34:in `process'
from /usr/local/bundle/gems/jekyll-3.3.1/lib/jekyll/commands/build.rb:16:in `block (2 levels) in init_with_program'
from /usr/local/bundle/gems/mercenary-0.3.6/lib/mercenary/command.rb:220:in `block in execute'
from /usr/local/bundle/gems/mercenary-0.3.6/lib/mercenary/command.rb:220:in `each'
from /usr/local/bundle/gems/mercenary-0.3.6/lib/mercenary/command.rb:220:in `execute'
from /usr/local/bundle/gems/mercenary-0.3.6/lib/mercenary/program.rb:42:in `go'
from /usr/local/bundle/gems/mercenary-0.3.6/lib/mercenary.rb:19:in `program'
from /usr/local/bundle/gems/jekyll-3.3.1/exe/jekyll:13:in `<top (required)>'
from /usr/local/bundle/bin/jekyll:17:in `load'
from /usr/local/bundle/bin/jekyll:17:in `<main>'

After hours of debuging, I find that the problem is at here:

    def self.context
      @context ||= ExecJS.compile("var self = this; " + File.read(script_path))
    end

It reads babel.js with File.read directly without any paramters, which return a string with encoding US-ASCII, while the correct encoding is UTF-8 indeed. Then execjs treats the string as a US-ASCII encoded string and trys to transcode it to UTF-8 so it fails because of non-ASCII characters.

The solution is to add a parameter to File.read according to the doc:

    def self.context
      script = File.read(script_path, :encoding => 'UTF-8')
      @context ||= ExecJS.compile("var self = this; " + script)
    end

I have little knowledge about ruby, please correct me if there are some mistakes.

[huiyiqun]

Hi, I need help.

[huiyiqun]

@josh @tricknotes

[dawaltconley]

Running into this same problem trying to build with jekyll-babel within an AWS Codebuild environment (same docker image). What @huiyiqun said seems right, would very much love a fix.

[lukepighetti]

Added jekyll-babel package to jekyll on macOS, getting this exact same error, no resolution found yet. Any ideas?

[dawaltconley]

It's possible that when I ran into this, I either wasn't using a Gemfile or I hadn't specified encoding in my Gemfile... I eventually ended up switching to gulp, but try adding to the top of your Gemfile: Encoding.default_external = Encoding::UTF_8

[spencertweedy]

Seeing the same issue, but only on DigitalOcean, not on my local machine…