Rails 7, fails to yarn install, command 'vite' not found, workaround

[jrochkind]

Is anyone else running vite-rails with Rails 7? I think maybe not yet?

Because I ran into a problem upgrading my Rails 6.1 app using vite-rails to Rails 7.0. Which I have researched some background on, and can provide a workaround and suggestion for vite-rails fix.

In my Rails 6.1 app, it turns out that Rails will do a yarn install as an initial part of rake assets:precompile. This is necessary, it installs vite itself, and all your other yarn npm dependencies.

In Rails 7.0 -- this stopped happening. So if deploying to a fresh machine, rake assets:precompile will get you an error that may look something like this:

Running: rake assets:precompile
Building with Vite ⚡️
yarn run v1.22.19
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command "vite" not found.

Diagnosis and history

Rails 6.1 wires up the asset:precompile rake task to do a yarn install as a rake pre-requisite, if a ./bin/yarn is detected:

if Rake::Task.task_defined?("assets:precompile") && File.exist?(Rails.root.join("bin", "yarn"))
  Rake::Task["assets:precompile"].enhance [ "yarn:install" ]
end

—https://github.com/rails/rails/blob/v6.1.7/railties/lib/rails/tasks/yarn.rake#L27-L30

But Rails 7.0 removes this, with this commit message:

Since yarn isn't required for Rails 7, assets:precompile should not depend on it by default. Instead, tasks for webpacker, jsbundling-rails, cssbundling-rails, etc should depend on yarn:install on their own.

— rails/rails@3e2f74c

What does jsbundling-rails do? Something very similar to what Rails 6.1 did:

namespace :javascript do
  desc "Build your JavaScript bundle"
  task :build do
    unless system "yarn install && yarn build"
      raise "jsbundling-rails: Command build failed, ensure yarn is installed and `yarn build` runs without errors"
    end
  end
end

if Rake::Task.task_defined?("assets:precompile")
  Rake::Task["assets:precompile"].enhance(["javascript:build"])
end

— https://github.com/rails/jsbundling-rails/blob/98780f28809ebad5d84921ce38dc238472a35812/lib/tasks/jsbundling/build.rake

What does webpacker do? The last non-prerelease version of webpacker, 5.4.3, also does it's own thing with it's own rake task for yarn install, wiring it up as a pre-req to assets:precompile:

unless skip_webpacker_precompile
  if Rake::Task.task_defined?("assets:precompile")
    enhance_assets_precompile
  else
    Rake::Task.define_task("assets:precompile" => ["webpacker:yarn_install", "webpacker:compile"])
  end
end

— https://github.com/rails/webpacker/blob/v5.4.3/lib/tasks/webpacker/compile.rake#L37-L42

The webpacker 6.x pre-releases change the code somewhat, but similar approach... although this time using a yarn:install task -- webpacker 6.x actually provides yarn:install task, but with a comment: "Duplicate of the yarn tasks still present in Rails until Webpacker <5 have been deprecated."

# Run Yarn prior to Sprockets assets precompilation, so dependencies are available for use.
if Rake::Task.task_defined?("assets:precompile") && File.exist?(Rails.root.join("bin", "yarn"))
  Rake::Task["assets:precompile"].enhance [ "yarn:install" ]
end

— https://github.com/rails/webpacker/blob/dc1482790326ca077577dc957b0be068724b9c7c/lib/tasks/yarn.rake#L35-L38

Local workaround

My app still has a ./bin/yarn (I am not sure if this is reliable for all Rails 7 apps, even all of them using yarn?). And Rails 7 still has a yarn:install task, it just isn't by default called by assets:precompile.

So I sucessfully fixed this issue simply by adding these lines to my local ./Rakefile, copied from what Rails 6.1.x used to do:

if Rake::Task.task_defined?("assets:precompile") && File.exist?(Rails.root.join("bin", "yarn"))
  Rake::Task["assets:precompile"].enhance [ "yarn:install" ]
end

Bingo, solved.

What should vite-rails do?

I think vite-rails should probably be wiring up a yarn install as a rake pre-req of assets:precompile, as jsbundling-rails and webpacker do, as the comment when Rails 7 removed the pre-req suggested.

Although maybe you don't technically need to use yarn to use vite and vite-rails? I'm not sure if that is meant to be supported -- if so, I guess it should only do a yarn install if yarn is present... but I think it should still do so if present, for an easy path Just Works path for people on the golden path of Rails. Checking for presence of ./yarn.lock might make more sense than checking for presence of ./bin/yarn, as you might not have a local yarn in bin? (Although technically you might not also (yet) have a yarn.lock... but you really probably should when running rake assets:precompile I think?)

But then how do we do the yarn install? We want to ensure a yarn install, but also want to avoid a double yarn install.

What if you are using vite-rails with Rails 6.1? It already does a yarn install, then vite-rails also wires it up... now you get it both?

That can be avoided if vite-rails does exactly what I do, the rake enhance with existing yarn:install. enhance should be idempotent in the sense that we do Rake::Task["assets:precompile"].enhance [ "yarn:install" ] as many times as we want, yarn:install will only be called once.

I don't really understand how jsbundling-rails or webpacker are avoiding this when used on Rails 6.1 though... since they don't use the existing yarn:install but appear to provide their own rake tasks? When used on Rails 6.1, are people getting a double yarn install? I'm not sure, I can't find anyone complaining about it, haven't tried to verify myself.

(For that matter, how do they avoid jsbundling-rails and cssbundling-rails each doing their own yarn install, being a double even on rails 7?!?!)

And wait, what if someone is using vite-rails and also has webpacker, jsbundling-rails, or both installed -- on Rails 6.1 or 7? Then, if vite-rails were also wiring up a yarn install, you could still be getting 2, 3, or more yarn installs happening. :(.

So I'm not totally sure what to do. I suppose it could just be a doc issue where vite-ruby/rails tells someone to do this themselves in Rails 7 only if they need it... but that seems not very friendly to me, it's pretty confusing to figure out if you need it. I think it would be best if vite-rails does it's best to automatically wire up the yarn install if and only if it's needed, but maybe with config to override in either direction if it gets it wrong?

@ElMassimo , any thoughts?

[ElMassimo]

Hi Jonathan!

Vite Ruby extends assets:precompile to run vite:install_dependencies, which runs npx ci, which in turn install dependencies using your preferred package manager instead of assuming yarn.

As a result, Vite Ruby is compatible with npm, pnpm, and yarn.

In summary, if you haven't set VITE_RUBY_SKIP_ASSETS_PRECOMPILE_EXTENSION and have yarn.lock, it will run yarn install before attempting to build.

---

Make sure you don't have a package-lock.json in the project.

If yarn install is running but vite is not found, make sure that vite and any build dependencies are not in devDependencies.

---

Regarding double installs, yarn will detect if the lockfile hasn't changed since the last time and skip installation, so it shouldn't be a concern in most cases.

[jrochkind]

PS: As far as I can tell, putting vite in devDependencies doesn't make any difference -- it's fine, if npx ci is going to work at all.

(which is good because that's what vite-ruby docs say it installs by default, devDependencies?)

If i have npx available due to the heroku npm buildpack -- it's fine that vite and plugins are in package.json devDependencies. And I don't even need to set NPM_CONFIG_INCLUDE='dev' YARN_PRODUCTION=false -- devDependencies appear to be.a non-issue for me anyway.

[ElMassimo]

But there is yarn

That's very specific to the current Ruby buildpack in Heroku, and could change in the future, which is why the Vite Ruby guide recommends adding the node buildpack.

fails completely silently

Thanks for checking, I'd like to fix this one to log a warning on failure.

don't even need to set YARN_PRODUCTION=false

Vite Ruby will attempt to do that for you.

[jrochkind]

Cool, thanks for reponses!

In the past, there have been cases where having both npm and ruby buildpacks on heroku caused problems, including performance problems, such as harvard-lil/h2o#744

That may not be relevant in current versions of buildpacks and rails, not sure.

but I understand why you like the npx ci solution, and like recommending the heroku npm buildpack to make npx available, as the main documented/supported thing.

I'm also feeling good about using an alternate solution as a personal choice in my app specifically, wiring up a yarn install as described above, for me. I think it's unlikely to cause any problems -- after all, you are already getting that anyway if you are using rails before 7.0, and/or have webpacker or cssbundling-rails or jsbundling-rails in your gemfile, which you might along with vite. So I'm thinking it is unlikely to cause a problem even if both my yarn install and vite's npx ci both succeed.

I think just having a visible error of some kind if npx ci fails would be a big improvement in transparency, so that'd be cool. Failed cause npx isn't found, or even if it is for any other reason. I started trying to PR such a thing, but it got messier than I anticipated and I know you have before preferred to come up with your own solutions, but happy to try a PR if you'd like one!

[ElMassimo]

Sounds good!

I was thinking something along these lines:

  task :install_dependencies do
    cmd = ViteRuby.commands.legacy_npm_version? ? 'npx ci --yes' : 'npx --yes ci'
    _stdout, stderr, status = ViteRuby::IO.capture({ 'NODE_ENV' => 'development' }, cmd, with_output: false)
    unless status.success?
      $stdout.puts stderr
      $stdout.puts status
    end
  end

[jrochkind]

I hadn't noticed ViteRuby::IO.capture. I did notice in trying to use stdlib Open3.capture3, that method raises an exception if you call a command that does not exist (like npx in a heroku environment), so handling that is what was making my code messier.

In general, I believe this is the only place that vite-ruby requires npx (or npm) command lines to exist. :(.

After having learned more about the heroku issues, i wonder if a PR to the heroku deploy guidance? Adding the context that the npm buildpack is recommended just to get the npx command there? Removing the ENV variable suggestions, since you pointed out vite-ruby should take care of it regardless?

[jpriollaud]

In my case a misconfigured capistrano-nvm step was failing to do its own yarn install before the vite execution

Note for Googlers, double check your steps that precede vite build.