Nix-Flakes: Dev-Server does not recognize file changes on new IHP version

[InftyCat]

Both in the new disaya version and textcontent I have trouble to autoreload. F5 also does not work. The only thing that worked yet is restarting the server.

[Eisfunke]

So when you change the source nothing happens at all? Is there any output in the terminal? Can you post the terminal output beginning from you starting the dev server?

[InftyCat]

No, there is no output at all. Marc already had a look on the terminal output but I send it to you anyhow:

20:56:11 system | devServer.1 started (pid=252603)
20:56:11 devServer.1 | IHP Version: 1.0.1
20:56:11 devServer.1 | waiting for server to shut down.... done
20:56:11 devServer.1 | server stopped
20:56:12 devServer.1 | GET /
20:56:12 devServer.1 | Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,/;q=0.8
20:56:12 devServer.1 | Status: 302 Found 0.183559304s
20:56:12 devServer.1 |
20:56:12 devServer.1 | ===> AppState {postgresState = NotStarted, appGHCIState = NotStarted, statusServerState = NotStarted, databaseNeedsMigration = (..), lastSchemaCompilerError = (..)}
20:56:12 devServer.1 | UpdateStatusServerState Started
20:56:12 devServer.1 | ===> AppState {postgresState = NotStarted, appGHCIState = NotStarted, statusServerState = Started, databaseNeedsMigration = (..), lastSchemaCompilerError = (..)}
20:56:12 devServer.1 | [("ihpVersion","1.0.1"),("os","linux"),("arch","x86_64"),("projectId","b7160aa803ab07b85d72149bb5cfbbce6f3e5c70d8352b0e5d8dab959e995d0545751f7ca77afd9b4f6e648e5d6c279e11b7df1f377c17b258b59f4c2d783c4d")]
20:56:12 devServer.1 | UpdatePostgresState Started
20:56:12 devServer.1 | ===> AppState {postgresState = Started, appGHCIState = NotStarted, statusServerState = Started, databaseNeedsMigration = (..), lastSchemaCompilerError = (..)}
20:56:12 devServer.1 | GHCI: :script build/ihp-lib//applicationGhciConfig
20:56:12 devServer.1 | GHCI: :set prompt ""
20:56:12 devServer.1 | GHCI: import qualified ClassyPrelude
20:56:12 devServer.1 | UpdateAppGHCIState Loading
20:56:12 devServer.1 | ===> AppState {postgresState = Started, appGHCIState = Loading, statusServerState = Started, databaseNeedsMigration = (..), lastSchemaCompilerError = (..)}
20:56:12 devServer.1 | GET /ShowTable
20:56:12 devServer.1 | Params: [("tableName","users")]
20:56:12 devServer.1 | Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,/;q=0.8
20:56:12 devServer.1 | Status: 200 OK 0.078854515s
20:56:12 devServer.1 |
20:56:12 devServer.1 | GET /vendor/bootstrap.min.css
20:56:12 devServer.1 | Params: [("v","1.0.1")]
20:56:12 devServer.1 | Accept: text/css,/;q=0.1
20:56:12 devServer.1 | Status: 200 OK 0.024282591s
20:56:12 devServer.1 |
20:56:12 devServer.1 | GET /vendor/popper.min.js
20:56:12 devServer.1 | Params: [("v","1.0.1")]
20:56:12 devServer.1 | Accept: /
20:56:12 devServer.1 | Status: 200 OK 0.000740032s
20:56:12 devServer.1 |
20:56:12 devServer.1 | GET /IDE/schema-designer.css
20:56:12 devServer.1 | Params: [("v","1.0.1")]
20:56:12 devServer.1 | Accept: text/css,/;q=0.1
20:56:12 devServer.1 | Status: 200 OK 0.001081684s
20:56:12 devServer.1 |
20:56:12 devServer.1 | GET /vendor/select2.min.css
20:56:12 devServer.1 | Params: [("v","1.0.1")]
20:56:12 devServer.1 | Accept: text/css,/;q=0.1
20:56:12 devServer.1 | Status: 200 OK 0.001162804s
20:56:12 devServer.1 |
20:56:12 devServer.1 | GET /vendor/morphdom-umd.min.js
20:56:12 devServer.1 | Params: [("v","1.0.1")]
20:56:12 devServer.1 | Accept: /
20:56:12 devServer.1 | Status: 200 OK 0.001126803s
20:56:12 devServer.1 |
20:56:12 devServer.1 | GET /vendor/turbolinks.js
20:56:12 devServer.1 | Params: [("v","1.0.1")]
20:56:12 devServer.1 | Accept: /
20:56:12 devServer.1 | Status: 200 OK 0.000640444s
20:56:12 devServer.1 |
20:56:12 devServer.1 | GET /vendor/bootstrap.min.js
20:56:12 devServer.1 | Params: [("v","1.0.1")]
20:56:12 devServer.1 | Accept: /
20:56:12 devServer.1 | Status: 200 OK 0.008094256s
20:56:12 devServer.1 |
20:56:12 devServer.1 | GET /vendor/turbolinksMorphdom.js
20:56:12 devServer.1 | Params: [("v","1.0.1")]
20:56:12 devServer.1 | Accept: /
20:56:12 devServer.1 | Status: 200 OK 0.002634691s
20:56:12 devServer.1 |
20:56:12 devServer.1 | GET /vendor/turbolinksInstantClick.js
20:56:12 devServer.1 | Params: [("v","1.0.1")]
20:56:12 devServer.1 | Accept: /
20:56:12 devServer.1 | Status: 200 OK 0.000277375s
20:56:12 devServer.1 |
20:56:12 devServer.1 | GET /vendor/jquery-3.6.0.min.js
20:56:12 devServer.1 | Params: [("v","1.0.1")]
20:56:12 devServer.1 | Accept: /
20:56:12 devServer.1 | Status: 200 OK 0.001362145s
20:56:12 devServer.1 |
20:56:12 devServer.1 | GET /vendor/timeago.js
20:56:12 devServer.1 | Params: [("v","1.0.1")]
20:56:12 devServer.1 | Accept: /
20:56:12 devServer.1 | Status: 200 OK 0.000456281s
20:56:12 devServer.1 |
20:56:12 devServer.1 | GET /IDE/contextmenu.js
20:56:12 devServer.1 | Params: [("v","1.0.1")]
20:56:12 devServer.1 | Accept: /
20:56:12 devServer.1 | Status: 200 OK 0.000778533s
20:56:12 devServer.1 |
20:56:12 devServer.1 | GET /helpers.js
20:56:12 devServer.1 | Params: [("v","1.0.1")]
20:56:12 devServer.1 | Accept: /
20:56:12 devServer.1 | Status: 200 OK 0.003166634s
20:56:12 devServer.1 |
20:56:12 devServer.1 | GET /vendor/src-min/ext-language_tools.js
20:56:12 devServer.1 | Params: [("v","1.0.1")]
20:56:12 devServer.1 | Accept: /
20:56:12 devServer.1 | Status: 200 OK 0.002715059s
20:56:12 devServer.1 |
20:56:12 devServer.1 | GET /vendor/src-min/ace.js
20:56:12 devServer.1 | Params: [("v","1.0.1")]
20:56:12 devServer.1 | Accept: /
20:56:12 devServer.1 | Status: 200 OK 0.00559803s
20:56:12 devServer.1 |
20:56:12 devServer.1 | GET /vendor/select2.min.js
20:56:12 devServer.1 | Params: [("v","1.0.1")]
20:56:12 devServer.1 | Accept: /
20:56:12 devServer.1 | Status: 200 OK 0.007419348s
20:56:12 devServer.1 |
20:56:12 devServer.1 | GET /IDE/ihp-schemadesigner.js
20:56:12 devServer.1 | Params: [("v","1.0.1")]
20:56:12 devServer.1 | Accept: /
20:56:12 devServer.1 | Status: 200 OK 0.000838742s
20:56:12 devServer.1 |

20:57:00 devServer.1 | GET /IDE/code.css
20:57:00 devServer.1 | Accept: text/css,/;q=0.1
20:57:00 devServer.1 | Status: 200 OK 0.000344607s
20:57:00 devServer.1 |
20:57:00 devServer.1 | GET /IDE/migrations.css
20:57:00 devServer.1 | Accept: text/css,/;q=0.1
20:57:00 devServer.1 | Status: 200 OK 0.000886508s
20:57:00 devServer.1 |
20:57:00 devServer.1 | GET /IDE/fonts/inter/inter-v8-latin-regular.woff2
20:57:00 devServer.1 | Accept: application/font-woff2;q=1.0,application/font-woff;q=0.9,/;q=0.8
20:57:00 devServer.1 | Status: 200 OK 0.002156043s
20:57:00 devServer.1 |
20:57:00 devServer.1 | GET /IDE/fonts/inter/inter-v8-latin-700.woff2
20:57:00 devServer.1 | Accept: application/font-woff2;q=1.0,application/font-woff;q=0.9,/;q=0.8
20:57:00 devServer.1 | Status: 200 OK 0.02159407s
20:57:00 devServer.1 |
20:57:00 devServer.1 | GET /ihp-icon-white-bg.svg
20:57:00 devServer.1 | Accept: image/avif,image/webp,/
20:57:00 devServer.1 | Status: 200 OK 0.025430915s
20:57:00 devServer.1 |
20:57:00 devServer.1 | IHP Telemetry is activated. This can be disabled by setting env variable IHP_TELEMETRY_DISABLED=1
20:57:02 devServer.1 | GET /EditColumn
20:57:02 devServer.1 | Params: [("tableName","users"),("columnId","22")]
20:57:02 devServer.1 | Accept: text/html, application/xhtml+xml
20:57:02 devServer.1 | Status: 200 OK 0.090655727s
20:57:02 devServer.1 |
20:57:03 devServer.1 | GET /EditColumn
20:57:03 devServer.1 | Params: [("tableName","users"),("columnId","23")]
20:57:03 devServer.1 | Accept: text/html, application/xhtml+xml
20:57:03 devServer.1 | Status: 200 OK 0.11810819s
20:57:03 devServer.1 |
20:57:03 devServer.1 | GET /EditColumn
20:57:03 devServer.1 | Params: [("tableName","users"),("columnId","26")]
20:57:03 devServer.1 | Accept: text/html, application/xhtml+xml
20:57:03 devServer.1 | Status: 200 OK 0.051463007s
20:57:03 devServer.1 |
20:57:03 devServer.1 | GET /EditIndex
20:57:03 devServer.1 | Params: [("tableName","users"),("indexName","users_stripe_customer_id_index")]
20:57:03 devServer.1 | Accept: text/html, application/xhtml+xml
20:57:03 devServer.1 | Status: 200 OK 0.045871846s
20:57:03 devServer.1 |
20:57:08 devServer.1 | AppModulesLoaded {success = True}
20:57:08 devServer.1 | GHCI: app <- ClassyPrelude.async (main catch (e :: SomeException) -> IHP.Prelude.putStrLn (tshow e))
20:57:09 devServer.1 | ===> AppState {postgresState = Started, appGHCIState = Loaded, statusServerState = Paused, databaseNeedsMigration = (..), lastSchemaCompilerError = (..)}
20:57:14 devServer.1 | AppModulesLoaded {success = True}
20:57:14 devServer.1 | AppGHCIModulesLoaded triggered multiple times. This happens when multiple file change events are detected. Skipping app start as the app is already starting from a previous file change event
20:57:14 devServer.1 | ===> AppState {postgresState = Started, appGHCIState = Loaded, statusServerState = Paused, databaseNeedsMigration = (..), lastSchemaCompilerError = (..)}
20:57:15 devServer.1 | AppStarted
20:57:15 devServer.1 | ===> AppState {postgresState = Started, appGHCIState = Running, statusServerState = Paused, databaseNeedsMigration = (..), lastSchemaCompilerError = (..)}
20:58:57 devServer.1 | GET /ShowEnum
20:58:57 devServer.1 | Params: [("enumName","length_units")]
20:58:57 devServer.1 | Accept: text/html, application/xhtml+xml
20:58:57 devServer.1 | Status: 200 OK 0.022846936s
20:58:57 devServer.1 |
20:58:58 devServer.1 | GET /EditColumn
20:58:58 devServer.1 | Params: [("tableName","users"),("columnId","15")]
20:58:58 devServer.1 | Accept: text/html, application/xhtml+xml
20:58:58 devServer.1 | Status: 200 OK 0.06175422s
20:58:58 devServer.1 |
20:58:58 devServer.1 | GET /EditColumn
20:58:58 devServer.1 | Params: [("tableName","users"),("columnId","4")]
20:58:58 devServer.1 | Accept: text/html, application/xhtml+xml
20:58:58 devServer.1 | Status: 200 OK 0.031610384s
20:58:58 devServer.1 |

[Eisfunke]

That looks indeed normal, weird. How did you start the dev server exactly? ./start?

Maybe try cloning the repo again from scratch?

[mpscholten]

That looks indeed normal, weird. How did you start the dev server exactly? ./start?

devenv up

Maybe try cloning the repo again from scratch?

We tried this already, same issue. Interesting observation: on old disaya commits it was working, after we pulled latest main of disaya, the same bug happend

[Eisfunke]

Does perhaps deleting the .devenv and .direnv paths help?

[InftyCat]

No unfortunately not.

[CSchank]

Can confirm this is also happening to me on Ubuntu 22.04 when running with the new devenv up command. I have to restart the server to get it to find changes.

[Eisfunke]

Okay, I still have no real answers, but here's an update.

I just tried various things with @InftyCat.

Running RunDevServer directly, without devenv up in between, still doesn't detect changes.

In a project where the IHP version used still has make .envrc, running that to replace .envrc with an old one, deleting .direnv and .devenv and then running RunDevServer did work (briefly, because that quickly after that just stopped starting at all, no clue why, but I'm ignoring that for now).

What that tells me: probably the culprit is not devenv as process manager or the devenv command. Either we broke something unrelated with the file watcher in the DevServer, or some of the changes we did to the build process had a side effect affecting the RunDevServer binary, breaking it in the process.

What's interesting is that at this point I know of multiple people and devices where we use the new devenv stuff with IHP: some with MacOS, some with NixOS, and some with Ubuntu or Debian. From this selection, the problems seem to only occur on Ubuntu and Debian.

It doesn't seem to be a problem with the Nix version, I just did an nix upgrade-nix with @InftyCat on Debian, and it's still broken. Maybe direnv? What's the output of direnv --version for you @CSchank?

The only other idea I have right now is that something about how Ubuntu/Debian handle their filesystem mounting or home directory might break file watching, but it still doesn't make sense that it's only triggered by our devenv build... Will look into that anyway.

I might try reproducing this in a Ubuntu VM...

[Eisfunke]

By the way, tagging @pauldub, who wrote the devenv integration, in case you have any ideas what might cause this...

[CSchank]

What's the output of direnv --version for you @CSchank?

$ direnv --version
2.32.3

[kodeFant]

I have been playing with a Ubuntu VM today and I can confirm and reproduce these problems.

I was also in addition getting this when trying to run nix develop --impure:

lillo@lillo-Standard-PC-Q35-ICH9-2009:~/test-ihp$ nix develop --impure
error (ignored): error: end of string reached
error:
       … while fetching the input 'path:/home/lillo/test-ihp'

       error: file '/home/lillo/test-ihp/build/db/.s.PGSQL.5432' has an unsupported type

This works as expected on my NixOS system. Are you also getting this @CSchank / @InftyCat / @amitaibu ?

[CSchank]

@kodeFant I don't recall seeing this error before no!

[amitaibu]

error: file '/home/lillo/test-ihp/build/db/.s.PGSQL.5432' has an unsupported type

@kodeFant No, I haven't seen this

[kodeFant]

thanks, likely unrelated then

[pauldub]

I'm catching up with the thread, my first impression is that it should not be related to devenv. I'll look into debugging these issues and confirm the root cause.

[amitaibu]

Just to emphasize that my experience (and the occasional stuck devenv up) is with working on the current master of IHP -- not with #1705

[CSchank]

And I'm having the same thing when trying to run IHP 1.0.1 but after using UPGRADE.md to upgrade my project to the Nix Flakes / Devenv environment.

[pauldub]

I've not been able to reproduce the issue on an Ubuntu VM after starting a fresh project pointing to IHP 80373b2 commit.

I've also tested the latest commit on the master branch and the auto reload works as well.

Looking through the logs posted by @InftyCat we can see that there is no HaskellFileChanged event dispatched, this might be a clue as to what is wrong? 🧐

@kodeFant any tip on how I could reproduce the issue?

I've recorded a video so you can see the reload in action:

devenv_reload_test_ubuntu.mov

[kodeFant]

@pauldub Running Ubuntu on a qemu virtual machine installed from image with default desktop setup

I just used ihp-new to create a new project, did nothing extra, and no reloading from the start

Then I exited devenv up, deleted .devenv and .direnv folder, and reload works again.

Each time I restart the system, it's back to not working, and only works again when I delete .devenv and .direnv (when devenv up is not running).

[mpscholten]

Each time I restart the system, it's back to not working, and only works again when I delete .devenv and .direnv (when devenv up is not running).

Thanks 👍 Maybe something is in the directories that is breaking the file watcher. I think at the beginning the file watcher starts indexing all the files and then listens for file changes. Maybe there is some bug in the initial building of the file list

[pauldub]

I've been able to reproduce the issue on a qemu VM, I'll try and see if excluding .direnv and .devenv directories from the file watcher gives better results.

[pauldub]

@kodeFant would you be so kind as to test this commit: pauldub@8f2fa65 ? I've changed the file watcher so that instead of blindly watching everty tree of the root directory it only watche file changes in the root directory and watches sub-trees that are not the .direnv nor the .devenv folders.

[kodeFant]

@pauldub Replaced ihp.url to your commit like this ihp.url = "github:pauldub/ihp?ref=8f2fa65"; in the flake.nix.

Autoreloading now works consistently in my VM, and I am no longer able to reproduce the issue :)

[Eisfunke]

Thank you all! 🎉 Although I'm still wondering why this didn't occur on NixOS or macOS. Some system-specific implementation differences in the file watcher I guess?

[CSchank]

So, sorry, is this now fixed without us having to do anything? Or do you have to do something with the url like @kodeFant mentioned?

[mpscholten]

@CSchank once it's merged, it will work for everyone running the latest IHP version 👍

[duelistone]

Thank you for this thread. For another data point, I wanted to add I was having the exact same issue since I moved to the new flake + devenv up approach, with the same direnv version, on Debian, and pauldub's fix worked perfectly.

[CSchank]

This is still not working for me on macOS (I'm having the absolute worst luck with this new Nix Flakes approach!). I created a new project using ihp-new, and it does not detect changes. Even the fix above with deleting those two folders doesn't seem to make it work. No matter how much I make changes, save them, and refresh the page in the browser, it only gives me the new changes if I Ctrl+C and run devenv up again.

[mokshasoft]

I have the same problem on NixOS. I created a new project with ihp-new and changes are not detected when staring with neither ./start nor devenv up. I also have to restart the server for the changes to take effect.

[mpscholten]

I can reproduce this on macOS 👍

[mpscholten]

Found a bug in the new file watcher implementation. This caused the file watcher to only watch a single sub directory (typically Config/). Fixed via #1746 (cc @pauldub )

[mokshasoft]

Thanks! It fixed my problem as well.