Subdirectories not deleted during `sync-glob --watch` on Windows 10 / Linux #39

KeithGillette · 2017-03-30T17:12:06Z

When running:

sync-glob --watch 'source/*' destination
or:
sync-glob --watch 'source/**/*' destination

directories and files are copied and updated (recursively) from source to destination, but the watch does not delete subdirectories deleted in source during the watch.

With:

sync-glob 1.3.6
npm 3.10.10
Node 6.9.5
Windows_NT 10.0.14393

The text was updated successfully, but these errors were encountered:

KeithGillette · 2017-03-30T22:40:03Z

Thanks for the fast patching, @AndyOGo! Your release 1.3.7 did fix issue #41, so files are now correctly deleted, but in my test it still does not detect (empty) subdirectory deletion on Windows 10.

AndyOGo · 2017-03-31T11:27:08Z

@KeithGillette
Thanks for your quick feedback.
I will investigate this issue. From your data it seems have a missing test case too.

AndyOGo · 2017-03-31T20:02:27Z

I just run the following test on my Mac
Note: all sub directories here are not empty.

node bin/sync-glob.js --watch 'test/**/*' tmp/

Initial Copy

DELETE tmp
COPY test/mock/bar to tmp/bar
COPY test/mock/foo to tmp/foo
COPY test/mock/@org to tmp/@org
COPY test/mock/a.txt to tmp/a.txt
COPY test/mock/b.txt to tmp/b.txt
COPY test/mock/transform.js to tmp/transform.js
COPY test/mock/bar/c.txt to tmp/bar/c.txt
COPY test/mock/foo/b.txt to tmp/foo/b.txt
COPY test/mock/foo/d.txt to tmp/foo/d.txt
COPY test/mock/@org/d.txt to tmp/@org/d.txt
COPY test/mock/@org/b.txt to tmp/@org/b.txt
COPY test/mock/@org/a.txt to tmp/@org/a.txt
WATCHING test/mock/**/*

Deleting test/mock/ sub directory

DELETE tmp/mock/@org/a.txt
DELETE tmp/mock/@org/b.txt
DELETE tmp/mock/@org/d.txt
DELETE tmp/mock/@org
DELETE tmp/mock/bar/c.txt
DELETE tmp/mock/bar
DELETE tmp/mock/foo/b.txt
DELETE tmp/mock/foo/d.txt
DELETE tmp/mock/foo
DELETE tmp/mock/b.txt
DELETE tmp/mock/a.txt
DELETE tmp/mock/transform.js
DELETE tmp/mock

Deleting test/mock/@org/ sub sub directory

DELETE tmp/mock/@org/b.txt
DELETE tmp/mock/@org/a.txt
DELETE tmp/mock/@org/d.txt
DELETE tmp/mock/@org

AndyOGo · 2017-03-31T20:08:56Z

So double I just created an empty sub directory and run following test:

node bin/sync-glob.js --watch 'test/**/*' tmp/

DELETE tmp
COPY test/lib to tmp/lib
COPY test/mock to tmp/mock
COPY test/mock/@org to tmp/mock/@org
COPY test/mock/bar to tmp/mock/bar
COPY test/mock/empty to tmp/mock/empty
COPY test/mock/foo to tmp/mock/foo
COPY test/copy.spec.js to tmp/copy.spec.js
COPY test/helpers.js to tmp/helpers.js
COPY test/lib/is-glob.spec.js to tmp/lib/is-glob.spec.js
COPY test/lib/resolve-target.spec.js to tmp/lib/resolve-target.spec.js
COPY test/lib/sources-bases.spec.js to tmp/lib/sources-bases.spec.js
COPY test/lib/trim-quotes.spec.js to tmp/lib/trim-quotes.spec.js
COPY test/mock/@org/a.txt to tmp/mock/@org/a.txt
COPY test/mock/@org/b.txt to tmp/mock/@org/b.txt
COPY test/mock/@org/d.txt to tmp/mock/@org/d.txt
COPY test/mock/a.txt to tmp/mock/a.txt
COPY test/mock/b.txt to tmp/mock/b.txt
COPY test/mock/bar/c.txt to tmp/mock/bar/c.txt
COPY test/mock/foo/b.txt to tmp/mock/foo/b.txt
COPY test/mock/foo/d.txt to tmp/mock/foo/d.txt
COPY test/mock/transform.js to tmp/mock/transform.js
COPY test/sync.spec.js to tmp/sync.spec.js
COPY test/transform.spec.js to tmp/transform.spec.js
WATCHING test/**/*

Delete test/mock/empty by running rmdir test/mock/empty

DELETE tmp/mock/empty

@KeithGillette
Seems all okay on OSX.
May I asked you if you get any errors? Or could you post your logs please?
Unfortunately I don't have a Windows 10 machine available 😞
And please use the --verbose option.

AndyOGo · 2017-03-31T20:13:21Z

@KeithGillette
I quickly googled if chokidar has a deletion issue.
And indeed it describes exactly your problem with following error:

Error: watch null EPERM

Still open: Deleting an empty directory triggers error instead of unlinkDir paulmillr/chokidar#566
Already closed, but related:
Windows: Cannot delete watched folder with not-empty subfolder paulmillr/chokidar#422
Windows 7, Chokidar exits with error when remove a directory paulmillr/chokidar#88

AndyOGo · 2017-03-31T20:17:53Z

@KeithGillette
Can you confirm that you have an permission error?

KeithGillette · 2017-03-31T20:32:50Z

Hi @AndyOGo. I get absolutely no errors in the console even with the --verbose flag. The deleted empty directories are simply ignored by node-glob-sync altogether: no console output appears at the time of deletion. The only time a directory is deleted is when it contains a file.

AndyOGo · 2017-03-31T20:53:06Z

@KeithGillette Okay, what a bummer. Thank you for instant feedback.
The point is, I do use chokidar for file watching and paulmillr/chokidar#566 describes exactly your bug with deletion of empty directories.
Unfortunately I can't reproduce this as long as I don't have a Windows 10 machine available.

KeithGillette · 2017-03-31T21:08:52Z

Bummer, indeed. My current needs don't actually require globbing functionality, so I have fallen back to node-sync-files which does not suffer from this issue. However, that leads me to believe that the problem may not be related to the open chokidar issue, as node-sync-files also has that as a dependency.

AndyOGo · 2017-04-01T06:39:56Z

Sure, use the tool which satisfies your needs best.

Hmh, interesting. But I don't think so because node-sync-files is using an outdated chokidar version ^1.0.0-rc3 and node-sync-glob is using a more up-to-date version ^1.6.1 of chokidar.

I will keep an eye on this and try to get access to a windows 10 box, so that I can prove what is going on exactly.

KeithGillette · 2017-04-01T10:28:13Z

Thanks again, @AndyOGo. While we can get by with a straight directory sync now, I foresee use cases for globbing in my project in the future, so I hope you are able to resolve the issue. If it would be helpful in testing, I can get you access to a virtualized Windows 10 environment on Azure.

AndyOGo · 2017-04-01T21:37:57Z

@KeithGillette Thanks for your offer to help.
Meanwhile I asked my brother if he has windows 10, and if I can grap his notebook.
So I will see what I can find out and post my results here as soon as I'm done.

AndyOGo · 2017-04-02T17:45:33Z

I just tested it on windows 10 and unfortunately I found even more bugs.

Deleting empty sub directory does nothing, neither syncing nor error logging.
Creating a New Folder does create a copy of New Folder, but after that folder has been renamed it does not sync it. - Windows 10: creating a new folder and immediatly renameing does not trigger paulmillr/chokidar#592
Even worse If I rename a folder it does not delete the old one - Windows 10: renaming of an empty folder triggers only once paulmillr/chokidar#593
Each name of a rename will only sync at first time, later renaming with the same name do just nothing. - Windows 10: renaming of an empty folder triggers only once paulmillr/chokidar#593
Linux rename event - rename raw event breaks further events paulmillr/chokidar#591

AndyOGo · 2017-04-02T17:52:41Z

I just installed node-inspector and it logs an error though the windows CLI does not print it...

Error watching file for changes: EPERM

I tracked the call stack down and chokidar is emitting this error

AndyOGo · 2017-04-11T06:51:28Z

I got a failing test now:

https://ci.appveyor.com/project/AndyOGo/node-sync-glob/build/1.0.71/job/mvjw85ufp2bhxxvg#L789

FAIL  test\sync.spec.js (8.87s)
  ● node-sync-glob watch › should sync empty sub directory deletion
    ENOTEMPTY: directory not empty, rmdir 'C:\projects\node-sync-glob\tmp\mock'
      
      at Object.fs.rmdirSync (fs.js:856:18)
      at rmkidsSync (node_modules\fs-extra\lib\remove\rimraf.js:300:11)
      at rmdirSync (node_modules\fs-extra\lib\remove\rimraf.js:289:7)
      at rimrafSync (node_modules\fs-extra\lib\remove\rimraf.js:254:7)
      at node_modules\fs-extra\lib\remove\rimraf.js:298:5
      at Array.forEach (native)
      at rmkidsSync (node_modules\fs-extra\lib\remove\rimraf.js:297:26)
      at rmdirSync (node_modules\fs-extra\lib\remove\rimraf.js:289:7)
      at Function.rimrafSync [as sync] (node_modules\fs-extra\lib\remove\rimraf.js:254:7)
      at Object.removeSync (node_modules\fs-extra\lib\remove\index.js:4:17)
  ● node-sync-glob watch › should sync empty sub directory deletion
    EPERM: operation not permitted, mkdir 'C:\projects\node-sync-glob\tmp\mock\bar\empty'
      
      at Object.fs.mkdirSync (fs.js:895:18)
      at Object.mkdirsSync (node_modules\fs-extra\lib\mkdirs\mkdirs-sync.js:29:9)
      at Object.ensureDirSync (test\helpers.js:13:154)
      at Object.<anonymous> (test\sync.spec.js:98:43)
  node-sync-glob watch

AndyOGo · 2017-04-11T08:20:50Z

Well the empty test dir tmp/mock/bar/empty is not recognized by glob-all on initial mirroring...

Windows: https://ci.appveyor.com/project/AndyOGo/node-sync-glob/build/1.0.75/job/yp5tk3x4sm246vj6#L865

console.log src\index.js:94
    [ 'tmp\\mock\\@org',
      'tmp\\mock\\@org\\a.txt',
      'tmp\\mock\\@org\\b.txt',
      'tmp\\mock\\@org\\d.txt',
      'tmp\\mock\\a.txt',
      'tmp\\mock\\b.txt',
      'tmp\\mock\\bar',
      'tmp\\mock\\bar\\c.txt',
      'tmp\\mock\\foo',
      'tmp\\mock\\foo\\b.txt',
      'tmp\\mock\\foo\\d.txt',
      'tmp\\mock\\transform.js' ]
 FAIL  test\sync.spec.js (8.917s)
  ● node-sync-glob watch › should sync empty sub directory deletion
    ENOTEMPTY: directory not empty, rmdir 'C:\projects\node-sync-glob\tmp\mock'
      
      at Object.fs.rmdirSync (fs.js:856:18)
      at rmkidsSync (node_modules\fs-extra\lib\remove\rimraf.js:300:11)
      at rmdirSync (node_modules\fs-extra\lib\remove\rimraf.js:289:7)
      at rimrafSync (node_modules\fs-extra\lib\remove\rimraf.js:254:7)
      at node_modules\fs-extra\lib\remove\rimraf.js:298:5
      at Array.forEach (native)
      at rmkidsSync (node_modules\fs-extra\lib\remove\rimraf.js:297:26)
      at rmdirSync (node_modules\fs-extra\lib\remove\rimraf.js:289:7)
      at Function.rimrafSync [as sync] (node_modules\fs-extra\lib\remove\rimraf.js:254:7)
      at Object.removeSync (node_modules\fs-extra\lib\remove\index.js:4:17)
  ● node-sync-glob watch › should sync empty sub directory deletion
    EPERM: operation not permitted, mkdir 'C:\projects\node-sync-glob\tmp\mock\bar\empty'
      
      at Object.fs.mkdirSync (fs.js:895:18)
      at Object.mkdirsSync (node_modules\fs-extra\lib\mkdirs\mkdirs-sync.js:29:9)
      at Object.ensureDirSync (test\helpers.js:13:154)
      at Object.<anonymous> (test\sync.spec.js:98:43)

In contrast OS X: https://travis-ci.org/AndyOGo/node-sync-glob/jobs/220861943#L1059

console.log src/index.js:94
    [ 'tmp/mock/@org',
      'tmp/mock/@org/a.txt',
      'tmp/mock/@org/b.txt',
      'tmp/mock/@org/d.txt',
      'tmp/mock/a.txt',
      'tmp/mock/b.txt',
      'tmp/mock/bar',
      'tmp/mock/bar/c.txt',
      'tmp/mock/bar/empty',
      'tmp/mock/foo',
      'tmp/mock/foo/b.txt',
      'tmp/mock/foo/d.txt',
      'tmp/mock/transform.js' ]

 PASS  test/sync.spec.js (9.997s)

  node-sync-glob watch
    ✓ should sync a file (2402ms)
    ✓ should sync an array of files (2397ms)
    ✓ should sync a directory (2429ms)
    ✓ should sync globstar (2428ms)
    ✓ should sync empty sub directory deletion (196ms)

AndyOGo · 2017-04-11T10:46:51Z

tmp/mock/bar folder does not exist on windows, hence no empty subfolder within can be created...

console.log src\index.js:101
    sources: tmp/mock/**/* -> tmp/mock/**/*
  console.log src\index.js:102
    target: tmp/sync -> tmp\sync
  console.log src\index.js:103
    globed files: 
    	tmp\mock\@org
    	tmp\mock\@org\a.txt
    	tmp\mock\@org\b.txt
    	tmp\mock\@org\d.txt
    	tmp\mock\a.txt
    	tmp\mock\b.txt
    	tmp\mock\bar
    	tmp\mock\bar\c.txt
    	tmp\mock\foo
    	tmp\mock\foo\b.txt
    	tmp\mock\foo\d.txt
    	tmp\mock\transform.js
  console.log src\index.js:237
    RAW: change -> b.txt
  console.log src\index.js:238
    { watchedPath: 'tmp\\mock\\foo' }
  console.log src\index.js:237
    RAW: change -> b.txt
  console.log src\index.js:238
    { watchedPath: 'tmp\\mock\\foo\\b.txt' }
  console.log src\index.js:237
    RAW: rename -> d.txt
  console.log src\index.js:238
    { watchedPath: 'tmp\\mock\\foo' }
  console.log src\index.js:237
    RAW: rename -> d.txt
  console.log src\index.js:238
    { watchedPath: 'tmp\\mock\\foo\\d.txt' }
  console.log src\index.js:237
    RAW: change -> foo
  console.log src\index.js:238
    { watchedPath: 'tmp/mock' }
 FAIL  test\sync.spec.js (8.937s)
  ● node-sync-glob watch › should sync empty sub directory deletion
    ENOTEMPTY: directory not empty, rmdir 'C:\projects\node-sync-glob\tmp\mock'
      
      at Object.fs.rmdirSync (fs.js:856:18)
      at rmkidsSync (node_modules\fs-extra\lib\remove\rimraf.js:292:11)
      at rmdirSync (node_modules\fs-extra\lib\remove\rimraf.js:283:7)
      at rimrafSync (node_modules\fs-extra\lib\remove\rimraf.js:252:7)
      at options.readdirSync.forEach.f (node_modules\fs-extra\lib\remove\rimraf.js:291:39)
      at Array.forEach (native)
      at rmkidsSync (node_modules\fs-extra\lib\remove\rimraf.js:291:26)
      at rmdirSync (node_modules\fs-extra\lib\remove\rimraf.js:283:7)
      at Function.rimrafSync [as sync] (node_modules\fs-extra\lib\remove\rimraf.js:252:7)
      at Object.removeSync (node_modules\fs-extra\lib\remove\index.js:6:17)
  ● node-sync-glob watch › should sync empty sub directory deletion
    EPERM: operation not permitted, mkdir 'C:\projects\node-sync-glob\tmp\mock\bar\empty'
      
      at Object.fs.mkdirSync (fs.js:895:18)
      at Object.mkdirsSync (node_modules\fs-extra\lib\mkdirs\mkdirs-sync.js:31:9)
      at Object.ensureDirSync (test\helpers.js:13:154)
      at Object.<anonymous> (test\sync.spec.js:100:45)
  node-sync-glob watch
    √ should sync a file (2190ms)
    √ should sync an array of files (2180ms)
    √ should sync a directory (2230ms)
    √ should sync globstar (2243ms)
    × should sync empty sub directory deletion
  console.log test\sync.spec.js:99
    EXISTS: tmp/mock/bar -> false
  console.log test\sync.spec.js:120
    { Error: EPERM: operation not permitted, mkdir 'C:\projects\node-sync-glob\tmp\mock\bar\empty'
        at Object.fs.mkdirSync (fs.js:895:18)
        at Object.mkdirsSync (C:\projects\node-sync-glob\node_modules\fs-extra\lib\mkdirs\mkdirs-sync.js:31:9)
        at Object.ensureDirSync (C:\projects\node-sync-glob\test\helpers.js:13:154)
        at Object.<anonymous> (C:\projects\node-sync-glob\test\sync.spec.js:100:45)
        at attemptAsync (C:\projects\node-sync-glob\node_modules\jest-jasmine2\vendor\jasmine-2.5.2.js:1984:24)
        at QueueRunner.run (C:\projects\node-sync-glob\node_modules\jest-jasmine2\vendor\jasmine-2.5.2.js:1939:9)
        at C:\projects\node-sync-glob\node_modules\jest-jasmine2\vendor\jasmine-2.5.2.js:1966:16
        at C:\projects\node-sync-glob\node_modules\jest-jasmine2\vendor\jasmine-2.5.2.js:1909:9
        at attemptAsync (C:\projects\node-sync-glob\node_modules\jest-jasmine2\vendor\jasmine-2.5.2.js:1987:9)
        at QueueRunner.run (C:\projects\node-sync-glob\node_modules\jest-jasmine2\vendor\jasmine-2.5.2.js:1939:9)
      errno: -4048,
      code: 'EPERM',
      syscall: 'mkdir',
      path: 'C:\\projects\\node-sync-glob\\tmp\\mock\\bar\\empty' }
 PASS  test\lib\sources-bases.spec.js

AndyOGo · 2017-07-09T14:39:57Z

A new file watcher just got released for node 😱
NSFW ... Node Sentinel File Watcher 🎉
I will try that

KeithGillette changed the title ~~Subdirectories not deleted during watch in Windows 10~~ Subdirectories not deleted during sync-glob --watch in Windows 10 Mar 30, 2017

KeithGillette changed the title ~~Subdirectories not deleted during sync-glob --watch in Windows 10~~ Subdirectories not deleted during sync-glob --watch on Windows 10 Mar 30, 2017

AndyOGo mentioned this issue Mar 30, 2017

Bugfix/error path must be a string on deletion #41

Merged

AndyOGo closed this as completed in #41 Mar 30, 2017

AndyOGo reopened this Mar 31, 2017

AndyOGo mentioned this issue Apr 5, 2017

ignore folders/files #20

Closed

AndyOGo added the bug label Apr 9, 2017

This was referenced Apr 9, 2017

Enhancement/run tests on various os #43

Merged

fix appveyor broken ci builds #45

Merged

Bugfix/empty folders windows #46

Open

AndyOGo changed the title ~~Subdirectories not deleted during sync-glob --watch on Windows 10~~ Subdirectories not deleted during sync-glob --watch on Windows 10 / Linux Apr 11, 2017

AndyOGo self-assigned this Apr 11, 2017

AndyOGo mentioned this issue Aug 22, 2017

Incorrect copy on watch #17

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Subdirectories not deleted during `sync-glob --watch` on Windows 10 / Linux #39

Subdirectories not deleted during `sync-glob --watch` on Windows 10 / Linux #39

KeithGillette commented Mar 30, 2017 •

edited

Loading

KeithGillette commented Mar 30, 2017 •

edited

Loading

AndyOGo commented Mar 31, 2017

AndyOGo commented Mar 31, 2017

AndyOGo commented Mar 31, 2017 •

edited

Loading

AndyOGo commented Mar 31, 2017 •

edited

Loading

AndyOGo commented Mar 31, 2017

KeithGillette commented Mar 31, 2017 •

edited

Loading

AndyOGo commented Mar 31, 2017

KeithGillette commented Mar 31, 2017

AndyOGo commented Apr 1, 2017

KeithGillette commented Apr 1, 2017

AndyOGo commented Apr 1, 2017

AndyOGo commented Apr 2, 2017 •

edited

Loading

AndyOGo commented Apr 2, 2017 •

edited

Loading

AndyOGo commented Apr 11, 2017

AndyOGo commented Apr 11, 2017

AndyOGo commented Apr 11, 2017

AndyOGo commented Jul 9, 2017 •

edited

Loading

Subdirectories not deleted during sync-glob --watch on Windows 10 / Linux #39

Subdirectories not deleted during sync-glob --watch on Windows 10 / Linux #39

Comments

KeithGillette commented Mar 30, 2017 • edited Loading

KeithGillette commented Mar 30, 2017 • edited Loading

AndyOGo commented Mar 31, 2017

AndyOGo commented Mar 31, 2017

AndyOGo commented Mar 31, 2017 • edited Loading

AndyOGo commented Mar 31, 2017 • edited Loading

AndyOGo commented Mar 31, 2017

KeithGillette commented Mar 31, 2017 • edited Loading

AndyOGo commented Mar 31, 2017

KeithGillette commented Mar 31, 2017

AndyOGo commented Apr 1, 2017

KeithGillette commented Apr 1, 2017

AndyOGo commented Apr 1, 2017

AndyOGo commented Apr 2, 2017 • edited Loading

AndyOGo commented Apr 2, 2017 • edited Loading

AndyOGo commented Apr 11, 2017

AndyOGo commented Apr 11, 2017

AndyOGo commented Apr 11, 2017

AndyOGo commented Jul 9, 2017 • edited Loading

Subdirectories not deleted during `sync-glob --watch` on Windows 10 / Linux #39

Subdirectories not deleted during `sync-glob --watch` on Windows 10 / Linux #39

KeithGillette commented Mar 30, 2017 •

edited

Loading

KeithGillette commented Mar 30, 2017 •

edited

Loading

AndyOGo commented Mar 31, 2017 •

edited

Loading

AndyOGo commented Mar 31, 2017 •

edited

Loading

KeithGillette commented Mar 31, 2017 •

edited

Loading

AndyOGo commented Apr 2, 2017 •

edited

Loading

AndyOGo commented Apr 2, 2017 •

edited

Loading

AndyOGo commented Jul 9, 2017 •

edited

Loading