Problems with long Pathnames on Windows

[tknerr]

Recently I'm running into No such file or directory - ...<quite long path> ... (Errno::ENOENT) errors more often.

The problem is due to the max path length limit of 256 chars on Windows.

The backtraces look similar to that:

W:/tools/ruby/ruby-1.9.3-p385-i386-mingw32/lib/ruby/1.9.1/fileutils.rb:1371:in `initialize': No such file or directory - R:/puroz-chef-workflow/test/esx-bootstrap/cookbooks/sample-app-0.1.0/sample-app/tmp/librarian/cache/source/chef/site/877777683730772c36b1e6a3fc3aa2c3/apache2/version-uri/4c32315570baa05b9c5678752b2968c9/package/files/default/tests/minitest/mod_auth_cas_test.rb (Errno::ENOENT)
        from W:/tools/ruby/ruby-1.9.3-p385-i386-mingw32/lib/ruby/1.9.1/fileutils.rb:1371:in `open'
        from W:/tools/ruby/ruby-1.9.3-p385-i386-mingw32/lib/ruby/1.9.1/fileutils.rb:1371:in `block in copy_file'
        from W:/tools/ruby/ruby-1.9.3-p385-i386-mingw32/lib/ruby/1.9.1/fileutils.rb:1370:in `open'
        from W:/tools/ruby/ruby-1.9.3-p385-i386-mingw32/lib/ruby/1.9.1/fileutils.rb:1370:in `copy_file'
        from W:/tools/ruby/ruby-1.9.3-p385-i386-mingw32/lib/ruby/1.9.1/fileutils.rb:1338:in `copy'
        from W:/tools/ruby/ruby-1.9.3-p385-i386-mingw32/lib/ruby/1.9.1/fileutils.rb:465:in `block in copy_entry'
        from W:/tools/ruby/ruby-1.9.3-p385-i386-mingw32/lib/ruby/1.9.1/fileutils.rb:1433:in `preorder_traverse'
        from W:/tools/ruby/ruby-1.9.3-p385-i386-mingw32/lib/ruby/1.9.1/fileutils.rb:462:in `copy_entry'
        from W:/tools/ruby/ruby-1.9.3-p385-i386-mingw32/lib/ruby/1.9.1/fileutils.rb:437:in `block in cp_r'
        from W:/tools/ruby/ruby-1.9.3-p385-i386-mingw32/lib/ruby/1.9.1/fileutils.rb:1515:in `block in fu_each_src_dest'
        from W:/tools/ruby/ruby-1.9.3-p385-i386-mingw32/lib/ruby/1.9.1/fileutils.rb:1524:in `block in fu_each_src_dest0'
        from W:/tools/ruby/ruby-1.9.3-p385-i386-mingw32/lib/ruby/1.9.1/fileutils.rb:1522:in `each'
        from W:/tools/ruby/ruby-1.9.3-p385-i386-mingw32/lib/ruby/1.9.1/fileutils.rb:1522:in `fu_each_src_dest0'
        from W:/tools/ruby/ruby-1.9.3-p385-i386-mingw32/lib/ruby/1.9.1/fileutils.rb:1513:in `fu_each_src_dest'
        from W:/tools/ruby/ruby-1.9.3-p385-i386-mingw32/lib/ruby/1.9.1/fileutils.rb:436:in `cp_r'
...

One of the issues is that the librarian dictated tmp paths (e.g. tmp/librarian/cache/source/chef/site/877777683730772c36b1e6a3fc3aa2c3/apache2/version-uri/4c32315570baa05b9c5678752b2968c9/package/) already take up 131 chars, leaving only 135 chars left for

1. the location of the project
2. the files inside the cookbooks

While you can work around 1. using subst R: C:\path\to\where\my\projects\are (that I have done already in the backtrace above) to some extent, you typically can't do this for 2.

I don't have a good idea how to really solve this issue, but there are some strategies to consider:

• shorten the librarian dictated tmp paths to reduce the likeliness of the problem to occur, e.g. tmp/librarian/c/s/c/s/877777683730772c36b1e6a3fc3aa2c3/apache2/vu/4c32315570baa05b9c5678752b2968c9/p/ instead of tmp/librarian/cache/source/chef/site/877777683730772c36b1e6a3fc3aa2c3/apache2/version-uri/4c32315570baa05b9c5678752b2968c9/package/ (saves 29 chars)
• use a different hashing algorithm which produces shorter hashes or truncate the hash to the first x digits (similar to what git does). Currently the MD5 hashes are 32 chars each, truncating them to 16 chars each would save another 32 chars while keeping the collision probability at around 1:1 billion
• find a proper solution how to deal with long paths on windows (impossible?)

Any ideas how to deal with this?

[WMeldon]

I was able to trigger this issue despite the shortened hashes. Reported it on the offending cookbook (edelight/chef-mongodb#160). Not sure if there is a solution, but I just moved my project directory to get it under the 256 length limit.

[Breeze773]

I am coming across this issue with librarian-puppet rather than chef, but am not able to work around it in one particular case. I am having our build agents (Team City) in my case do a librarian-puppet install so that they have the modules required to run tests against our internally developed modules. I'd rather not change the working directory for our agents. Any chance there's a permanent fix identified for this?

[vellotis]

I came across the problem on Win 8.1 Pro. Shouldn't KB2919355 fix it? I have this update installed, but still have the same problem.

I think this is more like the stackoverfow discussion...

[vellotis]

Answering with stackoverflow link: Does MAX_PATH issue still exists in Windows 10.