-
-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix large SFTP reads #638
base: devel
Are you sure you want to change the base?
Fix large SFTP reads #638
Conversation
Quality Gate failedFailed conditions |
Congratulations! One of the builds has completed. 🍾 You can install the built RPMs by following these steps:
Please note that the RPMs should be used only in a testing environment. |
src/pylibsshext/sftp.pyx
Outdated
with open(local_file, 'wb+') as f: | ||
with open(local_file, 'ab') as f: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will append content to the file which already exists. I've tested this diff with ansible-libssh v1.2.2
and only a60a6cb applied. I see multiple executions on the same destination file growing above original size:
remote-machine# ls -l /tmp/foo?
-rw-r--r-- 1 root wheel 1048576 Nov 15 12:31 /tmp/foo1
-rwx------ 1 root wheel 3145728 Nov 15 12:37 /tmp/foo3
local-machine$ ls -l /tmp/foo?
-rw-r--r-- 1 mkucharski wheel 3145728 Nov 15 12:35 /tmp/foo2
Above we see size 3x after 3 executions of:
t1 = sftp_session.get(remote_file, local_file)
t2 = sftp_session.put(local_file, new_remote_file)
I think this needs more work. Also mode doesn't match after put()
, but that is another issue.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for throughful testing! Indeed, you are right. I thought this small change will make it working, but it looks like its more complicated. Updated the PR.
For the modes, I think this would be a different issue or rfe. Right now, the code opens the file with normal python open, which will probably set modes based on your umask, which will result in sane permissions. We could update the permissions based on what is on the server (if the server uses unix/linux ACL), but given that they will likely have different usernames and groups, it might result in unwanted permissions.
Fixes: ansible#341 Signed-off-by: Jakub Jelen <jjelen@redhat.com>
Signed-off-by: Jakub Jelen <jjelen@redhat.com>
Signed-off-by: Jakub Jelen <jjelen@redhat.com>
Thanks @kucharskim for the report and testing! Signed-off-by: Jakub Jelen <jjelen@redhat.com>
Signed-off-by: Jakub Jelen <jjelen@redhat.com>
Signed-off-by: Jakub Jelen <jjelen@redhat.com>
Quality Gate failedFailed conditions |
SUMMARY
Fixes and reproducers for large SFTP reads that truncate file with the second chunk.
Fixes: #341
ISSUE TYPE