-
Notifications
You must be signed in to change notification settings - Fork 12
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #114 from billsacks/fix_large_output_hang
Fix hangs when a subcommand has a lot of output. Rework execute_subprocess timeout handling to prevent hangs The previous implementation, relying on the lower-level subprocess.Popen, was hanging when trying to get the status of a git repository with many changed files (see #113). It seems that the problem was that subprocess.Popen buffers output, and we would need to write some extra code to periodically clear the output buffer. This seemed tricky to get right, so I have backed up to using the higher-level subprocess.check_output, as we did prior to #81. I am using a new method to detect hangs and print a useful message (which was the point of PR #81), based on threading.Timer. I can't figure out a way to kill the subprocess when it appears to be hanging, but at least we can notify the user of this possible problem. This commit brings this part of the code back to how it was prior to d6423c6, except now we have a hang-detection timer. As a side-benefit, it appears that this change speeds up the unit and system tests, based on a few trial runs before and after this change. (This is probably due to the removal of the 0.02 sec delay associated with the previous subprocess.Popen implementation.) User interface changes?: No Fixes #113 (Timeout when getting status from a git repository with many changes) Testing: test removed: none unit tests: pass system tests: pass manual testing: (1) Tested the problem reported in #113 by putting a ton of changes in components/clm (changing permissions on all files, and adding about 7000 files in a testmods directory). The original hung, this version succeeded quickly. (2) Tested the problem reported in #79 and fixed in #81 (because this PR undoes the changes from #81): I tested with authentication required (on hobart, by pointing to the NGEET fates repo): I changed the timeout to 10 seconds, ran 'manage_externals/checkout_externals clm', and reviewed the printed message. It looks like this, after about 10 seconds: Checking out externals: fates, Username for 'https://github.com': Command 'git clone --quiet https://github.com/NGEET/fates fates' from directory /scratch/cluster/sacks/cesm/components/clm/src has taken 10 seconds. It may be hanging. The command will continue to run, but you may want to abort manage_externals with ^C and investigate. A possible cause of hangs is when svn or git require authentication to access a private repository. On some systems, svn and git requests for authentication information will not be displayed to the user. In this case, the program will appear to hang. Ensure you can run svn and git manually and access all repositories without entering your authentication information.
- Loading branch information
Showing
5 changed files
with
59 additions
and
94 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters