Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase file descriptor limit #313

Merged
merged 1 commit into from
Nov 11, 2024

Conversation

S81D
Copy link
Contributor

@S81D S81D commented Oct 21, 2024

When running the LoadANNIEEvent tool, which loads ANNIEEvents from the processed part files, it's been noticed that the tool fails to execute when including > ~1020 part files due to Linux's file descriptor limit. The tool runs smoothly until it reaches this limit, where it encounters an error like this:

[1]: LoadANNIEEvent: Loading new file /pnfs/annie/persistent/processed/processed_EBV2//R4783/ProcessedData_PMTMRDLAPPD_R4783S0p1017
[1]: LoadANNIEEvent: Loading new file /pnfs/annie/persistent/processed/processed_EBV2//R4783/ProcessedData_PMTMRDLAPPD_R4783S0p1018
[1]: Error Bad Filename: Cannot open /pnfs/annie/persistent/processed/processed_EBV2//R4783/ProcessedData_PMTMRDLAPPD_R4783S0p1019
[1]: LoadANNIEEvent: Loading new file /pnfs/annie/persistent/processed/processed_EBV2//R4783/ProcessedData_PMTMRDLAPPD_R4783S0p1019
[0]: LoadANNIEEvent: Filename /pnfs/annie/persistent/processed/processed_EBV2//R4783/ProcessedData_PMTMRDLAPPD_R4783S0p1019 not found! Proceed to next file

--> Warning errors in this tool and any other downstream tools, failure to load ANNIEEvents to the store

As this tool is frequently ran over all part files within a run, and we routinely have more than 1024 part files per run, I have included an increase in the file descriptor (4096) to Setup.sh. I have tested it and it alleviates the problem.

Expand file limit to allow tools like LoadANNIEEvent to loop over all part files of a run
@marc1uk marc1uk merged commit e29a1be into ANNIEsoft:Application Nov 11, 2024
1 check passed
@S81D S81D deleted the OS_File_Limit_fix branch December 5, 2024 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants