PHNX is a utility for managing and scheduling batch jobs on a SLURM-based HPC system. It allows users to manage a queue of tasks to be executed and schedule them according to the user-defined maximum number of concurrent jobs. The utility also includes a status script to monitor the progress of the tasks and provide an estimated time for completion.
- A Linux-based system with a SLURM-based HPC environment
- Bash shell
Clone the repository or download the files directly:
git clone git@github.com:zeroknowledgediscovery/phnx.git
Change to the repository directory:
cd phnx
Make sure that the scripts are executable:
chmod +x phnx.sh phnx_status.sh
Create a config.sh
file in the same directory as the scripts with the following parameters:
#!/bin/bash
# Maximum number of jobs allowed to run concurrently
MAXJOBS=5
# Path to the QUEUE file containing a list of commands to be executed (one command per line)
QUEUE="queue.txt"
# SLURM sbatch settings
TIME="02:00:00" # Target time for jobs (HH:MM:SS)
PARTITION="your_partition" # Partition name
MEMORY="2G" # Memory per node
CORES=1 # Number of cores per task
CPUS=1 # Number of CPUs per task
Make sure to replace your_partition with the appropriate partition name for your HPC system.
The phnx.sh
script is responsible for managing and scheduling jobs from the queue.txt
file. When executed, it checks if a job with the name PXMASTER
is running. If so, the script exits. If not, it proceeds to schedule the jobs according to the MAXJOBS
parameter defined in the configuration file.
To run the script:
./phnx.sh
The phnx_status.sh script is used to monitor the progress of the tasks being executed. It reads the phnx_info.txt file generated by the phnx.sh script and calculates the number of tasks completed, the percentage of completion, the average time per task, and the estimated time to completion.
To run the script:
./phnx_status.sh
If you want to check the status of tasks in a specific output directory, pass the directory as an argument:
./phnx_status.sh phnxrun_20230422-123456
- Create a
config.sh
file with the appropriate configuration parameters. - Create a
queue.txt
file containing a list of commands to be executed, one per line:
echo "Task 1"
echo "Task 2"
sleep 5
echo "Task 3"
- Run the
phnx.sh
script to start scheduling jobs:
./phnx.sh
- Monitor the progress of the tasks using the
phnx_status.sh
script:
./phnx_status.sh <newly_created_phnx_directory>
This project is licensed under the GNU Affero General Public License v3.0