Code Refactoring

[nsosio]

Description

This pull request fixes #66 by introducing a reorganisation of the benchmarking structure, removing code-wise distinctions for different types of benchmarks (python_bench, rust_bench). The new structure uses one folder for each benchmark, denoted as bench_{bench_name}, and mandates the inclusion of a bench.sh bash script in each folder. This script is responsible for handling all necessary setup, environment configuration, and running the benchmark.

Changes Made

• Removed Code-wise Distinctions:

  • Eliminated distinctions based on the programming language (python, rust) for benchmarking.

• New Benchmark Folder Structure:

  • Introduced a folder structure with the naming convention bench_{bench_name} for each benchmark.
  • Each benchmark folder must contain a bench.sh script to handle setup, environment configuration, and benchmark execution.

• Benchmark Script Parameters:

  • Updated the bench.sh script to support the following parameters:
    • prompt
    • max_tokens
    • repetitions
    • log_file
    • device
    • models_dir

• General Benchmark Script:

  • Added a general script, bench.sh, that downloads necessary files and iterates through all benchmark folders.
  • The general script runs the specified benchmark for the specified device (cpu, cuda, metal) if the relevant benchmark script exists.

Usage

To run a specific benchmark, navigate to the corresponding benchmark folder (e.g., bench_{bench_name}) and execute the bench.sh script with the required parameters.

Example:

./bench.sh --prompt <value> --max_tokens <value> --num_repetitions <value> --log_file <file_path> --device <cpu/cuda/metal> --models_dir <path_to_models>

[filopedraz]

Include the Usage you provided in this PR in the README as well :)

[nsosio]

@filopedraz it has already been updated, in the end the entry point has only slightly changed