On Optimal Caching and Model Multiplexing for Large Model Inference

Installation

pip install -e .

cd inferband
python run_exp.py
python plot.py

Name		Name	Last commit message	Last commit date
Latest commit History 2 Commits
inferband		inferband
real_run		real_run
train		train
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
setup.py		setup.py
sim_cache.ipynb		sim_cache.ipynb