For the future!

[qwertystars]

Is it possible to make a hierarchical system of models where one model classify the prompt into categories and then choose that specific model to respond to that prompt and when a prompt does not fit into any category then it will default to a model which will access the Internet for more info and rate which source is reliable and run another model which would parse the info and categorise it and add the data to one of the existing models so this way it trains it self and since we run only the models required to answer then we save on compute and storage resources? Or am I just thinking too much?

[thaumstrial]

Interesting idea!

With multiple models, however, either one model is loaded and one model is unloaded per conversation (which probably takes more time on loading and unloading than computation time),
or all models are loaded (which requires a surprisingly large amount of RAM), and one model is chosen to answer at a time.

So this does not seem to be cost-effective, and splitting into multiple models leads to sparse weights. Perhaps you can try to target training on different parts of the model, starting with one part of the model at a time.

Have you tried it on a small model?

[qwertystars]

But would it take more time when all of them are running on different server and connected to each other via internet or locally?
I mean that if we were to make a research preview like ChatGPT would it be a better approach?

[bitplane]

Mick might disagree here, but If you look at langchain, which seems to be what we need, models and agents are split up. An agent could use multiple models and manage its context on a per-user or per-task basis. So there would hopefully be no need to use multiple models if the main one is good enough, just have different use cases.