Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using local Models #17

Open
zzadiues opened this issue Oct 16, 2023 · 1 comment
Open

Using local Models #17

zzadiues opened this issue Oct 16, 2023 · 1 comment

Comments

@zzadiues
Copy link

Hi @BrettlyCD , thanks for the really instructive repo. I am having a hard time getting a local model to run this application.
Have you tried a version that utilizes a local model ?

@BrettlyCD
Copy link
Owner

Hi, I'm sorry for just now seeing this!

I've done a little bit of testing with some other projects on local models. My laptop doesn't have a ton of processing power, so I didn't have a ton of luck.

I'm finding the model restriction to really be the challenge here - using HuggingFace you are limited to the amount of input tokens. Using a local model you are restricted to just not having the best model due to size.

I have been prioritizing some other projects recently, but would love to get back into this and do some more exploring. Thank you for the question!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants