Balancing Groq API Limits is no fun, but its fun too ? #5
Drlordbasil
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
If I didn't run into free limits I'd be making this script less about small context and more about flow, but I will be working on the local model inference side to just do everything with full context windows in each api call, but using 6k tokens each api call means it has to sleep alottt. I avoid errors for most part now.
btw new whisper3 addition from Groq has rate limits too, or I'd be making it a different sped up groq only based script. If anyone from groq wants to give me unlimited API access, I'd love you.
Beta Was this translation helpful? Give feedback.
All reactions