Replies: 1 comment
-
Several parameters might affect the required GPU memory. So you should try it out yourself with your typical use cases. |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I want to know how much GPU memory would it need to run Yi-6b-200k and Yi-34b-200k for production environment?
I tried to run Yi-6b and Yi-6b-200k on 12GB GPU locally, but it is giving me out of memory exception.
Beta Was this translation helpful? Give feedback.
All reactions