You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hey,
I you mentioned this implementation was heavily inspired by the Huggingface one. I was wondering if it's possible to load a tokenizer trained with huggingface with this implementation ?
If it is, is there any gotchas to be aware of ?
I thought this information was lacking from the doc.
Thank you
The text was updated successfully, but these errors were encountered:
Hey,
I you mentioned this implementation was heavily inspired by the Huggingface one. I was wondering if it's possible to load a tokenizer trained with huggingface with this implementation ?
If it is, is there any gotchas to be aware of ?
I thought this information was lacking from the doc.
Thank you
The text was updated successfully, but these errors were encountered: