Simple Implementation of Rotary Positional Embeddings (RoPE) and Sinusoidal Positional Embeddings in JAX
-
Updated
Feb 6, 2024 - Python
#
rotary-embeddings
Here are 2 public repositories matching this topic...
Comparison of RoPE and xPos positional embeddings used in LLMs
-
Updated
Jun 16, 2024 - Python
Improve this page
Add a description, image, and links to the rotary-embeddings topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the rotary-embeddings topic, visit your repo's landing page and select "manage topics."