Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
Signed-off-by: Rui Campos <mail@ruicampos.org>
  • Loading branch information
RuiFilipeCampos authored Mar 21, 2024
1 parent 74dfdb3 commit 24257b7
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,9 @@ Speak with a language model.

---

## Cuda Kernel
## Cuda Kernel of the Metric Tensor Attention

### Determination of the analytic expression

Let $P$ be a projection of a sequence $x$ of $c$ embeddings from $R^d$ onto $n$ spaces of dimension $k$, expressed in tensor notation by

Expand Down Expand Up @@ -73,6 +74,10 @@ Computation of the gradients is straightforward,
$$ \partial_{\bar M^n _ {l}} q^{nul} _ {l} = \frac {q^{nul} _ {l}}{\bar M^n _ {l}} $$


### Implementation details




----

Expand Down

0 comments on commit 24257b7

Please sign in to comment.