From 24257b767bc09a0b67fbba2e51954514d0b13452 Mon Sep 17 00:00:00 2001 From: Rui Campos Date: Thu, 21 Mar 2024 13:38:07 +0000 Subject: [PATCH] Update README.md Signed-off-by: Rui Campos --- README.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 8ace235..b035e8d 100644 --- a/README.md +++ b/README.md @@ -4,8 +4,9 @@ Speak with a language model. --- -## Cuda Kernel +## Cuda Kernel of the Metric Tensor Attention +### Determination of the analytic expression Let $P$ be a projection of a sequence $x$ of $c$ embeddings from $R^d$ onto $n$ spaces of dimension $k$, expressed in tensor notation by @@ -73,6 +74,10 @@ Computation of the gradients is straightforward, $$ \partial_{\bar M^n _ {l}} q^{nul} _ {l} = \frac {q^{nul} _ {l}}{\bar M^n _ {l}} $$ +### Implementation details + + + ----