At its core, a GPT model that can take a text file from anywhere on the internet or from local files and imitate the linguistic style of the text

Synthesizer Self-Attention is a very recent alternative to causal self-attention that has potential benefits by removing this dot product.

Annotated vanilla implementation in PyTorch of the Transformer model introduced in 'Attention Is All You Need'.

Attention is all you need with Pytorch

Deployed locally

Implementation of Multihead attention mechanism using numpy and pyTorch

Machine Translation models (with and without attention) to convert sentences in Tamil to Hindi. Transformer models are also used for this same task and performance is compared.

PyTorch implementation of the Transformer architecture from the paper Attention is All You Need. Includes implementation of attention mechanism.

Official implementation of the paper "FedLSF: Federated Local Graph Learning via Specformers"

A repository for implementations of attention mechanism by PyTorch.

This is implementation of famous multi head attention mode for conversational ai paper. This model is trained on both Cornell movie data set and WikkiQna data set provided by microsoft

3D Printing Extrusion Detection using Multi-Head Attention Model

Implementing a GPT (Generative Pre-trained Transformer) model from scratch on Shakespeare's work.

Simple GPT with multiheaded attention for char level tokens, inspired from Andrej Karpathy's video lectures : https://github.com/karpathy/ng-video-lecture

Transformer model based on the research paper: "𝗔𝘁𝘁𝗲𝗻𝘁𝗶𝗼𝗻 𝗜𝘀 𝗔𝗹𝗹 𝗬𝗼𝘂 𝗡𝗲𝗲𝗱"

A Decoder-only Transfomer model for text generation.

The implementation of transformer as presented in the paper "Attention is all you need" from scratch.

This package is a Tensorflow2/Keras implementation for Graph Attention Network embeddings and also provides a Trainable layer for Multihead Graph Attention.

Testing the Reproducibility of the paper: MixSeq. Under the assumption that macroscopic time series follow a mixture distribution, they hypothesise that lower variance of constituting latent mixture components could improve the estimation of macroscopic time series.

This repository contains the code for a Multi Scale attention based module that was built and tested on a data set containing Concrete crack images. It was later tested with other data sets as well. Provided a better accuracy compared to the standard approach.