GitHub - sinjoysaha/tiktokenizer-js: A static site for visualizing the GPT-2 Byte-Pair Encoding (BPE) tokenization, replicating the official OpenAI API

sinjoysaha / tiktokenizer-js Public

A static site for visualizing the GPT-2 Byte-Pair Encoding (BPE) tokenization, replicating the official OpenAI API

sinjoysaha.github.io/tiktokenizer-js/

Notifications

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
data		data
app.js		app.js
index.html		index.html
style.css		style.css

About

A static site for visualizing the GPT-2 Byte-Pair Encoding (BPE) tokenization, replicating the official OpenAI API

sinjoysaha.github.io/tiktokenizer-js/