Here is the collection of my open-source work:
Open Source Contributions to HuggingFace:
- Added PyTorch
BigBird-Pegasusto Transformers - Added PyTorch
BigBird-RoBERTato Transformers - Added Flax/Jax
BigBird-RoBERTato Transformers - Added script for training
FlaxBigBirdon natural questions to Transformers - Integrated
Microsoft's DeepSpeedwith Accelerate - Added
ModelHubMixinin Hub
Open Source Contributions to TensorFlow (Google):
- Exported fine-tuned Wav2Vec2 model to TFHub
- Exported pre-trained Wav2Vec2 model to TFHub
- Added notebook for demonstrating Wav2Vec2 fine-tuning to TFHub
- Implemented & trained Wav2Vec2 model in TensorFlow
Blogs/Posters:
- Optimizing adapters for NMT
- Understanding BigBird's Block Sparse Attention
- BioBigBird: Leveraging Entire Articles for Biomedical Language Understanding
Talks/Lectures:
- BigBird: Transformers on long sequences
- Deep Dive into Pre-trained Transformers
- CUDA Concepts YouTube Series
Tutorials:
Flax BigBirdevaluation on natural-questions datasetPyTorch BigBirdevaluation on natural-questions datasetPyTorch BigBirdPegasusevaluation on PubMed dataset- How to use BigBird (RoBERTa & Pegasus) for inference
- Template for fine-tuning a pre-trained Wav2Vec2 SavedModel
- Conversion of TF Wav2Vec2 model to ONNX and compares the latency of ONNX exported model & TF model on CPU
- Wav2Vec2 evaluation (without any padding) on LibriSpeech data
- Wav2Vec2 SavedModel evaluation (with constant padding upto 246000 length) on LibriSpeech data
- Small demo of how to use Wav2Vec2 for inference for ASR task
Please visit my webpage: https://thevasudevgupta.com/ to know more about my work.