Skip to content

Latest commit

 

History

History
22 lines (14 loc) · 1.26 KB

File metadata and controls

22 lines (14 loc) · 1.26 KB

Training Handwriting Recognition on MNIST Dataset with Huggingface Trainer

Setup

  1. If you are running OSS Metaflow install pip install metaflow; If you are runnning it on the Outerbounds Platform, run pip install outerbounds

  2. Install other dependencies

    pip install -r requirements.txt

Executing the Flow

python flow.py run  

Salient Features

  • Metaflow Checkpointing with Huggingface Callbacks: This flow uses the @checkpoint functionality in tandem withe Huggingface's callback mechanism. The MetaflowCheckpointCallback class can be plugged into any huggingface trainer called within a Metaflow flow execution to enable checkpointing.
  • Checkpoint Loading in subsequent @steps: The callback exposese the lastest_checkpoint reference object which is returned by Checkpoint().save() or current.checkpoint.save(). This reference object can be set as a data artifact in Metaflow and then be loaded in subsequent steps. The flow.py demonstrates how to load the checkpoint in the test step by calling @model(load="best_checkpoint").