MLOps with Neptune

Intro

Neptune is a platform used to monitoring and recording information from machine learning experiments, often referred to as MLOps. It is popular with academic researchers and industry ML engineers alike because of its robust model tracking capabilities. As a researcher, you will often find yourself running a model dozens of times, monitoring loss over time, tuning hyperparameters and model architecture, and comparing various models and datasets for a given research problem. Neptune provides a convenient graphical interface to remotely monitor running models and store experiment results without relying on manually writing logging code.

Getting Access

Adding Neptune to a Pytorch Project

...

Installing Libraries

...

Initialize Neptune

Firstly, store your project name and api token in a file, you can name it neptune.ini.

[CLIENT_INFO]
project_id= ** project id **
api_token= ** api token **

Then in your main python script:

import neptune
import configparser
# process credentials
def _process_api_key(f_key: str) -> configparser.ConfigParser:
    api_key = configparser.ConfigParser()
    api_key.read(f_key)
    return api_key

creds = _process_api_key('./neptune.ini')

# create a neptune run
run = neptune.init_run(
    project=creds['CLIENT_INFO']['project_id'],
    api_token=creds['CLIENT_INFO']['api_token']
)

Neptune Callback

from neptune_pytorch import NeptuneLogger

Adding Images

from neptune.types import File