Very easy and efficient regularization technique for any regression tasks
DisturbValue injects noise into a portion of target values at random to alleviate the overfitting problem. The reference paper shows that the method outperforms L2 regularization and Dropout and that the best performance is achieved in more than half the datasets by combining our methods with either L2 regularization or Dropout.
pip install dvreg
alpha: maximum disturbance rate in [0,1] (When alpha=0, there is no disturbance.)
sigma: standard deviation to generate Gaussian noise
import torch
from disturbvalue import disturbvalue
...
reg = disturbvalue.DisturbValue(alpha=.3, sigma=1e-2)
num_epochs = 1000
...
for i in range(num_epochs):
for data, targets in dataloader:
...
pred = model(data)
dtargets = reg.dv(targets)
loss = criterion(pred, dtargets)
...