Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] SmoothL1Loss correctly mentioned instead of Huber #1458

Merged
merged 6 commits into from
Oct 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ If it looks like the model is overfitting to the training data (the live loss pl
you can reduce `epochs` and `learning_rate`, and potentially increase the `batch_size`.
If it is underfitting, the number of `epochs` and `learning_rate` can be increased and the `batch_size` potentially decreased.

The default loss function is the 'Huber' loss, which is considered to be robust to outliers.
The default loss function is the 'SmoothL1Loss' loss, which is considered to be robust to outliers.
However, you are free to choose the standard `MSE` or any other PyTorch `torch.nn.modules.loss` loss function.

## Increasing Depth of the Model
Expand Down
4 changes: 2 additions & 2 deletions docs/zh/超参数选取.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ NeuralProphet有一些超参数需要用户指定。如果没有指定,将使
| `learning_rate` | None |
| `epochs` | None |
| `batch_size` | None |
| `loss_func` | Huber |
| `loss_func` | SmoothL1Loss |
| `train_speed` | None |
| `normalize_y` | auto |
| `impute_missing` | True |
Expand All @@ -43,7 +43,7 @@ NeuralProphet采用随机梯度下降法进行拟合--更准确地说,是采

如果看起来模型对训练数据过度拟合(实时损失图在此很有用),可以减少 `epochs` 和 `learning_rate`,并有可能增加 `batch_size`。如果是低拟合,可以增加`epochs` 和`learning_rate` 的数量,并有可能减少`batch_size` 。

默认的损失函数是 "Huber "损失,该函数被认为对离群值具有鲁棒性。但是,您可以自由选择标准的 "MSE "或任何其他PyTorch `torch.nn.modules.loss`损失函数。
默认的损失函数是 "SmoothL1Loss "损失,该函数被认为对离群值具有鲁棒性。但是,您可以自由选择标准的 "MSE "或任何其他PyTorch `torch.nn.modules.loss`损失函数。

## 增加模型的深度

Expand Down
7 changes: 4 additions & 3 deletions neuralprophet/configure.py
Original file line number Diff line number Diff line change
Expand Up @@ -117,9 +117,10 @@ def __post_init__(self):

def set_loss_func(self):
if isinstance(self.loss_func, str):
if self.loss_func.lower() in ["huber", "smoothl1", "smoothl1loss"]:
self.loss_func = torch.nn.SmoothL1Loss(reduction="none")
elif self.loss_func.lower() in ["mae", "l1", "l1loss"]:
if self.loss_func.lower() in ["smoothl1", "smoothl1loss", "huber"]:
# keeping 'huber' for backwards compatiblility, though not identical
self.loss_func = torch.nn.SmoothL1Loss(reduction="none", beta=1.0)
elif self.loss_func.lower() in ["mae", "maeloss", "l1", "l1loss"]:
self.loss_func = torch.nn.L1Loss(reduction="none")
elif self.loss_func.lower() in ["mse", "mseloss", "l2", "l2loss"]:
self.loss_func = torch.nn.MSELoss(reduction="none")
Expand Down
4 changes: 2 additions & 2 deletions neuralprophet/forecaster.py
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@
Type of loss to use:

Options
* (default) ``Huber``: Huber loss function
* (default) ``SmoothL1Loss``: SmoothL1 loss function
* ``MSE``: Mean Squared Error loss function
* ``MAE``: Mean Absolute Error loss function
* ``torch.nn.functional.loss.``: loss or callable for custom loss, eg. L1-Loss
Expand Down Expand Up @@ -360,7 +360,7 @@
learning_rate: Optional[float] = None,
epochs: Optional[int] = None,
batch_size: Optional[int] = None,
loss_func: Union[str, torch.nn.modules.loss._Loss, Callable] = "Huber",
loss_func: Union[str, torch.nn.modules.loss._Loss, Callable] = "SmoothL1Loss",
optimizer: Union[str, Type[torch.optim.Optimizer]] = "AdamW",
newer_samples_weight: float = 2,
newer_samples_start: float = 0.0,
Expand Down Expand Up @@ -1005,7 +1005,7 @@
# Only display the plot if the session is interactive, eg. do not show in github actions since it
# causes an error in the Windows and MacOS environment
if matplotlib.is_interactive():
fig

Check warning on line 1008 in neuralprophet/forecaster.py

View workflow job for this annotation

GitHub Actions / pyright

Expression value is unused (reportUnusedExpression)

self.fitted = True
return metrics_df
Expand Down
2 changes: 1 addition & 1 deletion tests/test_configure.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ def generate_config_train_params(overrides={}):
"learning_rate": None,
"epochs": None,
"batch_size": None,
"loss_func": "Huber",
"loss_func": "SmoothL1Loss",
"optimizer": "AdamW",
}
for key, value in overrides.items():
Expand Down
2 changes: 1 addition & 1 deletion tests/test_uncertainty.py
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ def test_uncertainty_estimation_peyton_manning():

m = NeuralProphet(
n_forecasts=1,
loss_func="Huber",
loss_func="SmoothL1Loss",
quantiles=[0.01, 0.99],
epochs=EPOCHS,
batch_size=BATCH_SIZE,
Expand Down