Hyperparameter Optimization Pipelines and Supporting Tools

[GabrielBG0]

Change Log

This PR implements a series of new features:

Transforms

• new Crop
• new Normalize
• new Transpose
• new Resize
• new EmptyTransform
• new _RandomSyncedTransform
• new RandomResize
• new RandomFlip
• new ClassRatioCrop

Metrics

• new CroppedMetric
• new ResizedMetric

Callbacks

• new TrainerReportKeepOnlyLastCallback
• new TrainerReportOnIntervalCallback

Datasets

• SupervisedReconstructionDataset now supports support_context_transforms

Engines

• new _Engine
• new PatchInferencer

Models

• SETR_PUP now supports optimizers as parameters
• SETR_PUP now supports selection of the aux output layers and aux weights
• SETR_PUP now supports loading backbone weights and freezing backbone weights
• SETR_PUP now supports weighted losses
• SETR_PUP now supports interpolation of loaded weights input size
• SETR_PUP now supports different learning rates for backbone and head
• SETR_PUP now supports different prediction engines for val and test
• _VisionTransformerBackbone now supports selection of the aux output layers
• _VisionTransformerBackbone now supports loading backbone weights and freezing backbone weights
• _VisionTransformerBackbone now supports interpolation of loaded weights input size

Pipelines

• new RayHyperParameterSearch
• new HyperoptHyperParameterSearch

Other Improvements

• General typo fixes
• General typing fixes

Issue ticket number and link (If apply)

#48

Checklist before requesting a review

• I have performed a self-review of my code.
• I have added tests to this functionality.
• All tests are passing.
• I have updated the documentation as needed.
• I have followed the project's coding guidelines.