Uncertainty-aware Labelled Augmentations for High Dimensional Latent Space Bayesian Optimisation

Paper : https://openreview.net/pdf?id=C7pY5Wjwk0d

Abstract

Black-box optimization problems are ubiquitous and of importance in many critical areas of science and engineering. Bayesian optimisation (BO) over the past years has emerged as one of the most successful techniques for optimising expensive black-box objectives. However, efficient scaling of BO to high-dimensional settings has proven to be extremely challenging. Traditional strategies based on projecting high-dimensional input data to a lower-dimensional manifold, such as Variational autoencoders (VAE) and Generative adversarial networks (GAN) have improved BO performance in high-dimensional regimes, but their dependence on excessive labeled input data has been widely reported. In this work, we target the data-greedy nature of deep generative models by constructing uncertainty-aware task-specific labeled data augmentations using Gaussian processes (GPs). Our approach outperforms existing state-of-the-art methods on machine learning tasks and demonstrates more informative data representation with limited supervision.

Code

Calculate_objective.py

Create inital data points for bayesian optimisation.

Set sort_points to True to get the highest valued initial data points to run subsequent optimisation.

Usage

python ./MNIST_Norm_Task/scripts/Calculate_objective.py --dataset_path 'mnist_x.npy' \
                                               --label_path 'mnist_y.npy' \
                                               --save_dir 'path to save the initial data for BO' \
                                               --sort_points 'True'

GP_augment_norm_task.py

Attributes

To run desired experiments, set appropriate flags (GP_aug_flag, naive_aug_flag, etc.) to True as the argument.

More implementation details can be found in the code itself.

result_dir : Directory to save results

data_val_path : File path for validation data

data_dir : Directory which contains the initial data points

vae_model : File path for pretrained vae model

Usage

python ./MNIST_Norm_Task/scripts/GP_augment_norm_task.py --result_dir 'path to save results' \
                                               --data_val_path 'Created in Calculate_objective script' \
                                               --data_dir 'Directory path for inital data points' \
                                               --vae_model 'pretained vae model' \
                                               --naive_aug_flag 'True'