This is a TensorFlow implementation of the WaveNet generative neural network architecture for image generation.
|
Originally, the WaveNet neural network architecture directly generates a raw audio waveform, showing excellent results in text-to-speech and general audio generation (see the DeepMind blog post and paper for details). This network models the conditional probability to generate the next sample in the audio waveform, given all previous samples and possibly additional parameters.
After an audio preprocessing step, the input waveform is quantized to a fixed integer range.
The integer amplitudes are then one-hot encoded to produce a tensor of shape A convolutional layer that only accesses the current and previous inputs then reduces the channel dimension. The core of the network is constructed as a stack of causal dilated layers, each of which is a dilated convolution (convolution with holes), which only accesses the current and past audio samples. The outputs of all layers are combined and extended back to the original number of channels by a series of dense postprocessing layers, followed by a softmax function to transform the outputs into a categorical distribution. The loss function is the cross-entropy between the output for each timestep and the input at the next timestep. In this repository, the network implementation can be found in wavenet.py. |
|
This work is based in the implementation of the original WaveNet model (Wavenet), but with some modifications.
That's because we are going to use the WaveNet model as a image generator. We'll use raw pixel data (1D-channel), instead of raw audio files, and once the network is trained, we'll use the conditional probability finded to generate samples (pixels) into an autogenerative process.
Currently, there is no conditioning on extra information.