01c_Usage_Examples_Colab.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ybKLhIYyLCsh"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/cidcom/SatelliteCloudGenerator/blob/main/01c_Usage_Examples_Colab.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lYU8X9_ELCsl"
      },
      "source": [
        "# Example Usage of SatelliteCloudGenerator\n",
        "☁️ The following are example use cases of the [SatelliteCloudGenerator tool](https://github.com/cidcom/SatelliteCloudGenerator).\n",
        "\n",
        "The interface is largely based in PyTorch and can be readily applied in other PyTorch projects. 🔥"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "p-377bkNLCsm"
      },
      "outputs": [],
      "source": [
        "%load_ext autoreload\n",
        "%autoreload 2\n",
        "%matplotlib inline"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xSFVgm_RLCsn"
      },
      "source": [
        "## Imports and installs...\n",
        "Please note the `--recurse-submodules` setting of `git clone` which is necessary to clone the internal submodule for [LocalGaussianBlur](https://github.com/mikonvergence/LocalGaussianBlur).\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "J7QhnkoNLCso"
      },
      "outputs": [],
      "source": [
        "! git clone --recurse-submodules https://github.com/cidcom/SatelliteCloudGenerator.git"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "SSGc_3QsLCso"
      },
      "outputs": [],
      "source": [
        "!pip install -q torch==1.10.0 torchvision numpy imageio kornia"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "KOUNik_LLCsp"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "sys.path.append('./SatelliteCloudGenerator/')\n",
        "import matplotlib as mpl\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "from src import *\n",
        "import imageio\n",
        "\n",
        "plt.rcParams[\"figure.figsize\"] = (20,10)\n",
        "\n",
        "mwir_img = imageio.imread('./SatelliteCloudGenerator/imgs/mwir_example.png')[:256,:256,0]/255 # Thermal Infrared Imagery - © [2022] Satellite Vu\n",
        "rgb_img = imageio.imread('./SatelliteCloudGenerator/imgs/rgb_example.png')[...,:3]/255"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DbdSb4YdLCsp"
      },
      "source": [
        "## Cloud Types: Thick Cloud"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "z7SFuNj3LCsq"
      },
      "outputs": [],
      "source": [
        "cl, mask = add_cloud(rgb_img,\n",
        "                     return_cloud=True\n",
        "                    )\n",
        "\n",
        "plt.subplot(1,3,1)\n",
        "plt.imshow(rgb_img)\n",
        "plt.title('Input')\n",
        "plt.subplot(1,3,2)\n",
        "plt.imshow(cl)\n",
        "plt.title('Simulated')\n",
        "plt.subplot(1,3,3)\n",
        "plt.imshow(mask)\n",
        "plt.title('Channel-wise Cloud Mask')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L7daQa8NLCsr"
      },
      "source": [
        "# Cloud Types: Thick Foggy Cloud"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OiIMOi1mLCsr"
      },
      "outputs": [],
      "source": [
        "cl, mask = add_cloud(rgb_img,\n",
        "                     min_lvl=0.1,\n",
        "                     max_lvl=0.5,\n",
        "                     decay_factor=1.85,\n",
        "                     return_cloud=True)\n",
        "\n",
        "plt.subplot(1,3,1)\n",
        "plt.imshow(rgb_img)\n",
        "plt.title('Input')\n",
        "plt.subplot(1,3,2)\n",
        "plt.imshow(cl)\n",
        "plt.title('Simulated')\n",
        "plt.subplot(1,3,3)\n",
        "plt.imshow(mask)\n",
        "plt.title('Channel-wise Cloud Mask')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TR5AS8rHLCss"
      },
      "source": [
        "# Cloud Types: Thin Fog"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LPTd62xXLCss"
      },
      "outputs": [],
      "source": [
        "cl, mask = add_cloud(rgb_img,\n",
        "                     min_lvl=0.4,\n",
        "                     max_lvl=0.6,\n",
        "                     decay_factor=1.5,\n",
        "                     return_cloud=True)\n",
        "\n",
        "plt.subplot(1,3,1)\n",
        "plt.imshow(rgb_img)\n",
        "plt.title('Input')\n",
        "plt.subplot(1,3,2)\n",
        "plt.imshow(cl)\n",
        "plt.title('Simulated')\n",
        "plt.subplot(1,3,3)\n",
        "plt.imshow(mask)\n",
        "plt.title('Channel-wise Cloud Mask')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "n1_upEVLLCst"
      },
      "source": [
        "# Thermal Image Example"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "aKs3UaJ9LCsu"
      },
      "outputs": [],
      "source": [
        "cl, mask = add_cloud(mwir_img,\n",
        "                     max_lvl = 0.6,                    \n",
        "                     invert=True,\n",
        "                     blur_scaling=1.0,\n",
        "                     return_cloud=True\n",
        "                    )\n",
        "plt.subplot(1,3,1)\n",
        "plt.imshow(mwir_img, cmap='gray')\n",
        "plt.title('Input')\n",
        "plt.subplot(1,3,2)\n",
        "plt.imshow(cl[...,0],vmin=0.0,vmax=1.0, cmap='gray')\n",
        "plt.title('Simulated')\n",
        "plt.subplot(1,3,3)\n",
        "plt.imshow(mask[...,0],vmin=0.0,vmax=1.0, cmap='gray')\n",
        "plt.title('Channel-wise Cloud Mask')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KU31wlQnLCsv"
      },
      "source": [
        "## Gradual Cloud Increase\n",
        "\n",
        "An interesting result can be obtained by repeatedly applying the cloud augmentation. Eventually, the image becomes dominated by the cloud component."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "r0e4SIiALCsv"
      },
      "outputs": [],
      "source": [
        "new = rgb_img\n",
        "\n",
        "plt.figure(figsize=(10,20))\n",
        "for step in range(32):\n",
        "    \n",
        "    plt.subplot(8,4,1+step)\n",
        "    plt.imshow(new.clip(0,1))\n",
        "    \n",
        "    lims = sorted([0.6+0.4*el for el in np.random.rand(2).tolist()])\n",
        "    \n",
        "    new = add_cloud(new,\n",
        "                    min_lvl=0.5,\n",
        "                    max_lvl=1.0)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aMKINurnLCsw"
      },
      "source": [
        "# Parameter Ablations\n",
        "## 1. Basic Cloud\n",
        "\n",
        "A basic cloud with `max_lvl` of ground component (`1.0` guarantees some clear pixels) and `min_lvl` that controles the maximum value of cloud strength (`1.0` guarantees some pixels with non-transparent cloud)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "k_OBiSwMLCsx"
      },
      "outputs": [],
      "source": [
        "cl, mask = add_cloud(rgb_img,\n",
        "                     min_lvl=0.0,\n",
        "                     max_lvl=1.0,\n",
        "                     cloud_color=False,\n",
        "                     channel_offset=0,\n",
        "                     blur_scaling=0,\n",
        "                     return_cloud=True\n",
        "                    )\n",
        "\n",
        "plt.subplot(1,3,1)\n",
        "plt.imshow(rgb_img)\n",
        "plt.title('Input')\n",
        "plt.subplot(1,3,2)\n",
        "plt.imshow(cl)\n",
        "plt.title('Simulated')\n",
        "plt.subplot(1,3,3)\n",
        "plt.imshow(mask)\n",
        "plt.title('Channel-wise Cloud Mask')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Rq42AzsJLCsx"
      },
      "source": [
        "## 2. Cloud Color\n",
        "\n",
        "The color of the cloud component (white by default) can be adjusted with the average color from the clear underlying image by setting `cloud_color` to `True`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LglwFmyBLCsy"
      },
      "outputs": [],
      "source": [
        "cl, mask = add_cloud(rgb_img,\n",
        "                     min_lvl=0.0,\n",
        "                     max_lvl=1.0,\n",
        "                     cloud_color=True,\n",
        "                     channel_offset=0,\n",
        "                     blur_scaling=0,\n",
        "                     return_cloud=True\n",
        "                    )\n",
        "plt.subplot(1,3,1)\n",
        "plt.imshow(rgb_img)\n",
        "plt.title('Input')\n",
        "plt.subplot(1,3,2)\n",
        "plt.imshow(cl)\n",
        "plt.title('Simulated')\n",
        "plt.subplot(1,3,3)\n",
        "plt.imshow(mask)\n",
        "plt.title('Channel-wise Cloud Mask')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "E9bRvogFLCsy"
      },
      "source": [
        "## 3. Channel Offset\n",
        "\n",
        "The `channel_offset` setting controls the maximum positional offset in `x` or `y` direction between the channel-specific cloud masks and the base cloud mask (The value is sampled uniformly from the range `[-channel_offset,+channel_offset]`)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BK3dUCfYLCsz"
      },
      "outputs": [],
      "source": [
        "cl, mask = add_cloud(rgb_img,\n",
        "                     min_lvl=0.0,\n",
        "                     max_lvl=1.0,\n",
        "                     cloud_color=False,\n",
        "                     channel_offset=3,\n",
        "                     blur_scaling=0,\n",
        "                     return_cloud=True\n",
        "                    )\n",
        "\n",
        "plt.subplot(1,3,1)\n",
        "plt.imshow(rgb_img)\n",
        "plt.title('Input')\n",
        "plt.subplot(1,3,2)\n",
        "plt.imshow(cl)\n",
        "plt.title('Simulated')\n",
        "plt.subplot(1,3,3)\n",
        "plt.imshow(mask)\n",
        "plt.title('Channel-wise Cloud Mask')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TTXhHpbULCsz"
      },
      "source": [
        "## 4. Blur Under Cloud\n",
        "\n",
        "By setting blur_scaling argument to a value different than `0.0`, a locally varying Gaussian blur can be applied. The standard deviation is proportional to the cloud strength (ranging between `0` and `1`) multplied by the `blur_scaling`\n",
        "\n",
        " factor."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "BB0EJcmCLCs0"
      },
      "outputs": [],
      "source": [
        "cl, mask = add_cloud(rgb_img,\n",
        "                     min_lvl=0.3,\n",
        "                     max_lvl=1.0,\n",
        "                     cloud_color=False,\n",
        "                     channel_offset=0,\n",
        "                     blur_scaling=4.0,\n",
        "                     return_cloud=True\n",
        "                    )\n",
        "\n",
        "plt.subplot(1,3,1)\n",
        "plt.imshow(rgb_img)\n",
        "plt.title('Input')\n",
        "plt.subplot(1,3,2)\n",
        "plt.imshow(cl)\n",
        "plt.title('Simulated')\n",
        "plt.subplot(1,3,3)\n",
        "plt.imshow(mask)\n",
        "plt.title('Channel-wise Cloud Mask')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4RR-gwmvLCs0"
      },
      "outputs": [],
      "source": []
    }
  ],
  "metadata": {
    "language_info": {
      "name": "python",
      "pygments_lexer": "ipython3"
    },
    "colab": {
      "provenance": []
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}