GAN Graphics Real

^{image source}

Distinguishing Natural and Computer-Generated Images using Multi-Colorspace fused EfficientNet
Manjary P Gangan, Anoop K, and Lajish V L
Department of Computer Science
University of Calicut, India

📝 Paper : https://www.sciencedirect.com/science/article/abs/pii/S2214212622001247
🌏 Link: https://dcs.uoc.ac.in/cida/projects/dif/mceffnet.html

Abstract: The problem of distinguishing natural images from photo-realistic computer-generated ones either addresses natural images versus computer graphics or natural images versus GAN images, at a time. But in a real-world image forensic scenario, it is highly essential to consider all categories of image generation, since in most cases image generation is unknown. We, for the first time, to our best knowledge, approach the problem of distinguishing natural images from photo-realistic computer-generated images as a three-class classification task classifying natural, computer graphics, and GAN images. For the task, we propose a Multi-Colorspace fused EfficientNet model by parallelly fusing three EfficientNet networks that follow transfer learning methodology where each network operates in different colorspaces, RGB, LCH, and HSV, chosen after analyzing the efficacy of various colorspace transformations in this image forensics problem. Our model outperforms the baselines in terms of accuracy, robustness towards post-processing, and generalizability towards other datasets. We conduct psychophysics experiments to understand how accurately humans can distinguish natural, computer graphics, and GAN images where we could observe that humans find difficulty in classifying these images, particularly the computer-generated images, indicating the necessity of computational algorithms for the task. We also analyze the behavior of our model through visual explanations to understand salient regions that contribute to the model's decision making and compare with manual explanations provided by human participants in the form of region markings, where we could observe similarities in both the explanations indicating the powerful nature of our model to take the decisions meaningfully.

For other inquiries, please contact:
Manjary P Gangan 📧 manjaryp_dcs@uoc.ac.in 🌏 website
Anoop K 📧 anoopk_dcs@uoc.ac.in 🌏 website
Lajish V L 📧 lajish@uoc.ac.in 🌏 website

Citation

@article{gangan2022distinguishing,
  title={Distinguishing natural and computer generated images using Multi-Colorspace fused EfficientNet},
  author={Manjary {P. Gangan} and Anoop K. and Lajish {V. L.}},
  journal={Journal of Information Security and Applications},
  volume={68},
  pages={103261},
  year={2022},
  publisher={Elsevier},
  issn = {2214-2126},
  doi = {https://doi.org/10.1016/j.jisa.2022.103261},
  url = {https://www.sciencedirect.com/science/article/pii/S2214212622001247}
}

Acknowledgement

This work was supported by the Women Scientist Scheme-A (WOS-A) for Research in Basic/Applied Science from the Department of Science and Technology (DST) of the Government of India

* image source: The first (leftmost) image is GAN generated using StyleGAN2, available at: link, accessed: 2021-10-21 || the second (middle) image is Computer Graphics generated, available at: link, accessed: 2021-10-21 || and the third (rightmost) is a natural image, available at: Computer Graphics versus Photographs dataset link, accessed: 2021-10-21