TY - JOUR
T1 - TWIST-GAN
T2 - Towards Wavelet Transform and Transferred GAN for Spatiooral Single Image Super Resolution
AU - Dharejo, Fayaz Ali
AU - Deeba, Farah
AU - Zhou, Yuanchun
AU - Das, Bhagwan
AU - Jatoi, Munsif Ali
AU - Zawish, Muhammad
AU - Du, Yi
AU - Wang, Xuezhi
N1 - Funding Information:
Fayaz Ali Dharejo and Farah Deeba contributed equally to this research. This work was supported in part by the Key Research Program of Frontier Sciences, CAS, and Grant number ZDBS-LY-DQC016, Beijing Natural Science Foundation under Grant No. 4212030, Beijing Nova Program of Science and Technology under Grant No. Z191100001119090, Natural Science Foundation of China under Grant No. 61836013 and, Youth Innovation Promotion Association CAS. Authors’ addresses: F. A. Dharejo, F. Deeba, Y. Zhou (corresponding author), Y. Du (corresponding author), and X. Wang, Computer Network Information Center, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Beijing, 100190, China; emails: {fayazdharejo, deeba, zyc, duyi}@cnic.cn; B. Das, Department of Electronic Engineering, Quaid-e-Awam University Engineering Science and Technology, Nawasbshah, Sindh, 67450 Pakistan; email: engr.bhagwandas@ hotmail.com; M. A. Jatoi, Department of Biomedical Engineering, Salim Habib University, Karachi, Sindh, 74900 Pakistan; email: [email protected]; M. Zawish, Walton Institute for Information and Communication Systems Science, Waterford Institute of Technology, Waterford, X91WR86, Ireland; email: [email protected]. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. © 2021 Association for Computing Machinery. 2157-6904/2021/12-ART71 $15.00 https://doi.org/10.1145/3456726
Publisher Copyright:
© 2021 Association for Computing Machinery.
PY - 2021/12
Y1 - 2021/12
N2 - Single Image Super-resolution (SISR) produces high-resolution images with fine spatial resolutions from a remotely sensed image with low spatial resolution. Recently, deep learning and generative adversarial networks (GANs) have made breakthroughs for the challenging task of single image super-resolution (SISR). However, the generated image still suffers from undesirable artifacts such as the absence of texture-feature representation and high-frequency information. We propose a frequency domain-based spatiooral remote sensing single image super-resolution technique to reconstruct the HR image combined with generative adversarial networks (GANs) on various frequency bands (TWIST-GAN). We have introduced a new method incorporating Wavelet Transform (WT) characteristics and transferred generative adversarial network. The LR image has been split into various frequency bands by using the WT, whereas the transfer generative adversarial network predicts high-frequency components via a proposed architecture. Finally, the inverse transfer of wavelets produces a reconstructed image with super-resolution. The model is first trained on an external DIV2 K dataset and validated with the UC Merced Landsat remote sensing dataset and Set14 with each image size of 256 × 256. Following that, transferred GANs are used to process spatiooral remote sensing images in order to minimize computation cost differences and improve texture information. The findings are compared qualitatively and qualitatively with the current state-of-art approaches. In addition, we saved about 43% of the GPU memory during training and accelerated the execution of our simplified version by eliminating batch normalization layers.
AB - Single Image Super-resolution (SISR) produces high-resolution images with fine spatial resolutions from a remotely sensed image with low spatial resolution. Recently, deep learning and generative adversarial networks (GANs) have made breakthroughs for the challenging task of single image super-resolution (SISR). However, the generated image still suffers from undesirable artifacts such as the absence of texture-feature representation and high-frequency information. We propose a frequency domain-based spatiooral remote sensing single image super-resolution technique to reconstruct the HR image combined with generative adversarial networks (GANs) on various frequency bands (TWIST-GAN). We have introduced a new method incorporating Wavelet Transform (WT) characteristics and transferred generative adversarial network. The LR image has been split into various frequency bands by using the WT, whereas the transfer generative adversarial network predicts high-frequency components via a proposed architecture. Finally, the inverse transfer of wavelets produces a reconstructed image with super-resolution. The model is first trained on an external DIV2 K dataset and validated with the UC Merced Landsat remote sensing dataset and Set14 with each image size of 256 × 256. Following that, transferred GANs are used to process spatiooral remote sensing images in order to minimize computation cost differences and improve texture information. The findings are compared qualitatively and qualitatively with the current state-of-art approaches. In addition, we saved about 43% of the GPU memory during training and accelerated the execution of our simplified version by eliminating batch normalization layers.
KW - neural networks
KW - spatiooral
KW - super resolution
KW - Wavelet transform
UR - http://www.scopus.com/inward/record.url?scp=85123956833&partnerID=8YFLogxK
U2 - 10.1145/3456726
DO - 10.1145/3456726
M3 - Article
AN - SCOPUS:85123956833
SN - 2157-6904
VL - 12
JO - ACM Transactions on Intelligent Systems and Technology
JF - ACM Transactions on Intelligent Systems and Technology
IS - 6
M1 - 71
ER -