Aerial to Street View Image Translation using Cascaded Conditional GANs

Kshitij Singh, Alexia Briassouli, Mirela Popa

2022

Abstract

Cross view image translation is a challenging case of viewpoint translation which involves generating the street view image when the aerial view image is given and vice versa. As there is no overlap in the two views, a single stage generation network fails to capture the complex scene structure of objects in these two views. Our work aims to tackle the task of generating street level view images from aerial view images on the benchmarking CVUSA dataset by a cascade pipeline consisting of three smaller stages: street view image generation, semantic segmentation map generation, and image refinement, trained together in a constrained manner in a Conditional GAN (CGAN) framework. Our contributions are twofold: (1) The first stage of our pipeline examines the use of alternate architectures ResNet, ResUnet++ in a framework similar to the current State-of-the-Art (SoA), leading to useful insights and comparable or improved results in some cases. (2) In the 3rd stage, ResUNet++ is used for the first time for image refinement. U-net performs the best for street view image generation and semantic map generation as a result of the skip connections between encoders and decoders, while ResU-Net++ performs the best for image refinement because of the presence of the attention module in the decoders. Qualitative and quantitative comparisons with existing methods show that our model outperforms all others on the KL Divergence metric and ranks amongst the best for other metrics.

Paper Citation

in Harvard Style

Singh K., Briassouli A. and Popa M. (2022). Aerial to Street View Image Translation using Cascaded Conditional GANs. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP; ISBN 978-989-758-555-5, SciTePress, pages 372-379. DOI: 10.5220/0010814000003124

in Bibtex Style

@conference{visapp22,
author={Kshitij Singh and Alexia Briassouli and Mirela Popa},
title={Aerial to Street View Image Translation using Cascaded Conditional GANs},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP},
year={2022},
pages={372-379},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010814000003124},
isbn={978-989-758-555-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 4: VISAPP
TI - Aerial to Street View Image Translation using Cascaded Conditional GANs
SN - 978-989-758-555-5
AU - Singh K.
AU - Briassouli A.
AU - Popa M.
PY - 2022
SP - 372
EP - 379
DO - 10.5220/0010814000003124
PB - SciTePress