Background Image Editing with HyperStyle and Semantic Segmentation

Syuusuke Ishihata, Ryohei Orihara, Yuichi Sei, Yasuyuki Tahara, Akihiko Ohsuga

2023

Abstract

Recently, research has been conducted on applying StyleGAN to image editing tasks. Although the technique can be applied to editing background images, because they are more diverse than foreground images such as face images, specifying an object in background images to be edited is difficult. For example, because natural language instructions can be ambiguous, edited images become undesirable for the user. It is challenging to resolve style and content dependencies in image editing. In our study, we propose an editing method that adapts Style Transformer, the latest GAN inversion encoder approach, to HyperStyle by introducing semantic segmentation to maintain the reconstruction quality and separate the style and the content of the background image. The content is edited while keeping the original style by manipulating the coarse part of latent variables and the residual parameters obtained by HyperStyle, and the style is edited without changing the content by manipulating the medium and fine part of latent vectors as in the conventional StyleGAN. As a result, the qualitative evaluation confirms that our model enabled the editing of image content and style separately, and the quantitative evaluation validates that the reconstruction quality is comparable to the conventional method.

Paper Citation

in Harvard Style

Ishihata S., Orihara R., Sei Y., Tahara Y. and Ohsuga A. (2023). Background Image Editing with HyperStyle and Semantic Segmentation. In Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART, ISBN 978-989-758-623-1, pages 293-300. DOI: 10.5220/0011661500003393

in Bibtex Style

@conference{icaart23,
author={Syuusuke Ishihata and Ryohei Orihara and Yuichi Sei and Yasuyuki Tahara and Akihiko Ohsuga},
title={Background Image Editing with HyperStyle and Semantic Segmentation},
booktitle={Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,},
year={2023},
pages={293-300},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011661500003393},
isbn={978-989-758-623-1},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 15th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART,
TI - Background Image Editing with HyperStyle and Semantic Segmentation
SN - 978-989-758-623-1
AU - Ishihata S.
AU - Orihara R.
AU - Sei Y.
AU - Tahara Y.
AU - Ohsuga A.
PY - 2023
SP - 293
EP - 300
DO - 10.5220/0011661500003393