SSS: Similarity-Based Scheduled Sampling for Video Prediction

Ryosuke Hata, Yoshihisa Shinozawa

2025

Abstract

In video prediction tasks, numerous RNN-based models have demonstrated significant advancements. A well-established approach to enhancing these models during training is scheduled sampling. However, the adjustment of the probability parameter ε (scheduling) has not been adequately addressed, and current configurations are suboptimal for video prediction tasks. This issue arises because prior scheduling strategies depend solely on two factors: a function type and the total number of iterations, without considering the changes by motions, one of the most crucial features in videos. To address this gap, we propose similarity-based scheduled sampling, which accounts for the changes by motions. Specifically, we create difference frames between a given frame at a specific time step and another frame at a different time step for both the model’s predicted output and the ground truth. We then assess the similarity of these difference frames at each iteration, to determine whether the changes by motions are properly trained and to incorporate it into the scheduling. Evaluation experiment shows that proposed method outperforms previous methods. Furthermore, an ablation study confirms the effectiveness of leveraging difference frames and demonstrates the significance of considering the changes by motions.

Paper Citation

in Harvard Style

Hata R. and Shinozawa Y. (2025). SSS: Similarity-Based Scheduled Sampling for Video Prediction. In Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM; ISBN 978-989-758-730-6, SciTePress, pages 122-129. DOI: 10.5220/0013297600003905

in Bibtex Style

@conference{icpram25,
author={Ryosuke Hata and Yoshihisa Shinozawa},
title={SSS: Similarity-Based Scheduled Sampling for Video Prediction},
booktitle={Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM},
year={2025},
pages={122-129},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013297600003905},
isbn={978-989-758-730-6},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 14th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM
TI - SSS: Similarity-Based Scheduled Sampling for Video Prediction
SN - 978-989-758-730-6
AU - Hata R.
AU - Shinozawa Y.
PY - 2025
SP - 122
EP - 129
DO - 10.5220/0013297600003905
PB - SciTePress