DenseHMM: Learning Hidden Markov Models by Learning Dense Representations

Joachim Sicking, Joachim Sicking, Maximilian Pintz, Maximilian Pintz, Maram Akila, Tim Wirtz, Tim Wirtz

2022

Abstract

We propose DenseHMM – a modification of Hidden Markov Models (HMMs) that allows to learn dense representations of both the hidden states and the (discrete) observables. Compared to the standard HMM, transition probabilities are not atomic but composed of these representations via kernelization. Our approach enables constraint-free and gradient-based optimization. We propose two optimization schemes that make use of this: a modification of the Baum-Welch algorithm and a direct co-occurrence optimization. The latter one is highly scalable and comes empirically without loss of performance compared to standard HMMs. We show that the non-linearity of the kernelization is crucial for the expressiveness of the representations. The properties of the DenseHMM like learned co-occurrences and log-likelihoods are studied empirically on synthetic and biomedical datasets.

Paper Citation

in Harvard Style

Sicking J., Pintz M., Akila M. and Wirtz T. (2022). DenseHMM: Learning Hidden Markov Models by Learning Dense Representations. In Proceedings of the 11th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-549-4, pages 237-248. DOI: 10.5220/0010821800003122

in Bibtex Style

@conference{icpram22,
author={Joachim Sicking and Maximilian Pintz and Maram Akila and Tim Wirtz},
title={DenseHMM: Learning Hidden Markov Models by Learning Dense Representations},
booktitle={Proceedings of the 11th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2022},
pages={237-248},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010821800003122},
isbn={978-989-758-549-4},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 11th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - DenseHMM: Learning Hidden Markov Models by Learning Dense Representations
SN - 978-989-758-549-4
AU - Sicking J.
AU - Pintz M.
AU - Akila M.
AU - Wirtz T.
PY - 2022
SP - 237
EP - 248
DO - 10.5220/0010821800003122