loading
Papers Papers/2022 Papers Papers/2022

Research.Publish.Connect.

Paper

Authors: William Harvey 1 ; Andreas Munk 1 ; Atılım Güneş Baydin 2 ; Alexander Bergholm 1 and Frank Wood 3

Affiliations: 1 Department of Computer Science, University of British Columbia, Vancouver, BC, Canada ; 2 Department of Engineering Science, University of Oxford, U.K. ; 3 Mila - Quebec Anguilla Institute and Inverted AI, Canada

Keyword(s): Attention, Bayesian Inference, Probabilistic Programming, Inference Compilation.

Abstract: We present a neural network architecture for automatic amortized inference in universal probabilistic programs which improves on the performance of current architectures. Our approach extends inference compilation (IC), a technique which uses deep neural networks to approximate a posterior distribution over latent variables in a probabilistic program. A challenge with existing IC network architectures is that they can fail to capture long-range dependencies between latent variables. To address this, we introduce an attention mechanism that attends to the most salient variables previously sampled in the execution of a probabilistic program. We demonstrate that the addition of attention allows the proposal distributions to better match the true posterior, enhancing inference about latent variables in simulators.

CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 3.145.115.139

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Harvey, W.; Munk, A.; Baydin, A.; Bergholm, A. and Wood, F. (2022). Attention for Inference Compilation. In Proceedings of the 12th International Conference on Simulation and Modeling Methodologies, Technologies and Applications - SIMULTECH; ISBN 978-989-758-578-4; ISSN 2184-2841, SciTePress, pages 80-91. DOI: 10.5220/0011277700003274

@conference{simultech22,
author={William Harvey. and Andreas Munk. and Atılım Güneş Baydin. and Alexander Bergholm. and Frank Wood.},
title={Attention for Inference Compilation},
booktitle={Proceedings of the 12th International Conference on Simulation and Modeling Methodologies, Technologies and Applications - SIMULTECH},
year={2022},
pages={80-91},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011277700003274},
isbn={978-989-758-578-4},
issn={2184-2841},
}

TY - CONF

JO - Proceedings of the 12th International Conference on Simulation and Modeling Methodologies, Technologies and Applications - SIMULTECH
TI - Attention for Inference Compilation
SN - 978-989-758-578-4
IS - 2184-2841
AU - Harvey, W.
AU - Munk, A.
AU - Baydin, A.
AU - Bergholm, A.
AU - Wood, F.
PY - 2022
SP - 80
EP - 91
DO - 10.5220/0011277700003274
PB - SciTePress

- Science and Technology Publications, Lda.
RESOURCES

Proceedings

Papers

Authors

Ontology

CONTACTS

Science and Technology Publications, Lda
Avenida de S. Francisco Xavier, Lote 7 Cv. C,
2900-616 Setúbal, Portugal.

Phone: +351 265 520 185 (National fixed network call)
Fax: +351 265 520 186
Email: info@scitepress.org

EXTERNAL LINKS

PRIMORIS

INSTICC

SCITEVENTS

CROSSREF

PROCEEDINGS SUBMITTED FOR INDEXATION BY:

dblp

Ei Compendex

SCOPUS

Semantic Scholar

Google Scholar

Microsoft Academic