DiT-Head: High Resolution Talking Head Synthesis Using Diffusion Transformers

Aaron Mir; Eduardo Alonso; Esther Mondragón

Research.Publish.Connect.

*Please fill out at least one Field. *Value must be an number!

Title:
ISBN:
Year:
Acronym:
Subject:

Advanced Search Proceedings Search

If you're looking for an exact phrase use quotation marks on text fields.

*Please fill out at least one Field.

Title:
Author:
Affiliation:
Subject:

Advanced Search Papers Search

If you're looking for an exact phrase use quotation marks on text fields.

*Please fill out at least one Field.

Name:
Affiliation:
Country:
Conference:
Subject:

Advanced Search Authors Search

If you're looking for an exact phrase use quotation marks on text fields.

*Please fill out at least one Field.

Name:
Country:
Subject:

Advanced Search Affiliations Search

If you're looking for an exact phrase use quotation marks on text fields.

Proceedings

Proceedings Search *Please fill out at least one Field. *Value must be an number!

Title:
ISBN:
Year:
Acronym:
Subject:

Advanced Search Proceedings Search

If you're looking for an exact phrase use quotation marks on text fields.

Papers

Papers Search *Please fill out at least one Field.

Title:
Author:
Affiliation:
Subject:

Advanced Search Papers Search

If you're looking for an exact phrase use quotation marks on text fields.

Authors

Authors Search *Please fill out at least one Field.

Name:
Affiliation:
Country:
Conference:
Subject:

Advanced Search Authors Search

If you're looking for an exact phrase use quotation marks on text fields.

Advanced Search

Paper

DiT-Head: High Resolution Talking Head Synthesis Using Diffusion Transformers

Topics: AI and Creativity; Deep Learning

In Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART, 159-169, 2024 , Rome, Italy

Authors: Aaron Mir ; Eduardo Alonso and Esther Mondragón

Affiliation: Artificial Intelligence Research Centre, Department of Computer Science, City, University of London, Northampton Square, EC1V 0HB, London, U.K.

Keyword(s): Talking Head Synthesis, Diffusion Transformers.

Abstract: We propose a novel talking head synthesis pipeline called ”DiT-Head,” which is based on diffusion transformers and uses audio as a condition to drive the denoising process of a diffusion model. Our method is scalable and can generalise to multiple identities while producing high-quality results. We train and evaluate our proposed approach and compare against existing methods of talking head synthesis. We show that our model can compete with these methods in terms of visual quality and lip-sync accuracy. Our results highlight the potential of our proposed approach to be used for a wide range of applications including virtual assistants, entertainment, and education. For a video demonstration of results and our user study, please refer to our supplementary material.

CC BY-NC-ND 4.0

Guest: Register as new SciTePress user now for free.

SciTePress user: please login.

My Papers

You are not signed in, therefore limits apply to your IP address 3.128.171.192

In the current month:

Recent papers: 100 available of 100 total

2⁺ years older papers: 200 available of 200 total

Paper citation in several formats:

Mir, A.; Alonso, E. and Mondragón, E. (2024). DiT-Head: High Resolution Talking Head Synthesis Using Diffusion Transformers. In Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART; ISBN 978-989-758-680-4; ISSN 2184-433X, SciTePress, pages 159-169. DOI: 10.5220/0012312200003636

@conference{icaart24,
author={Aaron Mir. and Eduardo Alonso. and Esther Mondragón.},
title={DiT-Head: High Resolution Talking Head Synthesis Using Diffusion Transformers},
booktitle={Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART},
year={2024},
pages={159-169},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012312200003636},
isbn={978-989-758-680-4},
issn={2184-433X},
}

TY - CONF

JO - Proceedings of the 16th International Conference on Agents and Artificial Intelligence - Volume 3: ICAART
TI - DiT-Head: High Resolution Talking Head Synthesis Using Diffusion Transformers
SN - 978-989-758-680-4
IS - 2184-433X
AU - Mir, A.
AU - Alonso, E.
AU - Mondragón, E.
PY - 2024
SP - 159
EP - 169
DO - 10.5220/0012312200003636
PB - SciTePress