Reposted by Dominik KlementBUT Speech @butspeech.bsky.social · 02/04/2025Our papers to be presented at ICASSP in Hyderabad! Target Speaker ASR with Whisper, ieeexplore.ieee.org/document/108... Introduces a novel approach to training target-speaker ASR systems utilizing frame-level diarization outputs. Apr 11: 2:00 pm - 3:30 pm, Poster 2E, presented by Alexander Polok 101
Reposted by Dominik KlementBUT Speech @butspeech.bsky.social · 24/03/2025🗣️ Are you participating in the Interspeech 2025 Workshop on Multilingual Conversational Speech Language Models organised by Nexdata【旧Datatang株式会社公式】? We’ve released our baseline model for the community—ready for you to explore and build upon! 🔗 Try it here: pccnect.fit.vutbr.cz/gradio-demo/ [1/4] 101
Reposted by Dominik KlementBUT Speech @butspeech.bsky.social · 02/02/2025Speechers don’t do just math, code, experiments, papers and research proposals - they also skate, or at least try to skate! 1 hour on rented skate-rink was enough to test the endurance of pros as well as beginners. Of course, followed by “one” in Microbrewery Lisen ⛸️🍺 031
Dominik Klement @dklement.bsky.social · 11/01/2025🤝 Collaboration and Feedback Welcome We’re open to feedback, discussions, and collaborations. Let’s work together to shape the future of ASR and diarization technology! [14/14] 000
Dominik Klement @dklement.bsky.social · 11/01/2025🌟 Kudos to CHiME-8 NOTSOFAR-1 Organizers Thanks to Alon Vinnikov, Amir Ivry, Eyal Krupka (Microsoft) for organizing the CHiME-8 NOTSOFAR-1 Challenge, and to the CHiME-8 Steering Committee for their dedication to advancing speech recognition research! [13/14] 100
Dominik Klement @dklement.bsky.social · 11/01/2025💻Gradio-powered Demo pccnect.fit.vutbr.cz/gradio-demo - Test our DiCoW model to transcribe your own meetings! The demo is live for 72 hours only, so don’t miss this chance. [12/14]pccnect.fit.vutbr.czGradio 100
Dominik Klement @dklement.bsky.social · 11/01/2025🔗DiCoW Inference Demo Pipeline github.com/BUTSpeechFIT... [11/14]github.comGitHub - BUTSpeechFIT/DiCoWContribute to BUTSpeechFIT/DiCoW development by creating an account on GitHub. 100
Dominik Klement @dklement.bsky.social · 11/01/2025🔗Target-Speaker Whisper Source Code github.com/BUTSpeechFIT... [10/14]github.comGitHub - BUTSpeechFIT/TS-ASR-WhisperContribute to BUTSpeechFIT/TS-ASR-Whisper development by creating an account on GitHub. 100
Dominik Klement @dklement.bsky.social · 11/01/2025🌟Open-Source Tools and Demos We’re making our research accessible by open-sourcing training and inference codebases, and providing interactive demos: 🔗DiariZen Source Code github.com/BUTSpeechFIT... [9/14]github.comGitHub - BUTSpeechFIT/DiariZen: A toolkit for speaker diarization.A toolkit for speaker diarization. . Contribute to BUTSpeechFIT/DiariZen development by creating an account on GitHub. 110
Dominik Klement @dklement.bsky.social · 11/01/2025🌟 4. Leveraging Self-Supervised Learning for Speaker Diarization - Accepted to ICASSP 2025. This paper introduces DiariZen - our state-of-the-art diarization model and toolkit. arxiv.org/abs/2409.09408 [8/14]arxiv.orgLeveraging Self-Supervised Learning for Speaker DiarizationEnd-to-end neural diarization has evolved considerably over the past few years, but data scarcity is still a major obstacle for further improvements. Self-supervised learning methods such as WavLM hav... 120
Dominik Klement @dklement.bsky.social · 11/01/2025🌟 3. BUT/JHU System Description for CHiME-8 NOTSOFAR-1 Challenge - The work earned the 🏆Jury Prize for being one of the most practical, efficient, and novel systems. Our robust diarization-ASR integration is capable of tackling overlapped speech. isca-archive.org/chime_2024/p... [7/14]isca-archive.orgISCA Archive - BUT/JHU System Description for CHiME-8 NOTSOFAR-1 Challenge 100
Dominik Klement @dklement.bsky.social · 11/01/2025🌟 2. Target Speaker ASR with Whisper arxiv.org/abs/2409.09543 - Accepted to ICASSP 2025. This work enhances the Whisper ASR model for target-speaker recognition, demonstrating its applicability in complex acoustic scenarios. [6/14]arxiv.orgTarget Speaker ASR with WhisperWe propose a novel approach to enable the use of large, single speaker ASR models, such as Whisper, for target speaker ASR. The key insight of this method is that it is much easier to model relative d... 100
Dominik Klement @dklement.bsky.social · 11/01/2025By directly conditioning the ASR model on diarization outputs, we simplify the workflow for multi-speaker and target-speaker scenarios. Importantly, DiCoW maintains Whisper’s performance on single-speaker transcription, ensuring robustness across diverse use cases. [5/14] 100
Dominik Klement @dklement.bsky.social · 11/01/2025🌟Recent Papers 1. DiCoW: Diarization-Conditioned Whisper for Target Speaker Automatic Speech Recognition - Submitted to CSL. Our diarization-conditioned approach that eliminates the need for speaker enrollment or source separation. arxiv.org/abs/2501.00114 [4/14]arxiv.orgDiCoW: Diarization-Conditioned Whisper for Target Speaker Automatic Speech RecognitionSpeaker-attributed automatic speech recognition (ASR) in multi-speaker environments remains a significant challenge, particularly when systems conditioned on speaker embeddings fail to generalize to u... 100
Dominik Klement @dklement.bsky.social · 11/01/2025- Versatile and Robust: Despite all improvements, our systems retain high performance on single-speaker transcription tasks, ensuring broad applicability across use cases. [3/14] 100
Dominik Klement @dklement.bsky.social · 11/01/2025🌟Key Innovations - Simplifying Multi-Speaker ASR: Our models directly use diarization outputs as conditioning signals, bypassing the need for enrollment data or complex source separation techniques. [2/14] 100
Dominik Klement @dklement.bsky.social · 11/01/2025Transcribing multiple speakers with OpenAI’s Whisper? No problem. Check out our recent work at BUT Speech@FIT in collaboration with CLSP JHU. It is fully open-sourced. Do not forget to try out our demo: pccnect.fit.vutbr.cz/gradio-demo Read more in this thread 👇 [1/14] 2133