iiinova — where your digital self keeps evolving

Justin Salamon @justinsalamon.bsky.social · 24/06/2025

FLAM is trained jointly on instance (global) and frame-wise (local) objectives. The secret sauce: A memory-efficient and calibrated frame-wise objective with logit adjustment to address spurious correlations, such as event dependencies and label imbalances during training

110 0

Justin Salamon @justinsalamon.bsky.social · 24/06/2025

Enter FLAM: Frame-Wise Language-Audio Modeling. A model trained to produce a calibrated likelihood for *any* text prompt. FLAM outperforms prior self-supervised models on both closed-set and open-set SED, while preserving strong retrieval and zero-shot classification accuracy

100 0

Justin Salamon @justinsalamon.bsky.social · 24/06/2025

Our goal is for the model to detect *any* sound via free form text queries. "So use CLAP", some of you will say. The problem is its output likelihoods are not calibrated for different prompts :( That's ok ranked retrieval, but for detection it's a no go.

100 0

Justin Salamon @justinsalamon.bsky.social · 24/06/2025

Sound Event Detection models, ie finding sounds in audio/video recordings, are typically constrained to a predefined "closed" set of sounds, like in this (old!) model below for urban sound detection. It has some applications, but it doesn't address general purpose sound search.

100 0

Justin Salamon @justinsalamon.bsky.social · 24/06/2025

I think we finally cracked it? FLAM can detect *any* sound via text prompts arXiv (ICML'25): arxiv.org/abs/2505.053... demos: flam-model.github.io Led by Yusong Wu, with @tsirif.bsky.social Ke Chen, Cheng-Zhi Anna Huang, Aaron Courville, @urinieto.bsky.social @pseeth.bsky.social

1101 0

Justin Salamon @justinsalamon.bsky.social · 11/04/2025

Generative Extend in Premiere Pro just won *five* awards at NAB 2025, including the NAB Show Product of the Year award! SODA, our group, created the audio GenAI model in charge of audio extensions in the feature. Couldn't be more proud of the team! w/ @urinieto.bsky.social @pseeth.bsky.social

040 0

Justin Salamon @justinsalamon.bsky.social · 02/04/2025

Generative Extend just released in Premiere Pro! Use GenAI to extend your video *and audio* clips for a perfectly timed edit. The audio model was built by our team, the Sound Design AI (SODA) group at Adobe Research w/ @pseeth.bsky.social and @urinieto.bsky.social 🙌 www.youtube.com/watch?v=_Bv5...

youtube.com

Generative Extend | Premiere Pro 2025 Updates | Adobe Video

YouTube video by Adobe Video & Motion

030 0

Justin Salamon @justinsalamon.bsky.social · 22/02/2025

We didn't expect this... our Sketch2Sound demo video has gone viral on IG with more than 5.2 million views 🤯 Amazing job @hugofloresgarcia.bsky.social @pseeth.bsky.social @urinieto.bsky.social I should've done my hair... www.instagram.com/reel/DEEBRhd...

0131 0

Justin Salamon @justinsalamon.bsky.social · 19/12/2024

Really cool to see our AI for bioacoustics work in the MIT Technology Review! BirdVox was a fantastic project that beyond posing a fascinating research problem, introduced me to the world of bird watching and deepened my appreciation of the natural world www.technologyreview.com/2024/12/18/1...

technologyreview.com

AI is changing how we study bird migration

After decades of frustration, machine-learning tools are unlocking a treasure trove of acoustic data for ecologists.

0102 0

Reposted by Justin Salamon

Jonathan Le Roux @jonathanleroux.bsky.social · 14/12/2024

@waspaa.com is on Bluesky! Big changes are coming for WASPAA 2025!

0116 0

Justin Salamon @justinsalamon.bsky.social · 12/12/2024

Sketch2Sound is out! Takes a text prompt + vocal (or sonic) imitation and generates sound effects that perfectly match the energy and dynamics of your voice. It's an extremely intuitive (and fun!) way to create SFX that are perfectly timed to your video. Led by @hugofloresgarcia.bsky.social 👏

071 0

Justin Salamon @justinsalamon.bsky.social · 10/12/2024

I'm hiring a Research Engineer for my team at Adobe Research. Full details in the link in the post below, DM me if interested.

0132 0

Justin Salamon @justinsalamon.bsky.social · 09/12/2024

Here's another example of work from our group: MultiFoley, a Video-to-Audio model that generates perfectly synced audio for video at 48 kHz and supports multimodal conditioning. More on MultiFoley here: bsky.app/profile/czya...

020 0

Justin Salamon @justinsalamon.bsky.social · 09/12/2024

📢 Audio AI Job opportunity at Adobe! The Sound Design AI Group (SODA) is looking for an exceptional research engineer to join us in building the future of AI-assisted audio and video creation. Strong ML background, GenAI experience a plus. Details: adobe.wd5.myworkdayjobs.com/external_exp...

1113 0

Justin Salamon @justinsalamon.bsky.social · 27/11/2024

New model from the SODA group at Adobe and UMich! MultiFoley generates perfectly synced audio for video at full 48 kHz and supports multimodal conditioning. You can define the generated sound via a text prompt, an example SFX, or audio you want to extend. Led by our intern @czyang.bsky.social 👇

0121 0