iiinova — where your digital self keeps evolving

Jesse Engel @jesseengel.bsky.social · 10/03/2026

We’ll be running a few workshops for people interested in learning to build on the Infinite Crate plugin and the Magenta RealTime model. Join the discussion on our Discord if you’re interested: g.co/magenta/discord Looking forward to seeing what new plugins and interfaces you build 🙂

g.co

Join the Magenta Discord Server!

The official discord community for Magenta - an open source research project within Google DeepMind exploring the role of machine learning as a tool in the creative process. We welcome artists, musici...

030

Jesse Engel @jesseengel.bsky.social · 10/03/2026

In the near term, we hope to update the plugin to support on-device inference of the Magenta RealTime open-weights model for offline use. In the long term, we hope to support future music models with improved controls, such as audio and MIDI input.

210

Jesse Engel @jesseengel.bsky.social · 10/03/2026

The based frontend also makes it easier for developers to open the codebase in Antigravity (or similar LLM coding tool) and vibecode new interfaces and functionality.

110

Jesse Engel @jesseengel.bsky.social · 10/03/2026

This approach ensures latency sensitive operations (audio streaming and processing) happen in a compiled and unmanaged language with a tight clock, while leveraging web technologies for rapid prototyping of the UI (@reactjs, @shadcn, and @tailwindcss).

120

Jesse Engel @jesseengel.bsky.social · 10/03/2026

We architected the plugin using JUCE/C++ and React/Typescript - a clever idea presented by @spencersalazar at ADC 2020.

110

Jesse Engel @jesseengel.bsky.social · 10/03/2026

We’re now fully open sourcing the plugin for developers to fork, modify, and make their own under the permissive Apache 2.0 license.

110

Jesse Engel @jesseengel.bsky.social · 10/03/2026

Since we released The Infinite Crate last year, it’s been used by some of our favorite artists - including a wonderful showcase with @daitomanabe in Tokyo - and being featured as a top new music tool at NAMM 2026.

110

Jesse Engel @jesseengel.bsky.social · 10/03/2026

Today, we're open sourcing the code behind "The Infinite Crate," our VST/AU plugin that lets you play with Lyria RealTime directly inside your favorite DAW! 🎧🎶 💾 Get it here: github.com/magenta/the-...

github.com

GitHub - magenta/the-infinite-crate: The Infinite Crate is an experimental DAW plugin built on the new Lyria RealTime live music model

The Infinite Crate is an experimental DAW plugin built on the new Lyria RealTime live music model - magenta/the-infinite-crate

110

Jesse Engel @jesseengel.bsky.social · 10/02/2026

I think a good shorthand for “slop” is that you spend less time creating something than it takes for someone to consume it. What I love about live generation and performance is that it naturally balances the two through the helpful bottleneck on shared human experience.

130

Jesse Engel @jesseengel.bsky.social · 06/11/2025

One of the most fun things about latent spaces, is that they are actual spaces. Take a ride in you own spaceship to explore the constellations of latent music! magenta.withgoogle.com/spacedj-anno... Powered by the Lyria RealTime API in AI Studio.

magenta.withgoogle.com

Space DJ: Navigating a Musical Universe

Today, we’re excited to launch Space DJ, a web application from Magenta thatturns music exploration into an interactive journey through a constellation ofsou...

020

Jesse Engel @jesseengel.bsky.social · 08/08/2025

Thanks again to @hollyherndon.bsky.social, it's always fun collaborating with you 🙏

010

Jesse Engel @jesseengel.bsky.social · 08/08/2025

These latest features take us one step closer to live interactive generative music models that are open, personalizable, and run locally on-device, so you can really trust them to hack them like any other instrument. Next step, live on-device inference, stay tuned… 😀

120

Jesse Engel @jesseengel.bsky.social · 08/08/2025

As we’ve seen with Genie, real-time interactive generative models opens up new worlds, and we’re excited to keep pushing the boundaries of what’s possible.

100

Jesse Engel @jesseengel.bsky.social · 08/08/2025

This new wave of features is the culmination of effort from many folks at Google DeepMind, on the Lyria team and beyond. We are grateful to work with so many amazing collaborators to build the next generation of human-centered music AI tools. 🙏

100

Jesse Engel @jesseengel.bsky.social · 08/08/2025

We hope Magenta RT finetuning can serve that purpose, and are happy to have their model as a first example of personalizing live generative music.

100

Jesse Engel @jesseengel.bsky.social · 08/08/2025

For years, Holly and Mat Dryhurst have pioneered artist-centric ways in which AI can be used to push the boundaries of expression, and how artists and communities can have more agency over developing models for their own purposes (as they open sourced her voice to holly.plus for experimentation).

holly.plus

Holly+

Holly Herndon's digital twin

110

Jesse Engel @jesseengel.bsky.social · 08/08/2025

We're thrilled to have worked with Holly Herndon to train a model finetuned on her vocal style to share with all of you. Playing with Holly’s model, you can use additional controls to get vocalizations that contour the music or "color" the instruments with a timbre that resembles Holly's voice

110

Jesse Engel @jesseengel.bsky.social · 08/08/2025

We’re also excited to add support for artists to customize Magenta RT on their own training data! g.co/magenta/rt-finetuning-colab

110

Jesse Engel @jesseengel.bsky.social · 08/08/2025

Audio injection also supports *live* music input. You can jam along with it and use it as an intelligent live looper! Try it out for yourself on free Colab TPUs! g.co/magenta/rt-audio-injection-colab

110

Jesse Engel @jesseengel.bsky.social · 08/08/2025

Magenta RT now supports what we call *audio injection* - a novel method for real-time audio input You can use audio injection to transform and add layers to prerecorded music!

100

Jesse Engel @jesseengel.bsky.social · 08/08/2025

We’ve since been hard at work preparing new features. Check out our tech report on *Live Music Models*, including both Lyria RT (API) and Magenta RT (open) 📜 Paper: arxiv.org/abs/2508.046... 🔊Samples: g.co/magenta/live...

arxiv.org

Live Music Models

We introduce a new class of generative models for music called live music models that produce a continuous stream of music in real-time with synchronized user control. We release Magenta RealTime, an ...

110

Jesse Engel @jesseengel.bsky.social · 08/08/2025

ICYMI, in June we shared a preview of Magenta RealTime - the first open weights model capable of *live* music generation 🤗 Model: huggingface.co/google/magen... ⭐ Code: github.com/magenta/mage...

huggingface.co

google/magenta-realtime · Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

110

Jesse Engel @jesseengel.bsky.social · 08/08/2025

Realtime interactive generative models FTW! Announcing a new 🌊 of details and features for Magenta RealTime, the open weights live music AI model from GDM! * Live Jamming with audio input 🎤🎸🎵 * Personalize your own models 🔧 * Tech report 📜 Links below in the 🧵...

172

Jesse Engel @jesseengel.bsky.social · 09/07/2025

Looking forward to seeing all sorts of interesting ways you mangle and twist the outputs into unique creations 🙂.

010

Jesse Engel @jesseengel.bsky.social · 09/07/2025

We are excited to make these models more accessible, and more tightly integrated into musician’s workflows as it will open up more opportunities to customize and personalize how you use them to come up with truly personal expression.

110

Jesse Engel @jesseengel.bsky.social · 09/07/2025

Built on the hard work of the Lyria Team, the VST plugin has been a labor of love of Ashutosh Desai. Thanks so much also to the support of artists Sidecar Tommy from Beats Antique, Toro y Moi in exploring the possibilities of DAW integration of our models.

110

Jesse Engel @jesseengel.bsky.social · 09/07/2025

The Infinite Crate is cross-platform, and currently supports Mac VST/AU, Windows VST, and a Mac Standalone application (no VST host required). It is built on the Lyria RealTime API and thus requires an internet connection and API key (usage is free).

110

Jesse Engel @jesseengel.bsky.social · 09/07/2025

Be a Prompt DJ 🎧🎶: Play it like a live instrument. Use prompts and advanced controls to vary the instrumentation, texture, and genre. Incorporate with a DJ set to apply effects and mix with other loops.

110

Jesse Engel @jesseengel.bsky.social · 09/07/2025

Jam Along 🎸🥁: Create dynamic backing tracks that are different every time, varying and evolving as you play on top.

110

Jesse Engel @jesseengel.bsky.social · 09/07/2025

What can you do with it? Dig in the crate 🎛️💡 : Explore new sounds by blending and merging genres, moods, instruments, or anything you can think of. Sample the stream of music to spark your own inspiration or help you get past a creative block.

120

Jesse Engel @jesseengel.bsky.social · 09/07/2025

New VST/AU Plugin! 🚨 Play with Lyria RealTime directly from inside your favorite DAW with “The Infinite Crate” 🎧🎶 With a VST plugin you can feed audio directly into your DAW for sampling, live performance, or even a practice partner to jam with. 💾 Get it here: g.co/magenta/infinite-crate

151

Jesse Engel @jesseengel.bsky.social · 05/06/2025

Try it out and fork it here: bit.ly/3ZOp7KS Don't forget you can type in new prompts, Curious to see what else you can make :).

000

Jesse Engel @jesseengel.bsky.social · 05/06/2025

Pleasantly surprised with how easy/fun it is to "vibe code" new musical experiences in AI Studio. I'm not much of a JS guy, but had Gemini help me make a "Kaoss Pad" emulator using the Lyria RealTime API to explore musical latent spaces in realtime.

100

Jesse Engel @jesseengel.bsky.social · 23/04/2025

How are they being impacted by the current approach to deploying/integrating AI in our societal habitat? How could we be more sustainably approaching things?

100

Jesse Engel @jesseengel.bsky.social · 23/04/2025

There's a lot of inspiration to be found in nature, seeing the interconnectedness of a habitat and the effects of disruption on one part of it. What are the core pillars of our societal habitat that we need to make sure survive going forward? How do support the broader system?

100

Jesse Engel @jesseengel.bsky.social · 23/04/2025

It brings up interesting questions. What are the things most essential that we want to preserve? How can we adapt them to help survive AI? How can we co-adapt AI to sustainably become a better part of the habitat? What inspiration and cautionary lessons can we learn from history?

110

Jesse Engel @jesseengel.bsky.social · 23/04/2025

Ecology isn't just about preserving by removing humans, it's about understanding how an environment responds to different impacts and thinking of interventions to help it adapt (+tech co-adapt) such as to not lose the things we find precious and essential to future thriving.

110

Jesse Engel @jesseengel.bsky.social · 23/04/2025

Lately I've been finding it interesting to think of AI from an ecology/conservationist perspective. In the analogy, we are the native habitat (our society, culture, lives) and AI is the outside disruptive influence (i.e. invasive species, pollution, human effects, etc.)

100

Reposted by Jesse Engel

Chris Donahue @chrisdonahue.com · 05/03/2025

Inaugurating new acct to share work from my PhD student! Wayne et al have been running a live eval platform Copilot Arena - a VSCode extension serving code completions from AI systems to real developers. See 🧵 for findings and preprint Excited to be evaluating human-AI *workflows* holistically!

0103

Jesse Engel @jesseengel.bsky.social · 13/01/2025

Those are the technologies that we have always strived to build. Reduce the barriers to entry, but simultaneously raise the ceiling for new expression

000

Jesse Engel @jesseengel.bsky.social · 13/01/2025

Great tools help people start that journey (accessible), but also provide challenges that beg you to dig deeper, and reward you for your struggle with unique and personal expression (mastery).

110

Jesse Engel @jesseengel.bsky.social · 13/01/2025

What an astonishingly tone deaf take. He misses the entire point of creative practice. Accessibility and mastery are not opposed, they're two ends of the same personal creative journey.

161

Jesse Engel @jesseengel.bsky.social · 11/01/2025

I'm not saying that automation doesn't have a place to improve things. I just think we'd all be a lot happier if it had its separate lane (like it currently does). That would allow it to not be hindered by conforming to our manual interfaces, and to not devalue our own manual agency.

000

Jesse Engel @jesseengel.bsky.social · 11/01/2025

But very quickly it will suck for everyone once the default changes to assume that 99.99% traffic on 99.99% of sites is bots, magnifying the arms race for attention, and reducing the value of any manual work that has to try to cut through the noise.

100

Jesse Engel @jesseengel.bsky.social · 11/01/2025

In the short term it will lead to gains for those who automate, as the default will be to assume that the output is actually done by people, that you're multiplying the power of an individual.

100

Jesse Engel @jesseengel.bsky.social · 11/01/2025

While that's empowering in the short term, it can also quickly lead to enshitification, because we're automating things that aren't actually designed to be automated. If they were, they'red be an API. It basically forces a redesign of the internet, or leaves it to the bots.

110

Jesse Engel @jesseengel.bsky.social · 11/01/2025

The Internet was designed so that browsers are supposed to be used manually by people and public APIs are for automation. Giving bots (...sorry, "agents") browser control breaks this design, turns everything into an API.

150

Jesse Engel @jesseengel.bsky.social · 04/12/2024

Jam of the day #14: mbira DnB 16th note tornado Sometimes less is more with prompts Remix here: aitestkitchen.withgoogle.com/tools/music-... Discord: discord.gg/googlelabs #MusicFXDJ

010

Jesse Engel @jesseengel.bsky.social · 03/12/2024

Jam of the day #13: Chiptune drumline madness Happy Monday! Remix here: aitestkitchen.withgoogle.com/tools/music-... Discord: discord.gg/googlelabs #MusicFXDJ

000