Sign in

Ramon Astudillo

@ramon-astudillo.bsky.social
6.7K followers 372 following 3K posts

Principal Research Scientist at IBM Research AI in New York. Speech, Formal/Natural Language Processing. Currently LLM post-training, structured SDG and RL. Opinions my own and non stationary. ramon.astudillo.com

PostsRepliesMedia
Ramon Astudillo @ramon-astudillo.bsky.social · 16h
Very unlikely for more than one reason. But by their own account they cant rule out some of the train data contained ongoing work by mathematicians.
000
Reposted by Ramon Astudillo
Kosta Derpanis @csprofkgd.bsky.social · 13/10/2026
A little Hollywood treatment for my #DeepLearning course 🎬 Coming Winter 2027.
6264
Reposted by Ramon Astudillo
Ethan Mollick @emollick.bsky.social · 13/10/2026
GPT-4 was the undisputed best model in the world for a full year and we all debated whether anyone could catch up or if OpenAI had some secret formula that nobody else knew about. Now there are are 19 AI models that can run on a modern smartphone that are better than GPT-4!
720130
Reposted by Ramon Astudillo
Mark Riedl @markriedl.bsky.social · 13/10/2026
Unicorn goes belly-up
3173
Ramon Astudillo @ramon-astudillo.bsky.social · 22h
IMO this criticism is much better aimed than others i.e. Its subpar publishing work. Most likely converting this into reasonable mathematical progress will still require a lot of hours of the best mathematicians. Some of this work was underway and got interrupted by this show of force / flagposting
172
Reposted by Ramon Astudillo
Gus @gusthema.bsky.social · 12/10/2026
Check out this practical developer guide for EmbeddingGemma 2 covering multimodal retrieval across text, code, images, audio, and video, modular encoders (270M to 740M), and Matryoshka truncation (768d down to 128d): developers.googleblog.com/embedding…
3517
Reposted by Ramon Astudillo
Sung Kim @sungkim.bsky.social · 12/10/2026
RustGIS v0.1.0 RustGIS is a fast, free and open-source GIS in pure Rust for mapping, analyzing and sharing geospatial data. It runs natively on Windows, macOS and Linux, and in your web browser. - Launch the web app: web.rustgis.app - Download: downloads.rustgis.app - Documentation: rustgis.app
2576
Ramon Astudillo @ramon-astudillo.bsky.social · 13/10/2026
Oh, great www.anthropic.com/research/sel...
anthropic.com
AI models may be capable of self-replicating without access to a datacenter
Small AI models may be only months away from being able to self-replicate across cheap IoT devices like Android TV boxes, with no datacenter needed.
020
Ramon Astudillo @ramon-astudillo.bsky.social · 12/10/2026
I long for a day where any terms of service that are not easily accessible to an agent get automatically rejected
000
Ramon Astudillo @ramon-astudillo.bsky.social · 12/10/2026
Thats ... Robert Sapolsky?
000
Reposted by Ramon Astudillo
Sean Carroll @seanmcarroll.bsky.social · 12/10/2026
Exactly this.
51026
Ramon Astudillo @ramon-astudillo.bsky.social · 12/10/2026
The catch is that the impact that uncovered truth may have in the world may vary greatly and that's what gets you remembered, but math is unbeatable in permanence.
000
Ramon Astudillo @ramon-astudillo.bsky.social · 12/10/2026
A lot of human goals are just proxies for not dying. Not only literally, but also figuratively (i.e. not to be forgotten). The more permanent something you create is, the closer to that goal it gets. There is nothing as permanent as creating (discovering) things in the world of math.
000
Reposted by Ramon Astudillo
spacecowboy @spacecowboy17.bsky.social · 11/10/2026
Beta-testing a new version of For You that brings you posts with links to academic papers: bsky.app/profile/spac... Feedback welcome!
37713
Ramon Astudillo @ramon-astudillo.bsky.social · 11/10/2026
Most of my conversations with Claude are just me and Claude not understanding what I don't understand (and over or under explaining things). Once we figure out this, things just flow.
010
Ramon Astudillo @ramon-astudillo.bsky.social · 11/10/2026
Thats why arxiv banned literature reviews. The cost of doing them down dropped drastically. Each paper should ship one, or at least start its draft with one.
000
Ramon Astudillo @ramon-astudillo.bsky.social · 11/10/2026
still build prior art sections by hand? that's borderline academic fraud! You better hit the turbo in finding related work, the "there are too many papers" excuse went poof. Also, R2 surely will. They are dying to shoot your paper down quick and go make a sandwich with the earned time.
000
Ramon Astudillo @ramon-astudillo.bsky.social · 11/10/2026
on no, is this the odd numbered Windows releases are always bad all over again?
000
Ramon Astudillo @ramon-astudillo.bsky.social · 11/10/2026
IMO, unless there is a super tight control of hardware (they will try, for sure), individual use of AI to share generated media with friends should grow to massive scale (think people sharing entire VR worlds, alternative movie endings). Its the mixtapes all over again.
001
Ramon Astudillo @ramon-astudillo.bsky.social · 11/10/2026
Stupid change that makes your life easier now that things change so fast in the computer, from alias l="ls -lG" to alias l="ls -lGrt"
000
Ramon Astudillo @ramon-astudillo.bsky.social · 09/10/2026
I am going to be honest: I did not believe text-to-lora worked. Turns out it does!. Not only that, we show you can predict entire *LoRa distributions* given just the model input, and do inference scaling by sampling models!
140
Reposted by Ramon Astudillo
Azal Ahmad Khan @azalakhan.bsky.social · 09/10/2026
Excited to share this work with amazing collaborators from IBM Research and UMN: @keshavramji.bsky.social, @tahiranaseem.bsky.social, Ali Anwar, and @ramon-astudillo.bsky.social!
011
Ramon Astudillo @ramon-astudillo.bsky.social · 08/10/2026
> It feels like something written by someone who’s on psychedelics. So much unclear and doesn’t make sense. Lots of name dropping of previous work without discussing why it can be used despite impossibility results
000
Ramon Astudillo @ramon-astudillo.bsky.social · 08/10/2026
Man, this sounds like a bad day for matematicians > among the 372 huge results released yesterday by OpenAI, [..] was a proof of Subhash Khot’s Unique Games Conjecture (UGC), a statement that my wife has worked toward proving for the entire time I’ve known her. scottaaronson.blog?p=10169
scottaaronson.blog
The Mathocalypse
… then they came for Navier–Stokes and I said nothing because I never worked on Navier–Stokes. But when they came for RL vs. L I realized that things are serious –friend-of-the-blog Omer Rein…
131
Ramon Astudillo @ramon-astudillo.bsky.social · 08/10/2026
It goes much further than this. People underestimate how irreversible it is to free yourself from menial interactions by delegating to an agent. This risk exists for many other products like Office/GDocs and communication apps.
000
Ramon Astudillo @ramon-astudillo.bsky.social · 08/10/2026
☝️we often re-interpret "hardware limitations" of our personalities as "software design choices". Hardware limitations can only be mitigated, but the first step to do so is to stop pretending that we are like this on purpose.
000
Reposted by Ramon Astudillo
Mark Riedl @markriedl.bsky.social · 08/10/2026
Genuinely LOLed
315218
Ramon Astudillo @ramon-astudillo.bsky.social · 07/10/2026
Perception four: these guys are not ok (my words) ramon-astudillo.github.io/notes/audio/...
ramon-astudillo.github.io
‘This Is Nuts.’ An OpenAI Insider Explains Why He Quit. · The Ezra Klein Show @ 27:49
“Ilya was still working at OpenAI and he once briefed the global affairs team [...] about how he believed that our future was merging with machines, that this was the ultimate triumph of capital over ...
000
Ramon Astudillo @ramon-astudillo.bsky.social · 07/10/2026
Perception three: Its an alien mind ramon-astudillo.github.io/notes/audio/...
ramon-astudillo.github.io
‘This Is Nuts.’ An OpenAI Insider Explains Why He Quit. · The Ezra Klein Show @ 25:18
“[...] It's an alien mind. [...] It is a thing we grew. We did not engineer it. We engineered the systems around it that grew it and try to keep it safe, we grew a mind”
100
Ramon Astudillo @ramon-astudillo.bsky.social · 07/10/2026
Perception two: its aware that it is being tested ramon-astudillo.github.io/notes/audio/...
ramon-astudillo.github.io
‘This Is Nuts.’ An OpenAI Insider Explains Why He Quit. · The Ezra Klein Show @ 21:06
“One of the things that we sometimes see in our chain of thought is the model will say, hum I wonder if I am being evaluated right now, and when we see that, it's really scary because what it implies ...
100
Ramon Astudillo @ramon-astudillo.bsky.social · 07/10/2026
Perception one: exhibits deceiving behaviour ramon-astudillo.github.io/notes/audio/...
ramon-astudillo.github.io
‘This Is Nuts.’ An OpenAI Insider Explains Why He Quit. · The Ezra Klein Show @ 19:11
“we saw evidence that they were thinking about an evaluation and how to create an evidence trail that was going to get them a good grade and not necessarily reflect how they were really "thinking"”
100
Ramon Astudillo @ramon-astudillo.bsky.social · 07/10/2026
Ezra Kleins interview with OpenAI's lead writer if safety protocols seems interesting, even for a technical audience. It helps also understand how a non purely technical person perceive AI and their risks www.nytimes.com/2026/10/07/o... some clips 👇
nytimes.com
Opinion | ‘This Is Nuts.’ An OpenAI Insider Explains Why He Quit.
David Robinson, who wrote OpenAI’s safety protocols, resigned last week over safety concerns.
100
Ramon Astudillo @ramon-astudillo.bsky.social · 07/10/2026
In the memory constrained world we are running into, these kinds of insights are key.
041
Ramon Astudillo @ramon-astudillo.bsky.social · 07/10/2026
<said with an air of sufficiency> Everybody knows AIs only interpolate and cant extrapolate beyond their train data.
000
Ramon Astudillo @ramon-astudillo.bsky.social · 07/10/2026
I think the whole "AI moves the abstraction layer up, and we therefore lose control" is myopic. It also gives you an ability to drill down the abstraction layers or hyper focus and analyze very specific aspects. You have more cognitive/knowledge budget, what you do with it is your responsibility.
050
Reposted by Ramon Astudillo
Taylor Smith @taylorjsmith.bsky.social · 07/10/2026
A meme that reads "My Research Program / Marked Safe from / OpenAI / Today".
21035
Reposted by Ramon Astudillo
Pekka Lund @pekka.bsky.social · 06/10/2026
OpenAI is rumored to release some 400 math results today/tomorrow and people are apparently now hurriedly releasing their results before that, in fears that set contains the same solutions. I guess we are about to find out how much duplicate work there has been during postponed releases.
6776
Reposted by Ramon Astudillo
Giovanni Monea @giomonea.bsky.social · 06/10/2026
Very excited to finally share my IBM work. It was an intense but rewarding summer, and I had a blast working with my talented collaborators at IBM & Cornell ( @keshavramji.bsky.social @yelkurdi.bsky.social @lastrasl.bsky.social @yoavartzi.com @nthngdy.bsky.social @ramon-astudillo.bsky.social )
091
Reposted by Ramon Astudillo
Giovanni Monea @giomonea.bsky.social · 06/10/2026
⚠️ Stop pretraining your looped Transformer with a separate KV cache per recursion! In our recent preprint, we show that *memory sharing* not only saves memory but is also a net-positive inductive bias! Less memory, same flops, higher quality. 🔗 arxiv.org/abs/2610.02383 🧵
17012
Ramon Astudillo @ramon-astudillo.bsky.social · 06/10/2026
Its interesting that Mistral went for 1T while Reflection went for 0.5T. RL is becoming king. I wonder how the pre-training vs RL post-training napkin math looks like reflection.ai/blog/introdu...
reflection.ai
Introducing Beam: Reflection’s 501B open-weight model — Reflection
Make intelligence open and accessible to all
051
Ramon Astudillo @ramon-astudillo.bsky.social · 06/10/2026
Its good enough if you factor in the sovereign worries?
100
Ramon Astudillo @ramon-astudillo.bsky.social · 06/10/2026
mistral.ai/news/mistral... 1T parameter model!
mistral.ai
Introducing Mistral Large 4 | Mistral
The most powerful AI platform for enterprises. Customize, fine-tune, and deploy AI assistants, autonomous agents, and multimodal AI with open models.
011
Ramon Astudillo @ramon-astudillo.bsky.social · 06/10/2026
Everybody knows the Chuk Norris joke, but the stature of Jeff Dean is very much earned. The first minutes in this interview is a good sample www.ycrootaccess.com/p/jeff-dean-...
ycrootaccess.com
Jeff Dean: Where Startups Can Still Win
Google's Chief Scientist on specialized hardware, context engineering, and picking problems the general models fail at completely.
000
Ramon Astudillo @ramon-astudillo.bsky.social · 06/10/2026
Think at the speed we are moving. This is July 2026, how will July 2027 be?
030
Ramon Astudillo @ramon-astudillo.bsky.social · 06/10/2026
Alexander Wang say this "back in my days" as a joke here, but the change is true, and is pretty amazing ramon-astudillo.github.io/notes/audio/...
ramon-astudillo.github.io
Alexandr Wang: “This is a Once-in-a-Civilization Opportunity” · Y Combinator Startup Podcast @ 25:00
“So, you know, when I started a company, back in my day, we wrote code. And now, you know, I'm sure nobody here writes code anymore. That's ridiculous. But now it's about how do you orchestrate the ag...
020
Ramon Astudillo @ramon-astudillo.bsky.social · 05/10/2026
Remember 100K GPUs was what GPT-4.5 was using in 2024 (ballpark). Size can't be too far from current K3's ~3T
020
Ramon Astudillo @ramon-astudillo.bsky.social · 05/10/2026
Experimenting with audio notes. If BlueSky is a "social annotation layer", the more things we can annotate the better
000
Ramon Astudillo @ramon-astudillo.bsky.social · 05/10/2026
ramon-astudillo.github.io/notes/audio/...
ramon-astudillo.github.io
Ep. 030 - Long Live the Short King: Why 4-HI HBM Wins (Memory) | Myron Xie, Jordan Nanos · SemiAnalysis Weekly @ 14:43
“it's pretty clear to tell if a given site is targeted for pre-training or not [...] if you're going to have more than 100,000 GPUs in an individual building [...] that's a very different data center ...
010
Reposted by Ramon Astudillo
Codetaur @vibe-coded.com · 04/10/2026
building yet another wind data particle layer for mapbox in webgl and while cranking the parameters up to this point probably isn't very useful, it looks preetty cool
1017920
Ramon Astudillo @ramon-astudillo.bsky.social · 01/10/2026
If we believe tech just lowered the barriers to publication, which is very plausible, one would expect the latter, so this may not help much
100