Sign in

Pablo Samuel Castro

@pcastr.bsky.social
3.6K followers 343 following 426 posts

Señor swesearcher @ Google DeepMind, adjunct prof at Université de Montréal and Mila. Musician. From 🇪🇨 living in 🇨🇦. psc-g.github.io

PostsRepliesMedia
Pablo Samuel Castro @pcastr.bsky.social · 20/08/2026
Sorry to have missed it, but glad Joelle was able to join!
010
Pablo Samuel Castro @pcastr.bsky.social · 19/08/2026
Actually, I won't be able to make it tomorrow, unfortunately!
100
Pablo Samuel Castro @pcastr.bsky.social · 18/08/2026
Third #runconference at #RLC2026 going strong! We'll do one last tomorrow, same place, same time, for those who are still around.
150
Pablo Samuel Castro @pcastr.bsky.social · 17/08/2026
Another great #runconference at #RLC2026 ! Same place, same time tomorrow!
080
Pablo Samuel Castro @pcastr.bsky.social · 16/08/2026
If you're closer to the venue, Josiah will be starting at 7 right in front of the conference venue #rlconference #RLC2026 maps.app.goo.gl/UoqpDjGgQDch...
010
Pablo Samuel Castro @pcastr.bsky.social · 16/08/2026
Great turnout for our first #runconference at #RLC2026 ! Same place, same time tomorrow!
080
Pablo Samuel Castro @pcastr.bsky.social · 16/08/2026
Great talk by Marc G. Bellemare at #RLC2026 , ranging from deep RL representations, to stable LLM post-training, to robust statistical measures of performance!
0121
Pablo Samuel Castro @pcastr.bsky.social · 15/08/2026
At #RLC2026 and want to run? Come for a #runconference tomorrow (Sunday)! Let's meet at 7am at the pin below and run up the mountain for a beautiful morning view. See you there! maps.app.goo.gl/w7e7CqRbARdG...
062
Pablo Samuel Castro @pcastr.bsky.social · 15/08/2026
If you're at #RLC2026, come by, starting in 15ish minutes! I'm currently making my way from the airport but should arrive just in the nick of time 😅 Looking forward to moderating this debate!
1102
Pablo Samuel Castro @pcastr.bsky.social · 11/07/2026
Fifth and last #runconference at #ICML2026 was great! Thanks to everyone who came out, and until next time!
0110
Pablo Samuel Castro @pcastr.bsky.social · 10/07/2026
I love banter, but I almost never see it in panels. Usually the banter I like happens in the hallway outside amongst those of us escaping the panel inside the room 🤭 More seriously, though, see Mark's response
010
Pablo Samuel Castro @pcastr.bsky.social · 10/07/2026
This.
010
Pablo Samuel Castro @pcastr.bsky.social · 10/07/2026
in workshops, when panels start, i leave.
260
Pablo Samuel Castro @pcastr.bsky.social · 10/07/2026
Fourth #runconference at #ICML2026 was a bit of a bust, as separate running club cancelled last minute 🫤 Still had a nice (and very humid) solo run. Tomorrow will be the last one, we'll meet at 8am at Hannam station for a run around the river, join us! naver.me/F5swAa7n
030
Pablo Samuel Castro @pcastr.bsky.social · 09/07/2026
020
Pablo Samuel Castro @pcastr.bsky.social · 09/07/2026
Interested in the differences in strategic behavior between humans and LLMs? Come chat with me and @caroline-wang.bsky.social this Thursday afternoon, 5pm, #ICML2026 poster session in Hall A (Poster #401)!
1113
Pablo Samuel Castro @pcastr.bsky.social · 09/07/2026
No rain for third #runconference at #ICML2026 but it was quite humid! Tomorrow we'll join a separate running club at 8am, running around some of the main sights in Seoul, join us! Apparently anyone can join, just register here: luma.com/o0wqzdyc
000
Pablo Samuel Castro @pcastr.bsky.social · 08/07/2026
Second #runconference at #ICML2026 was *very* wet, but we did it! Join us tomorrow, same place and same time!
040
Pablo Samuel Castro @pcastr.bsky.social · 08/07/2026
011
Pablo Samuel Castro @pcastr.bsky.social · 08/07/2026
Sanity prevailed!
010
Pablo Samuel Castro @pcastr.bsky.social · 08/07/2026
Today I'll be presenting this spotlight poster with Ali at 2:30-4:15, in Hall A, poster #303. Come chat with us about good representations for RL! #ICML2026
160
Pablo Samuel Castro @pcastr.bsky.social · 08/07/2026
At the Google booth until 11. Stop by! #icml2026
180
Pablo Samuel Castro @pcastr.bsky.social · 07/07/2026
Today at the #ICML2026 LatinX in AI workshop Ali did a great job presenting our paper! We'll be presenting the poster at the main conference, tomorrow Wednesday July 8th, 2:30-4:15, Hall A, poster #303. Come chat with Ali and me then!
080
Pablo Samuel Castro @pcastr.bsky.social · 07/07/2026
First #runconference at #ICML2026 was rainy but good! We ran up to the city wall and got nice views of the city. Join us tomorrow, same place, same time!
140
Pablo Samuel Castro @pcastr.bsky.social · 06/07/2026
At Seoul for #ICML2026 and want to start your days with a run? Meet me tomorrow Tuesday at 7am in front of COCODA [NAVER Maps] COCODA 398-6 Jongno 5(o)-ga Jongno-gu Seoul naver.me/xuFb2ppi #runconference
051
Pablo Samuel Castro @pcastr.bsky.social · 01/07/2026
Finally, most of the credit goes to @johanobandoc.bsky.social who led this project! Thanks to a great set of co-authors Lu Li, Scott Fujimoto, @pierrelucbacon.bsky.social , and Aaron Courville for countless insightful discussions throughout this work.
050
Pablo Samuel Castro @pcastr.bsky.social · 01/07/2026
Paper: arxiv.org/abs/2606.05555 Code: github.com/johanobandoc... Thanks for reading! If you're interested in multitask RL, representation learning, or scalable reinforcement learning, we'd love to hear your thoughts and feedback. I'll also be at #ICML2026 next week 🇰🇷
arxiv.org
Representation Learning Enables Scalable Multitask Deep Reinforcement Learning
Scaling reinforcement learning (RL) to diverse multitask settings remains a central challenge. While recent advances in model-based RL achieve strong performance, they rely on planning and complex tra...
160
Pablo Samuel Castro @pcastr.bsky.social · 01/07/2026
The broader takeaway is not that planning is unnecessary. Rather, we should carefully distinguish the benefits of representation learning from the benefits of planning. As RL continues to scale, understanding which ingredients matter is essential Paper and code are below. 👇
130
Pablo Samuel Castro @pcastr.bsky.social · 01/07/2026
Perhaps even more interesting: The learned representations transfer remarkably well. MR.Q achieves stronger zero-shot performance on unseen tasks and adapts faster during few-shot finetuning, suggesting that better representations naturally lead to better transfer. 10/🧵
120
Pablo Samuel Castro @pcastr.bsky.social · 01/07/2026
What do those representations look like? Each color corresponds to a different task. Predictive representations lead to structured, high-dim latent spaces. W/out them, representations collapse & are less expressive. 9/🧵
130
Pablo Samuel Castro @pcastr.bsky.social · 01/07/2026
Why does it work? Our analyses suggest predictive objectives lead to: 💰Richer latent representations 💠Higher effective representation dimensionality 🧠Fewer dormant neurons ⚖️More stable value learning 8/🧵
140
Pablo Samuel Castro @pcastr.bsky.social · 01/07/2026
The improvements go well beyond final performance. We observe: 📈Higher sample efficiency ⏱️Better wall-clock efficiency 🦣Better scaling with larger models 💪🏾Greater robustness in low-data regimes 🚀All without explicit planning. 7/🧵
130
Pablo Samuel Castro @pcastr.bsky.social · 01/07/2026
The results were surprisingly strong. MR.Q consistently matches or outperforms, the model-based baseline across diverse multitask domains, while relying on a significantly simpler learning pipeline and without explicit planning. 6/🧵
120
Pablo Samuel Castro @pcastr.bsky.social · 01/07/2026
We eval on MMBench [1] (@ncklashansen.bsky.social ), a benchmark w/ 200 diverse tasks spanning multiple domains and embodiments, each w/ lang instructions, demonstrations, and optional visual observations. We compare against Newt, a recent large-scale multitask world model built for it. 5/🧵
140
Pablo Samuel Castro @pcastr.bsky.social · 01/07/2026
If representation learning is key ingredient, then a purely model-free algo should achieve competitive multitask perf— even without planning. To test this, we revisit MR.Q (Fujimoto et al), a model-free RL algorithm and augment it with predictive auxiliary objectives. 4/🧵
130
Pablo Samuel Castro @pcastr.bsky.social · 01/07/2026
Modern world-model methods combine several ingredients: • Predictive objectives • Latent representations • Planning • Large neural networks We set out to determine which ones are actually responsible for the observed gains. 3/🧵
140
Pablo Samuel Castro @pcastr.bsky.social · 01/07/2026
What actually makes modern RL systems scale? Many recent successes attribute their performance to world models and planning. Our work suggests a different perspective: 👉 It may be representation learning, not planning, that's doing most of the heavy lifting. 2/🧵
140
Pablo Samuel Castro @pcastr.bsky.social · 01/07/2026
I'm excited to share our latest work “Representation Learning Enables Scalable Multitask Deep Reinforcement Learning”. In this work, we revisit a fundamental question in reinforcement learning: What if representation learning was the key ingredient behind scalable RL? 1/🧵
15213
Pablo Samuel Castro @pcastr.bsky.social · 04/06/2026
really proud of this work done with fantastic collaborators, in particular daniel, @kevinjmiller.bsky.social & @neurokim.bsky.social ! i learned so much about neuroscience, & it was great to work on using AI techniques to advance science. read kevin's thread below or, even better, read the paper!
0121
Pablo Samuel Castro @pcastr.bsky.social · 29/04/2026
I also feel bad for these famous researchers... Giving multiple talks in the span of a few days is a lot of work!
010
Pablo Samuel Castro @pcastr.bsky.social · 29/04/2026
This of course has to be coordinated by workshop chairs, and I acknowledge it's not an easy solution. Usually workshop applications include the keynotes, so maybe chairs could facilitate coordination across workshops to avoid this, and encourage a bit more diversity.
110
Pablo Samuel Castro @pcastr.bsky.social · 29/04/2026
Those famous researchers are of course incredible, which is why they're famous, and I always learn from their talks. But I think having them on just one workshop (as opposed to multiple) would still be great and allow for other, less famous but still great, researchers to opine.
110
Pablo Samuel Castro @pcastr.bsky.social · 29/04/2026
I wish there were a way to increase diversity in workshop keynotes/panelists. There are a few famous researchers who end up being keynotes/panelists on multiple workshops, which means lots of other great researchers are not getting those opportunities.
130
Pablo Samuel Castro @pcastr.bsky.social · 27/04/2026
Fun last #runconference at #ICLR2026, included sidewalk and beach running, a little soccer ball kicking, and going in the Ocean! also passed some lifeguards in training during my last sprint
070
Pablo Samuel Castro @pcastr.bsky.social · 26/04/2026
That's an interesting question! We didn't try that as we were trying to shape a single point in the network (i.e. the "representation"). Using residuals makes sense to avoid gradient issues, perhaps something like the MultiSkip residuals we proposed in another recent paper: bsky.app/profile/roge...
010
Pablo Samuel Castro @pcastr.bsky.social · 26/04/2026
Another great #runconference this morning at #ICLR2026 ! Tomorrow is my last day so trying something different: BEACH RUN Let's meet at the pin below at 7:30am, run for a bit, then go in the water for a swim! maps.app.goo.gl/KZfuFtgsdmn4...
060
Pablo Samuel Castro @pcastr.bsky.social · 26/04/2026
I will be giving a talk about this work at the "From Human Cognition to AI Reasoning: Models, Methods, and Applications" #ICLR2026 workshop 🗓️Sunday, April 26, 2026 10:30 AM BRT 📍 Room 202 C As well as a poster at 3:30pm in the same room. Come by!
050
Pablo Samuel Castro @pcastr.bsky.social · 25/04/2026
Come by!
040
Pablo Samuel Castro @pcastr.bsky.social · 25/04/2026
Still at #ICLR2026 and still into RL? Come talk to @johanobandoc.bsky.social @waltermayor.bsky.social and me at our poster today (Saturday)! 🗓️April 25, 3:15 PM -5:45 PM BRT 📍Pavilion 4· #️⃣ Poster #4515
292
Pablo Samuel Castro @pcastr.bsky.social · 25/04/2026
Today (Saturday) I'll be at the Google both at #ICLR2026 , 1:30-2pm, if you'd like to come chat.
070