Sign in

Edoardo Ponti

@edoardo-ponti.bsky.social
1.3K followers 78 following 27 posts

Assistant professor in Natural Language Processing at the University of Edinburgh and visiting professor at NVIDIA | A Kleene star shines on the hour of our meeting.

PostsRepliesMedia
Reposted by Edoardo Ponti
Digital Futures @digitaluom.bsky.social · 09/06/2025
Up next on stage, Dr. @edoardo-ponti.bsky.social ( @edinburgh-uni.bsky.social / NVIDIA) 🎤 “Adaptive Units of Computation: Towards Sublinear-Memory and Tokenizer-Free Foundation Models” Fascinating glimpse into the next gen of foundation models. #FoundationModels #NLP #TokenizerFree #ADSAI2025
121
Edoardo Ponti @edoardo-ponti.bsky.social · 06/06/2025
🚀 By *learning* to compress the KV cache in Transformer LLMs, we can generate more tokens for the same compute budget. This unlocks *inference-time hyper-scaling* For the same runtime or memory load, we can boost LLM accuracy by pushing reasoning even further!
153
Reposted by Edoardo Ponti
Emile van Krieken @emilevankrieken.com · 21/05/2025
We propose Neurosymbolic Diffusion Models! We find diffusion is especially compelling for neurosymbolic approaches, combining powerful multimodal understanding with symbolic reasoning 🚀 Read more 👇
49327
Edoardo Ponti @edoardo-ponti.bsky.social · 25/04/2025
Sparse attention is one of the most promising strategies to unlock long-context processing and long-generation reasoning in LLMs. We performed the most comprehensive study on training-free sparse attention to date. Here is what we found:
1246
Reposted by Edoardo Ponti
Digital Futures @digitaluom.bsky.social · 01/04/2025
🚀 Excited to welcome Dr. @edoardo-ponti.bsky.social to #ADSAI2025! Lecturer in NLP @edinburghuni.bsky.social , Affiliated Lecturer @cambridgeuni.bsky.social & Visiting Prof NVIDIA. 🎟️ Tickets for Advances in Data Science & AI Conference 2025 are live! 🔗Secure your spot: tinyurl.com/yurknk7y #AI
043
Reposted by Edoardo Ponti
Benjamin Minixhofer @bminixhofer.bsky.social · 02/04/2025
We created Approximate Likelihood Matching, a principled (and very effective) method for *cross-tokenizer distillation*! With ALM, you can create ensembles of models from different families, convert existing subword-level models to byte-level and a bunch more🧵
Image illustrating that ALM can enable Ensembling, Transfer to Bytes, and general Cross-Tokenizer Distillation.
12514
Edoardo Ponti @edoardo-ponti.bsky.social · 31/01/2025
I have a scholarship for a PhD in efficient memory and tokenization in LLM architectures at @edinburgh-uni.bsky.social! Eligibility: UK home fee status Starting date: flexible, from July 2025 onwards. informatics.ed.ac.uk/study-with-u... Please contact me if you're interested!
064
Edoardo Ponti @edoardo-ponti.bsky.social · 31/01/2025
Code and models for Dynamic Memory Compression are finally available! Stay tuned for architectures with even more efficient inference. developer.nvidia.com/blog/dynamic...
developer.nvidia.com
Dynamic Memory Compression | NVIDIA Technical Blog
Despite the success of large language models (LLMs) as general-purpose AI tools, their high demand for computational resources make their deployment challenging in many real-world scenarios.
040
Edoardo Ponti @edoardo-ponti.bsky.social · 22/12/2024
We're hiring a lecturer or reader in embodied NLP at the University of Edinburgh! Deadline: 31 Jan 2025 Call for applications: elxw.fa.em3.oraclecloud.com/hcmUI/Candid...
02911
Edoardo Ponti @edoardo-ponti.bsky.social · 20/12/2024
**Grounded typology**: a new paradigm. Traditionally, linguists posit functions to compare forms in different languages; however, these are aprioristic and partly arbitrary. Instead, we resort to perceptual modalities (like vision) as measurable proxies for function.
140
Edoardo Ponti @edoardo-ponti.bsky.social · 12/12/2024
Two amazing papers from my students at #NeurIPS today: ⛓️💥 Switch the vocabulary and embeddings of your LLM tokenizer zero-shot on the fly (@bminixhofer.bsky.social) neurips.cc/virtual/2024... 🌊 Align your LLM gradient-free with spectral editing of activations (Yifu Qiu) neurips.cc/virtual/2024...
2468
Edoardo Ponti @edoardo-ponti.bsky.social · 28/11/2024
We had a blast at this year's @ellis.eu Dagstuhl seminar on "Modular and Agentive LLMs". Thanks everyone for participating!
0273
Reposted by Edoardo Ponti
Pasquale Minervini @neuralnoise.com · 26/11/2024
Check out this piece on Strawberry 🍓/o1 we just authored on TheConversation! theconversation.com/ai-that-mimi... with @edoardo-ponti.bsky.social and Kolya 🚀
theconversation.com
AI that mimics human problem solving is a big advance – but comes with new risks and problems
Chain of thought reasoning has been used in OpenAI’s new AI model.
174
Reposted by Edoardo Ponti
Sasha Rush @srushnlp.bsky.social · 21/11/2024
Several incredible NeurIPS tutorials this year. Worth navigating through the Swifties.
1406
Edoardo Ponti @edoardo-ponti.bsky.social · 21/11/2024
P.S. Make sure to follow @pnawrot.bsky.social!
110
Reposted by Edoardo Ponti
Alessandro Sordoni @murefil.bsky.social · 21/11/2024
Explore zero-shot routing of parameter-efficient experts with Phatgoose arxiv.org/abs/2402.05859 and Arrow arxiv.org/abs/2405.11157 w. github.com/microsoft/mttl 👉 github.com/sordonia/pg_mb… Part of "Dynamic Sparsity in ML" tuto #neurips2024, feedback welcome and join for discussions! 😊
051
Edoardo Ponti @edoardo-ponti.bsky.social · 21/11/2024
Last 5 days to apply for a PhD at #EdinburghNLP! Deadline: November 25 www.ed.ac.uk/studying/pos... If you are passionate about: - adaptive tokenization and memory in foundation models - modular deep learning - computational typology please message me or meet me at #NeurIPS2024!
ed.ac.uk
Informatics: ILCC: Language Processing, Speech Technology, Information Retrieval, Cognition
Study Informatics: ILCC: Language Processing, Speech Technology, Information Retrieval, Cognition at the University of Edinburgh. Our postgraduate degree programmes focus on natural language processin...
0208
Edoardo Ponti @edoardo-ponti.bsky.social · 20/11/2024
Another nano gem from my amazing student Piotr Nawrot! A repo & notebook on sparse attention for efficient LLM inference: github.com/PiotrNawrot/... This will also feature in my #NeurIPS 2024 tutorial "Dynamic Sparsity in ML" with André Martins: dynamic-sparsity.github.io Stay tuned!
A sparse mask of attention scores based on VerticalAndSlashAttention and a plot of loss vs sparsity ratio for various methods.
2428