Sign in

Petar Veličković

@petar-v.bsky.social
2.9K followers 154 following 38 posts

Senior Staff Research Scientist, Google DeepMind Affiliated Lecturer, University of Cambridge Associate, Clare Hall GDL Scholar, ELLIS @ellis.eu 🇷🇸🇲🇪🇧🇦

PostsRepliesMedia
Petar Veličković @petar-v.bsky.social · 20/06/2025
You may read this chapter in the usual place: geometricdeeplearning.com/book/ 📖 Any and all feedback / thoughts / questions on the content... words of encouragement for finishing this book (pretty please! 😇) warmly welcomed! @mmbronstein.bsky.social @joanbruna.bsky.social @taco-cohen.bsky.social
1193
Petar Veličković @petar-v.bsky.social · 20/06/2025
The first draft 'G' chapter of the geometric deep learning book is live! 🚀 Alice enters the magical, branchy world of Graphs and GNNs 🕸️ (LLMs are there too!) I've spent 7+ years studying, researching & talking about graphs -- This text is my best attempt at conveying everything i've learnt 💎
1235
Petar Veličković @petar-v.bsky.social · 26/05/2025
...but our model regressed on Quickselect rather than improving it, so I'm indeed curious to explore what you've done differently :)
110
Petar Veličković @petar-v.bsky.social · 26/05/2025
Cool stuff! Will give it a read. We also built a 'fully tropical' GNN following similar principles a few years back (from the perspective of asynchrony invariance), might be a good reference: arxiv.org/abs/2306.15632
110
Petar Veličković @petar-v.bsky.social · 29/04/2025
I would say it definitely does! The book does rely on theoretical material but is designed to be a self-contained, deep learning-oriented introduction to those topics. So if you read the chapters in sequence it _should_ be just fine :) (would appreciate any feedback if you do try it!)
000
Petar Veličković @petar-v.bsky.social · 29/04/2025
Read on at geometricdeeplearning.com/book 📖 Comments, errata reports, and feedback warmly welcomed!
geometricdeeplearning.com
GDL Book
Grids, Groups, Graphs, Geodesics, and Gauges
040
Petar Veličković @petar-v.bsky.social · 29/04/2025
4yrs ago, the Geometric Deep Learning proto-book hit the arXiv 📚 Today, we proudly release *Chapter 4* of the GDL Book! 📖 Wrapping up foundations, exceeding the proto-book page count 🚀 Next: deep-dives into the 5 Gs! #⃣🔁🕸️🌐🪢 @mmbronstein.bsky.social @joanbruna.bsky.social @taco-cohen.bsky.social
2575
Petar Veličković @petar-v.bsky.social · 31/03/2025
youtu.be/J1YCdVogd14?...
youtu.be
LLMs as Graph Neural Networks | Petar Veličković @ GLOW
YouTube video by GLOW
050
Petar Veličković @petar-v.bsky.social · 31/03/2025
The recording of my 'LLMs as GNNs' talk is now public (link in thread) 💬🕸️ Thank you to everyone in the GLOW community for having me -- I had a fantastic time! 🌟 (And I look forward to refining this content even further -- especially as new works come trickling in 👀)
1121
Petar Veličković @petar-v.bsky.social · 26/03/2025
This is happening today! 🌟 Join me at the virtual GLOW seminar (5pm CET) for the first public showing of my 'LLMs as GNNs' talk. 💬🕸️
0122
Reposted by Petar Veličković
Max Thiessen @maxthiessen.bsky.social · 20/03/2025
GLOW is returning on 𝗠𝗮𝗿𝗰𝗵 𝟮𝟲𝘁𝗵, 𝟱𝗽𝗺 𝗖𝗘𝗧 with a special guest: @petar-v.bsky.social 🌟 He will lecture on LLMs as GNNs – a topic which received quite some attention at our last session. Specifically, we will learn how Graph ML tools can help understand LLM generalisation
1127
Petar Veličković @petar-v.bsky.social · 14/02/2025
Seminar details + join instructions here: www.cst.cam.ac.uk/seminars/lis... Hope to see many of you there!
cst.cam.ac.uk
Probabilistic weather forecasting with machine learning | Department of Computer Science and Technology
Abstract: Leading up to 2024, rapid advances in machine learning (ML)-based weather prediction (MLWP) had produced ML-based models which exhibit less forecast error than single NWP simulations.
020
Petar Veličković @petar-v.bsky.social · 14/02/2025
I'm excited to share that we'll have Ilan Price giving a talk at the University of Cambridge on GenCast -- a state-of-the-art model for probabilistic weather forecasting 🌦️⛈️⛅️ If you're in Cambridge next Wednesday (19 Feb), consider joining us -- it's open to all! All details (+ Zoom link) below!
1102
Petar Veličković @petar-v.bsky.social · 11/02/2025
It's been great working on this with Alex, @joao.omg.lol and Marc on this over the past couple of months! We don't believe we solved the problems studied here -- mainly brought them to attention -- and we welcome any and all comments! Read our work on the arXiv: arxiv.org/abs/2502.06751
arxiv.org
What makes a good feedforward computational graph?
As implied by the plethora of literature on graph rewiring, the choice of computational graph employed by a neural network can make a significant impact on its downstream performance. Certain effects ...
060
Petar Veličković @petar-v.bsky.social · 11/02/2025
And so we set out to understand _feedforward_ graphs (i.e. graphs w/o back edges) ⏩ Turns out these graphs are rather understudied for how often they are used in practice! 😮 We work towards a framework to analyse them, which I hope both my GNN and my LLM friends will enjoy 🚀
1293
Reposted by Petar Veličković
Chaitanya K. Joshi @chaitjo.bsky.social · 10/02/2025
Trying to copy the greats Pietro and @petar-v.bsky.social giving a guest lecture for the GDL class today!
0151
Reposted by Petar Veličković
Simone Scardapane @sscardapane.bsky.social · 10/02/2025
*Round and Round We Go! What makes Rotary Positional Encodings useful?* by @petar-v.bsky.social et al. They show RoPE has distinct behavior for different rotation angles - high freq for position, low freq for semantics. arxiv.org/abs/2410.06205
061
Petar Veličković @petar-v.bsky.social · 05/02/2025
This summer, a group of elite AI scientists and engineers will come over to Sarajevo 🇧🇦 for #EEML2025 ❤️ We've been working hard behind the scenes to prepare a stellar program for our attendees! Trust me, you don't want to miss this 🚀 Applications open now! (details in post)
0140
Petar Veličković @petar-v.bsky.social · 19/12/2024
Application forms should open early 2025 :)
010
Petar Veličković @petar-v.bsky.social · 16/12/2024
Exciting news: the NeurIPS Sci4DL Workshop has recognised our softmax paper as a best paper runner-up for its 'Debunking Challenge'! 🚀🧑‍🔬 I'm really grateful to Christos and Federico for tirelessly presenting our work throughout the day, and to all attendees who kindly stopped by our talks/posters! 🙌
0100
Petar Veličković @petar-v.bsky.social · 15/12/2024
It's official! 🎉 I am excited to share that EEML will be coming to Sarajevo 🇧🇦 next summer! For those not familiar, EEML is the flagship AI summer school for the Eastern European region (and beyond!). Watch this space! More information on how to apply, lecturers, etc... is coming soon 👀
1141
Petar Veličković @petar-v.bsky.social · 15/12/2024
Today it is time to present our softmax paper to the world! 🌡️ If you're at #NeurIPS2024 today, check out our two Workshop spotlights 🔦 🧑‍🔬 Sci4DL (by Christos Perivolaropoulos), West Meeting Rooms 205-207, 10:50--11:05am 🔢 System II (by Federico Barbero), West Ballroom B, 9:20--9:30am
0212
Petar Veličković @petar-v.bsky.social · 08/12/2024
newsletter.pnote.eu/p/hashcode-d...
newsletter.pnote.eu
AI from DeepMind beats top human competitive programmers on Hash Code problems
🏆 A battle of heuristics. Evolutionary approach to improving a program. A "Deep Blue" moment for competitive programming?
030
Petar Veličković @petar-v.bsky.social · 08/12/2024
A very nice blog from Przemek Pietrzkiewicz, offering thoughts on our recent result in AI for competitive programming 🏆 Przemek co-led the Hash Code contest, which we used as the main test-bed to evaluate our approach 🚀 Worth a read if you want to understand implications of our work! Link below ⬇️
1223
Reposted by Petar Veličković
Marc Lanctot @sharky6000.bsky.social · 06/12/2024
If you will be at #NeurIPS2024 @neuripsconf.bsky.social and would like to come see our models in action, come say hi 👋 and check out our demo at the GDM booth! Wednesday, Dec. 11th @ 9:30-10:00. Lots of other great things to see as well! Check it out: 👇 deepmind.google/discover/blo...
1416
Petar Veličković @petar-v.bsky.social · 05/12/2024
🚀 To apply: 1⃣ Fill out the "Student Researcher 2025" application form for EMEA*: www.google.com/about/career... 2⃣ Email Larisa Markeeva (the project host) and myself, letting us know you've applied 🎉 Also please feel free to contact us if you have any questions! *this role is onsite in London 🇬🇧
google.com
081
Petar Veličković @petar-v.bsky.social · 05/12/2024
Our team at Google DeepMind is hiring Student Researchers for 2025! 🧑‍🔬 Interested in understanding reasoning capabilities of neural networks from first principles? 🧑‍🎓 Currently studying for a BS/MS/PhD? 🧑‍💻 Have solid engineering and research skills? 🌟 We want to hear from you! Details in thread.
2585
Reposted by Petar Veličković
Nenad Tomasev @nenadtomasev.bsky.social · 05/12/2024
I'm excited to share a new paper: "Mastering Board Games by External and Internal Planning with Language Models" storage.googleapis.com/deepmind-med... (also soon to be up on Arxiv, once it's been processed there)
storage.googleapis.com
47614
Reposted by Petar Veličković
Jack Parker-Holder @jparkerholder.bsky.social · 04/12/2024
Introducing 🧞Genie 2 🧞 - our most capable large-scale foundation world model, which can generate a diverse array of consistent worlds, playable for up to a minute. We believe Genie 2 could unlock the next wave of capabilities for embodied agents 🧠.
1523461
Petar Veličković @petar-v.bsky.social · 04/12/2024
Just a poster I'm very proud of. Mainly because it's the first time I led a theory-focused paper 🥳 Coming soon to NeurIPS Workshops near you (two spotlights!!) 🔦 I unfortunately won't be there myself, but Christos and Federico will be around 🚀
1424
Petar Veličković @petar-v.bsky.social · 02/12/2024
Thank you to all who worked together with me on this endeavour: Alex, Larisa, Borja, Lars, Matej & Sasha! As well as to the many folks at Google & GDM who supported us on our journey. It truly took a village! Want to know more? Check out our technical report on the arXiv: arxiv.org/abs/2411.19744
arxiv.org
Amplifying human performance in combinatorial competitive programming
Recent years have seen a significant surge in complex AI systems for competitive programming, capable of performing at admirable levels against human competitors. While steady progress has been made, ...
000
Petar Veličković @petar-v.bsky.social · 02/12/2024
To gather additional evidence that results are robust, we ran our approach on a variant of one of the very few heuristic contests that took place _after_ Gemini 1.5 Flash 002 was released: the AtCoder Heuristic Contest 039. To our delight, the approach still hillclimbs well! 🏔️
120
Petar Veličković @petar-v.bsky.social · 02/12/2024
While Hash Code contests are in the past, we've strong reason to believe this is not solution retrieval: * Gemini isn't given the task statement; * Solutions must conform to the backbone API (not in the training data); * Chain of 10+ LLM calls is needed! OQR'15 (Optimize a Data Center) example:
120
Petar Veličković @petar-v.bsky.social · 02/12/2024
The system works quite well! 🔥 On _all_ eight previous Hash Code Online Qualification Rounds, our approach breaks into the first percentile of all human competitors' scores (including some of the world's best). 🎉 It even achieves a rank-1 result on five of the past rounds! 🤯
100
Petar Veličković @petar-v.bsky.social · 02/12/2024
Our AI system is based on the FunSearch algorithm, with one key difference: modern generalist LLMs are sufficiently strong that we no longer require fine-tuning specialist code models. We instead leverage the mighty Gemini 1.5 Flash 002 ⚡️⚡️⚡️ which proved rather potent!
100
Petar Veličković @petar-v.bsky.social · 02/12/2024
This complementarity is embraced by our approach, which is _synergistic_ 🤝 human competitors write the backbone of a greedy solution, and we ask evolutionary programming and LLMs to evolve the function that steers it! This division of work streamlines the requirements of the AI system.
100
Petar Veličković @petar-v.bsky.social · 02/12/2024
In contrast, combinatorial tasks present a very different challenge: tasks are intractable (NP-hard). This levels the playing field: humans will not know the optimal solution, and suboptimal solutions score >0 points. This allows AI to explore the solution space in a way that complements humans 🔎
100
Petar Veličković @petar-v.bsky.social · 02/12/2024
There's been a rightful surge of AI-powered competitive programming systems, typically deployed on classical contests such as Codeforces. While very impressive results have been achieved (ELO ~1,900), they are still significantly away from the highest percentiles of competitors.
110
Petar Veličković @petar-v.bsky.social · 02/12/2024
A clear step towards achieving my dream: building AI that assists competitive programmers 🧑‍💻 “This is an exciting approach to combine work of human competitive programmers and LLMs, to achieve results that neither would achieve on their own.” --Petr Mitrichev More details below! 🧵
2233
Petar Veličković @petar-v.bsky.social · 28/11/2024
'In research, take the time needed to do something, multiply it by ~10, and that's how long it will actually take'
000
Petar Veličković @petar-v.bsky.social · 27/11/2024
The key productivity hack I unlocked in recent years was accepting and being cognisant of the fact that I likely won't finish everything I was planning to do at the start of the day. 😶 Ironically, this led to ~2x more things being finished, because I spend less time worrying about falling behind 🥳
3420
Reposted by Petar Veličković
logconference.bsky.social @logconference.bsky.social · 24/11/2024
✨EXCITING NEWS! We now have a Bluesky account!! AND registration for the 3rd Learning on Graphs conference is open 😊 It is virtual, free to attend, livestreamed, and recorded 📹 Sign up today! The conference is this coming week 😱 forms.gle/eYiDCopJGUc8...
Screen capture of LoG website. The conference is Nov 26-29, 2024. See our website for more details: https://logconference.org/.
02612
Petar Veličković @petar-v.bsky.social · 23/11/2024
arxiv.org/abs/2410.01104
arxiv.org
softmax is not enough (for sharp out-of-distribution)
A key property of reasoning systems is the ability to make sharp decisions on their input data. For contemporary AI systems, a key carrier of sharp behaviour is the softmax function, with its capabili...
0110
Petar Veličković @petar-v.bsky.social · 23/11/2024
As an example from our recent paper, turns out that assuming a _tokenised vocabulary_ input allows one to apply mathematical analysis results due to Weierstrass, which in turn implies that all sharp attention heads must collapse. Crazy chain of thought, and not at all what we expected at the start.
2150
Petar Veličković @petar-v.bsky.social · 23/11/2024
My fav theoretical proofs in AI papers tend to be ones that make assumptions that: * realistically conform to the 'situation on the ground', i.e. ~match models used in practice; * completely catch you off-guard, i.e. you see the assumption and you have no upfront idea what issue it will help patch.
0182