iiinova — where your digital self keeps evolving

Galileo.ai @rungalileo.bsky.social · 03/07/2025

Success for AI agents varies greatly by domain and requires nuanced, domain-specific metrics. @erinmikail.bsky.social's new tutorial shows how to build and track tailored custom metrics using Galileo for reliable AI evaluation. Read Erin's blog here: galileo.ai/blog/silly-s...

020

Galileo.ai @rungalileo.bsky.social · 11/06/2025

Deploying an LLM without the right infrastructure in place is like casting spells without a spellbook. #AI #LLM #AIEvaluation #MLOps #DataQuality #Cohesity #GalileoAI #ChainOfThought #Podcast

100

Galileo.ai @rungalileo.bsky.social · 11/06/2025

We’re excited to release 2 new AI agent interfaces that make agent observability & evaluations even more effective. - Timeline View: No more guessing where your agent gets stuck, see execution flow & bottlenecks quickly. - Conversation View: Debug from the user's perspective, not just the system's

100

Galileo.ai @rungalileo.bsky.social · 11/06/2025

What do LLM evals and comedy have in common? Timing. Join @erinmikail.bsky.social at the #databricks #DataAISummit as she breaks down what it really takes to test LLMs in unexpected domains—like generating humor. Come for the eval benchmarks. Stay for the chaos. #GenAI #LLMevals #AIUX #LLMops

121

Galileo.ai @rungalileo.bsky.social · 09/06/2025

Make sure to stop by booth #120 to say hi to @erinmikail.bsky.social and the Galileo team during the #DataAISummit this week!

Galileo.ai @rungalileo.bsky.social · 06/06/2025

Next week, Galileo is headed to San Francisco for the Databricks Data + AI Summit! If you’re building with LLMs, testing agents, or just trying to trust what your models are doing in production, come find us at Booth #120

111

Reposted by Galileo.ai

Jim Bennett @jimbobbennett.dev · 03/06/2025

On my way to SF. If you’re attending the AI Engineering worlds fair and want ti learn why your AI needs reliability and evaluations come say hi at the @rungalileo.bsky.social booth.

Jim sitting on a plane wearing a white hoodie and orange glasses

Galileo.ai @rungalileo.bsky.social · 02/06/2025

Siva Surendira, CEO of Lyzr, perfectly captures why enterprises need robust AI evaluation: "I recommend Galileo as the antivirus equivalent for your AI system - you need these checks & balances. A MacBook is secure by nature, having that additional layer catches things the core system might miss."

121

Galileo.ai @rungalileo.bsky.social · 30/05/2025

🚨 Heading to the AI Engineers World’s Fair in SF next week? I’ll (@JimBobBennett) be there with the Galileo crew—booth, talks, party, and all. I’m giving a talk on “Taming Your AI Agents with Evaluations”, aka how to stop your AI from making up entire book reports (Chicago Sun-Times, we see you 👀).

023

Galileo.ai @rungalileo.bsky.social · 29/05/2025

We just dropped a new walkthrough showing you how to build powerful agents by combining MongoDB Atlas with Galileo.

101

Galileo.ai @rungalileo.bsky.social · 27/05/2025

𝗧𝗵𝗲 𝗔𝗜 𝗔𝗴𝗲𝗻𝘁 𝗘𝘃𝗮𝗹𝘂𝗮𝘁𝗶𝗼𝗻 𝗕𝗹𝘂𝗲𝗽𝗿𝗶𝗻𝘁: 𝗣𝗮𝗿𝘁 𝟭

110

Galileo.ai @rungalileo.bsky.social · 23/05/2025

In less than three years, your new coworker might not be human. 🤖 @poolsideai co-founders @JasoncWarner and @EisoKant believe AI will soon collaborate with teams inside high-consequence environments such as banking, energy, and healthcare-grade software.

000

Galileo.ai @rungalileo.bsky.social · 22/05/2025

Agentic AI isn't just reactive, it's a proactive partner. On the Chain of Thought podcast with @ConorBronsdon, @Amplitude_HQ's Chief Engineering Officer, @Wade Chambers, explains how systems like Ask Amplitude transform AI from a tool into a team of PhDs embedded in your product.

010

AI isn't a toy anymore. It's in production. Wade Chambers, Chief Engineering Officer at Amplitude, explains on the Chain of Thought podcast with ‪@conorbronsdon.bsky.social‬ that we've passed the leap-of-faith phase. The proof: ✅ Better decision-making ✅ Faster research ✅ Real agentic solutions

010

We’re live at the Microsoft for Startups booth until 2PM- come say hi! 🙌 It’s been amazing connecting with builders, developers, and curious minds. If you’re interested in agents, LLM apps, or just want to see how we’re helping to ship reliable AI apps, stop by for a Galileo demo. #MSbuild

131

We're mentioned in Jensen Huang's slides at COMPUTEX in Taipei! 🤩 Delighted to be part of NVIDIA's AI Factory Validated Designs!

100

Jim Bennett @jimbobbennett.dev · 20/05/2025

"We started Poolside because we saw AGI differently." While everyone rushed to scale the next-token prediction post-ChatGPT, Jason Warner and Eiso Kant (poolside's CEO and CTO) bet on a different path: reinforcement learning as the deeper scaling axis of intelligence itself.

000

Reposted by Galileo.ai

You at #msbuild? Here’s all the @rungalileo.bsky.social fun you can expect today.

132

Galileo partners with NVIDIA's Enterprise AI Factory, enhancing AI systems with its reliability platform. This collaboration, launched at COMPUTEX, provides tools for guardrails, synthetic data, model evaluation, and experiment tracking. Explore more: buff.ly/uYDp76M #NVIDIACOMPUTEX #AgenticAI

“Enterprise-Scale Agentic AI with Galileo + NVIDIA” featuring architecture flow diagrams showing Galileo’s evaluation metrics, agent observability, and NVIDIA’s optimized inference powering multi-agent systems.

140

Say hi to @jimbobbennett.dev, Roie Schwaber-Cohen‬, and @conorbronsdon.bsky.social at the @msft4startups.bsky.social Find us today at #MSbuild: - 1:30p - 2:05p at Hub Theater A - 2:30p - 6:30p Galileo's AI Reliability + Evals platform on Azure - 3:30p - 5p at the NVIDIA Inception Partner Showcase

032

Galileo is at #MSbuild this week! See you at the Microsoft for Startups booth.

010

Galileo.ai @rungalileo.bsky.social · 18/05/2025

“I’d never ask a customer to send me their source code & just say ‘trust me.’” — @jcw.bsky.social, CEO @poolsideai.bsky.social AI infra shouldn’t trade privacy for performance. ✅ Trust through transparency ✅ No overreach We’re here for it.

100

Oh hi! What are you excited about at #MSBuild? We’re excited to see you next week!

000

Galileo.ai @rungalileo.bsky.social · 17/05/2025

🆕 May 13th Release: Smarter Evaluations, Clearer Insights Galileo’s latest release notes update introduces powerful tools to enhance your LLM evaluation workflows: ➡️ Sessions Support ➡️ Continuous Learning for Human Feedback (CLHF) ➡️ Playground Enhancements 📄 v2docs.galileo.ai/release-notes

v2docs.galileo.ai

Release notes - Galileo

This page provides information about the latest updates and enhancements to Galileo.

000

Galileo.ai @rungalileo.bsky.social · 16/05/2025

Join @jimbobbennett.dev and the Galileo team next week at #MSbuild Can't wait to see you there!

Galileo.ai @rungalileo.bsky.social · 15/05/2025

Join us at #MSbuild on Tuesday for Evaluating Multi-Turn AI Agents with Azure AI Foundry and Galileo with Yash! Add the talk to your schedule: build.microsoft.com/en-US/sessio...

Graphic saying "Demo Session. Evaluating multi-turn Al Agents with Azure Al Foundry and Galileo Tuesday, May 20 | 1:50PM - 2:05PM Galileo Microsoft for Startups Yash Sheth Co-founder | COO Galileo"

000

Galileo.ai @rungalileo.bsky.social · 14/05/2025

What does AGI actually mean for the future of knowledge work? @poolsideai.bsky.social co-founders @jcw.bsky.social (CEO) and Eiso Kant (CTO) sat down with @conorbronsdon.bsky.social on the Chain of Thought podcast to unpack their definition of AGI. What's your working definition of AGI? 👇

Galileo.ai @rungalileo.bsky.social · 14/05/2025

What are you excited for at #MSBuild? "🛰️ Agentic applications aren’t the future—they’re already here. At #MSBuild next week, we’re especially excited to explore how developers move past traditional prompt-based LLMs into fully agentic workflows. 🔍 What talks or launches are you most excited for?

Jim Bennett @jimbobbennett.dev · 12/05/2025

Reposted by Galileo.ai

- Team onsite! Excited to meet the @rungalileo.bsky.social folks in person. - finish an epic demo for #MSBuild - my first ever baseball game! I hear the food, beer, and music are great and no one cares about the sportsball

131

Galileo.ai @rungalileo.bsky.social · 13/05/2025

Save time debugging with AI Evaluation and Monitoring- See you at #MSbuild next week!

000

Galileo.ai @rungalileo.bsky.social · 12/05/2025

The AI Translation Tipping Point? Olga Beregovaya (VP of AI at Smartling) didn’t hold back on The Chain of Thought podcast with @conorbronsdon.bsky.social: “The trajectory toward human parity in AI translation is rising sharply—especially for Romance languages.”

Galileo.ai @rungalileo.bsky.social · 09/05/2025

Check out @erinmikail.bsky.social's new AI Agent tutorial video 🤩

Galileo.ai @rungalileo.bsky.social · 09/05/2025

Logan Kilpatrick joins Conor Bronsdon on the Chain of Thought podcast to talk about Gemini’s rise in the agentic era. “Gemini 2.0 Flash just topped the Galileo AI Agent Leaderboard—93% performance at ~$0.07 per million tokens.”

001

Galileo.ai @rungalileo.bsky.social · 08/05/2025

Chasing shiny objects in AI? Let’s talk strategy. Denny Lee (Dir. of Product Mgmt, DevRel @Databricks) nailed it on see what sticks during the Chain of Thought podcast with @conorbronsdon.bsky.social!

112

Reposted by Galileo.ai

Outshift by Cisco @outshift.cisco.com · 07/05/2025

Maintaining stability, performance, and safety—the Evaluate phase—is the focus of what these #AGNTCY members do: 🔹Yash Sheth, @rungalileo.bsky.social 🔸Arvind Jain, Glean 🔹Nir Gazit, Traceloop 🔸Leonard Tang, Haize Labs 🔹Shailesh Manjrekar, Fabrix AI ➡️ cs.co/633222Cv2M

AGNTCY members Glean, Fabrix.ai, Galileo

Galileo.ai @rungalileo.bsky.social · 06/05/2025

🗞️ 𝑾𝒉𝒂𝒕 𝒅𝒐𝒆𝒔 𝑨𝑰 𝒍𝒐𝒐𝒌 𝒍𝒊𝒌𝒆 𝒊𝒏 𝒂 𝒏𝒆𝒘𝒔𝒓𝒐𝒐𝒎? For media strategy firm Magid, it means helping local news teams actually trust the GenAI tools they use to create content — and tracking performance at every step.

A case study webpage from Galileo and Magid titled “How Galileo helps Magid empower its newsroom clients with accelerated workflows and higher quality content.” The main banner features a man in a newsroom control room working with multiple monitors. The page includes Magid’s industry (AI Technology) and country (US), a summary of Magid’s use of generative AI for content repurposing, and a highlighted challenge section describing the need for consistent, high-quality journalistic outputs. A sidebar shows the product used: Observe®.

Continuation of the Galileo and Magid case study. This page includes three sections: “Solution,” “Results,” and a quote. The solution section describes Magid’s integration of Galileo’s real-time observability to support AI-driven newsroom workflows. The results section explains how Galileo enables visibility into AI outputs, helps detect hallucinations, and supports metric customization. A testimonial quote from CTO Alberto Melgoza emphasizes the importance of Galileo as a real-time trust layer in newsroom workflows. The background features a soft pink spiral line art design.

020

Galileo.ai @rungalileo.bsky.social · 06/05/2025

👀 Agent Leaderboard UPDATE: New OpenAI Models Evaluated! We've added the latest OpenAI models to our Agent Leaderboard, and the results are revealing some fascinating trends in the AI agent landscape! Check out our updated leaderboard for the complete analysis .

A performance comparison table of AI language models ranked from 1 to 15. Claude-3-7-sonnet-20250219 from Anthropic ranks 1st with the highest category score of 8.953, costing $3.00/$15.00. Google's Gemini models rank 2nd and 3rd, followed by various models from Google, OpenAI, Mistral, Deepseek, and Writer. The table includes columns for rank, model name, type (mostly labeled as "Private" with a few "Open source"), vendor, cost (input/output), and average category score, with visual bar indicators showing relative performance.

230

Galileo.ai @rungalileo.bsky.social · 05/05/2025

@officiallogank.bsky.social joins the Chain of Thought podcast with @conorbronsdon.bsky.social to explore what’s next for AI agents—and what’s been holding them back. “Most agents today? They’re like async junior digital employees. Helpful, but not fully capable.” Listen to find out why 🧐

Galileo.ai @rungalileo.bsky.social · 05/05/2025

Galileo.ai @rungalileo.bsky.social · 04/05/2025

“When you work with closed AI tools, you’re just giving feedback. With Goose, you can ship the fix yourself.” @blackgirlbytes.bsky.social from @opensource.block.xyz joins @conorbronsdon.bsky.social on the Chain of Thought podcast to share why Goose—Block’s AI agent—is open source by design.

033

Galileo.ai @rungalileo.bsky.social · 02/05/2025

This Years Earth Day, We Celebrated Something Bigger Than Just Metrics 🌎 Our team isn’t just building advanced AI evaluation & observability platform—we’re also building with purpose. We celebrated the people who make AI more sustainable. Responsible AI isn’t just a feature. It’s a team effort.

Galileo.ai @rungalileo.bsky.social · 01/05/2025

The Agent Train Has Officially Left the Station Siva Surendira, CEO of Lyzr, joins Galileo’s @conorbronsdon.bsky.social on Chain of Thought to unpack the rise of agentic systems in 2025—and how Lyzr positioned itself at the front of the movement.

Galileo.ai @rungalileo.bsky.social · 29/04/2025

Did you see that you can join @erinmikail.bsky.social, @chloecondon.bsky.social, Francesco Ciulla, and Ebony Louis at @getpieces.bsky.social AI Productivity Summit 🤩 Join us 📅 Thursday, May 1at Pieces AI Productivity Summit for a panel on The Future of Developer Experience and Productivity in AI.

122

Galileo.ai @rungalileo.bsky.social · 28/04/2025

What happens when you combine agentic workflows, the Agent Connect Protocol (ACP), and a love for music? You get the Weather DJ Agent—a fun, fully ACP-compatible AI that curates playlists based on your local weather forecast. 🕺 🔗 Read the tutorial by @erinmikail.bsky.social buff.ly/ek4Tnlk

galileo.ai

Build your own ACP-Compatible Weather DJ Agent. - Galileo AI

Learn how to create your own intelligent, multi-API AI agent using the Agent Connect Protocol (ACP). This tutorial walks you through building a weather-predicting, fashion-recommending, mood-setting…

141

Galileo.ai @rungalileo.bsky.social · 26/04/2025

Ready to break your LLM out of the sandbox? Join @erinmikail.bsky.social and @getpieces.bsky.social at Pieces AI Productivity Summit with @getpieces for a hands-on walkthrough of AgentKit! 📅 Wednesday, April 24 at 9am PT 📍 Free RSVP here lu.ma/m7wfrn4v?utm...

Outshift by Cisco @outshift.cisco.com · 23/04/2025

Reposted by Galileo.ai

See what AGNTCY's founding members @langchain.bsky.social and @rungalileo.bsky.social have to say about building the #InternetOfAgents with #AGNTCY. Get involved: cs.co/633232XwOX #OpenSource