iiinova — where your digital self keeps evolving

Alvaro Bartolome @alvarobartt.com · 01/07/2026

I wrote a post so you can try GLM 5.2 on Microsoft Foundry from Codex too. alvarobartt.com/goal-glm-5.2...

Alvaro Bartolome @alvarobartt.com · 01/07/2026

GLM 5.2, open frontier-scale intelligence on Microsoft Foundry with AMD MI300X. Running a Codex goal with an open model never felt this good!

150

Alvaro Bartolome @alvarobartt.com · 17/02/2026

- 💚 NVIDIA Blackwell support, ready for next-gen GPUs as B200, GB200, or RTX 50-series.

100

Alvaro Bartolome @alvarobartt.com · 17/02/2026

- 🔄 Add bidirectional attention support for 3, enabling newer embedding models as Voyage AI by MongoDB.

100

Alvaro Bartolome @alvarobartt.com · 17/02/2026

- 🦙 Add support for Meta Llama 2 and 3 architectures with Flash Attention support, enabling embedding models as NVIDIA Llama Embed Nemotron.

100

Alvaro Bartolome @alvarobartt.com · 17/02/2026

- 🎉 Add support for Microsoft Deberta V2 and V3, for both feature-extraction (and sentence-similarity) and text-classification, enabling models as Meta Llama Prompt Guard.

100

Alvaro Bartolome @alvarobartt.com · 17/02/2026

More embedding models and an even more reliable inference engine is what you get with @hf.co Text Embeddings Inference v1.9.0 💥 More in the thread 🧵

143

Alvaro Bartolome @alvarobartt.com · 05/01/2026

`hf-mem` is all you need to estimate the required VRAM for inference of any model on @huggingface based on Safetensors metadata. - Written in Python - Lightweight, only depends on `httpx` - Runs w/ `uvx` as `uvx hf-mem ...` - Works with any Safetensors repository - Output inspired by usgraphics.com

101

Alvaro Bartolome @alvarobartt.com · 10/02/2025

For anyone interested in Zig I wrote a small post titled "How to read and parse JSON with Zig 0.13" that explains how to read JSON from a file with keys with different value types and how to access those values.

110

Alvaro Bartolome @alvarobartt.com · 31/01/2025

🤗 Here's a simple script that calculates the required VRAM for serving DeepSeek R1 from @huggingface Hub safetensor's metadata! P.S. The result of the script above is: "model_id='deepseek-ai/DeepSeek-R1' requires memory=756.716GB"

040

Alvaro Bartolome @alvarobartt.com · 23/01/2025

🐐 DeepSeek is not on the @hf.co Hub to take part, they are there to take over! Amazing stuff from the DeepSeek team, ICYMI they recently released some reasoning models (DeepSeek-R1 and DeepSeek-R1-Zero), fully open-source, their performance is on par with OpenAI-o1 and it's MIT licensed!

1101

Alvaro Bartolome @alvarobartt.com · 19/11/2024

Read more about the Serverless Inference API in the documentation! huggingface.co/docs/api-inference

000

Alvaro Bartolome @alvarobartt.com · 19/11/2024

🔥 Finally, if you are willing to get started quickly and experiment with LLMs feel free to give the recently released Inference Playground a try! huggingface.co/playground

100

Alvaro Bartolome @alvarobartt.com · 19/11/2024

👨‍💻 Alternatively, you can also use the Serverless Inference API programmatically via cURL, the huggingface_hub Python SDK, the openai SDK for chat completion, and much more! Find all the alternatives at huggingface.co/docs/api-inference

100

Alvaro Bartolome @alvarobartt.com · 19/11/2024

🔎 Now let's explore some of the different alternatives to run inference via the Serverless API! The most straightforward one is via the Hugging Face Hub available on the model card of the Serverless API supported models!

100

Alvaro Bartolome @alvarobartt.com · 19/11/2024

🔒 Before going on, you will first need to generate a Hugging Face fine-grained token with access to the Serverless API, as the requests need to be authenticated so keep the token safe and avoid exposing it!

100

Alvaro Bartolome @alvarobartt.com · 19/11/2024

❄️ Additionally, there are a bunch of models (around 1000) that are "Cold", meaning that those are not loaded in the Serverless API, but can be loaded when sending a request to them, also meaning that the first request may take a while until the model is loaded!

100

Alvaro Bartolome @alvarobartt.com · 19/11/2024

You may be wondering how do you know what models out of the over a million publicly available on the Hub can be used via the Serverless API? Well, we have the "Warm" tag that indicates that a model is loaded in the Serverless API and ready to be used 🔥

100

Alvaro Bartolome @alvarobartt.com · 19/11/2024

💡 Did you know that you can use over 13700 public open models and adapters on the @huggingface Hub for FREE? You just need a free account on the Hugging Face Hub (you can also subscribe to PRO to increase the requests per hour) More details on the thread 🧵

220