iiinova — where your digital self keeps evolving

Moritz Laurer @moritzlaurer.bsky.social · 15/01/2025

Prompt-templates docs: moritzlaurer.github.io/prompt_templ... Templates on the hub: huggingface.co/datasets/Mor... Prompt-templates collection: huggingface.co/collections/... Paper: arxiv.org/pdf/2501.04519

moritzlaurer.github.io

Overview - Prompt Templates

A library for working with prompt templates locally or on the Hugging Face Hub

020

Moritz Laurer @moritzlaurer.bsky.social · 15/01/2025

—without GPT-4-based data distillation. 💾 While we wait for the release of code and datasets, you can already download the prompts they used from the HF Hub! Details here 👇

110

Moritz Laurer @moritzlaurer.bsky.social · 15/01/2025

🤖 A Process Preference Model (PPM) enables fine-grained evaluation of intermediate steps, improving training data quality. 🧪 The system underwent four rounds of self-evolution, progressively refining both the policy and reward models to tackle Olympiad-level math problems

110

Moritz Laurer @moritzlaurer.bsky.social · 15/01/2025

📏 The paper introduces rStar-Math, which claims to rival OpenAI o1's math reasoning capabilities by integrating Monte Carlo Tree Search (MCTS) with step-by-step verified reasoning trajectories.

100

Moritz Laurer @moritzlaurer.bsky.social · 15/01/2025

.@microsoft.com's rStar-Math paper claims that 🤏 ~7B models can match the math skills of o1 using clever train- and test-time techniques. You can now download their prompt templates from @hf.co ! 🧵

110

Moritz Laurer @moritzlaurer.bsky.social · 11/01/2025

- prompt-templates docs: moritzlaurer.github.io/prompt_templ... - all templates on the HF Hub: huggingface.co/datasets/Mor... - FACTS paper: storage.googleapis.com/deepmind-med...

moritzlaurer.github.io

Overview - Prompt Templates

A library for working with prompt templates locally or on the Hugging Face Hub

030

Moritz Laurer @moritzlaurer.bsky.social · 11/01/2025

💾 You can now download and reuse these prompt templates via the prompt-templates library! 🔄 The library simplifies sharing prompt templates on the HF hub or locally via standardized YAML files. Let’s make LLM work more transparent and reproducible by sharing more templates like this! Links 👇

110

Moritz Laurer @moritzlaurer.bsky.social · 11/01/2025

🧪 The authors tested different prompt templates on held-out data to ensure their generalization. 📚 It's highly educational to read these templates to learn how frontier labs design prompts and understand their limitations.

120

Moritz Laurer @moritzlaurer.bsky.social · 11/01/2025

📏 The paper introduces the FACTS Grounding benchmark for evaluating the factuality of LLM outputs. 🤖 Fact-checking is automated by an ensemble of LLM judges that verify if a response is fully grounded in a factual reference document.

110

Moritz Laurer @moritzlaurer.bsky.social · 11/01/2025

FACTS is a great paper from @GoogleDeepMind on measuring the factuality of LLM outputs. You can now download their prompt templates from @huggingface to improve LLM-based fact-checking yourself! 🧵

130

Moritz Laurer @moritzlaurer.bsky.social · 09/01/2025

Release: github.com/huggingface/... Mergekit: github.com/arcee-ai/mer... Mixture of judges paper: huggingface.co/papers/2409....

000

Moritz Laurer @moritzlaurer.bsky.social · 09/01/2025

⚖️ Mixture of judges: The new AllTrueJudge combines decisions from multiple binary judges for more nuanced evaluation. Read the release notes and other resources here 👇

100

Moritz Laurer @moritzlaurer.bsky.social · 09/01/2025

🛠️ Tool call support: TRL preprocessing now supports tool integration, laying the groundwork for agent fine-tuning with examples like dynamic temperature fetching in prompts.

100

Moritz Laurer @moritzlaurer.bsky.social · 09/01/2025

Perfect for tasks like stepwise reasoning. 🔀 Model merging: A new callback leverages mergekit to merge models during training, improving performance by blending reference and policy models - optionally pushing merged models to the Hugging Face Hub.

100

Moritz Laurer @moritzlaurer.bsky.social · 09/01/2025

The TRL v0.13 release is 🔥! My highlight are the new process reward trainer to train models similar to o1 and tool call support: 🧠 Process reward trainer: Enables training of Process-supervised Reward Models (PRMs), which reward the quality of intermediate steps, promoting structured reasoning.

120

Moritz Laurer @moritzlaurer.bsky.social · 07/01/2025

on revenue of $3.7 billion last year, with ChatGPT alone once costing an estimated $700,000 per day to operate. 💸🔥 - They build strong models and do great research. Whether this business model will work in the long run is one of the biggest questions in the AI economy. Source with the numbers 👇

techcrunch.com

OpenAI is losing money on its pricey ChatGPT Pro plan, CEO Sam Altman says | TechCrunch

OpenAI CEO Sam Altman says that the company is currently losing money on its $200-per-month plan because people use it more than expected.

100

Moritz Laurer @moritzlaurer.bsky.social · 07/01/2025

OpenAI is losing money on the $200/month subscription 🤯. It's crazy how expensive it is to run these largest LLMs: - ChatGPT Pro costs $200/month ($2,400/year) and is still unprofitable for OpenAI due to higher-than-expected usage. - OpenAI reportedly expected losses of about $5 billion

144

Moritz Laurer @moritzlaurer.bsky.social · 06/01/2025

Base model: huggingface.co/MoritzLaurer... Large model: huggingface.co/MoritzLaurer... Updated zeroshot collection: huggingface.co/collections/... ModernBERT collection with paper: huggingface.co/collections/...

huggingface.co

MoritzLaurer/ModernBERT-base-zeroshot-v2.0 · Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

000

Moritz Laurer @moritzlaurer.bsky.social · 06/01/2025

Great work by @answerdotai ! If you’re looking for a high-speed zeroshot classifier, give it a try! 📄 Resources below: 👇

100

Moritz Laurer @moritzlaurer.bsky.social · 06/01/2025

- 💡 What’s next? I’m preparing a newer version trained on better + longer synthetic data to fully leverage the 8k context window and improve upon the training mix of my older zeroshot-v2.0 models. I also hope that there will be a multilingual variant in the future.

110

Moritz Laurer @moritzlaurer.bsky.social · 06/01/2025

- 📉 Performance tradeoff: It performs slightly worse than DeBERTav3 on average across my zeroshot classification task collection - 🧠 Use cases: I recommend using it for scenarios requiring speed and a larger context window (8k).

100

Moritz Laurer @moritzlaurer.bsky.social · 06/01/2025

🚀 Releasing a new zeroshot-classifier based on ModernBERT! Some key takeaways: - ⚡ Speed & efficiency: It's multiple times faster and uses significantly less memory than DeBERTav3. You can use larger batch sizes and enabling bf16 (instead of fp16) gave me a ~2x speed boost - 📉 Performance tradeoff:

141

Moritz Laurer @moritzlaurer.bsky.social · 20/12/2024

Congrats @answerdotai, @LightOnIO and collaborators like @tomaarsen.com ! Paper and models here 👇https://huggingface.co/collections/answerdotai/modernbert-67627ad707a4acbf33c41deb

010

Moritz Laurer @moritzlaurer.bsky.social · 20/12/2024

Quite excited by the ModernBERT release! 0.15/0.4B small, 2T modern pre-training data and tokenizer with code, 8k context window, great efficient model for embeddings & classification! This will probably be the basis for many future SOTA encoders! I can finally stop using DeBERTav3 2021 :D

181

Moritz Laurer @moritzlaurer.bsky.social · 17/12/2024

"Open-source AI: year in review 2024": amazing Space with lots of data-driven insights into AI in 2024! Check it out 👇

121

Moritz Laurer @moritzlaurer.bsky.social · 12/12/2024

```py # !pip install prompt-templates from prompt_templates import PromptTemplateLoader prompt_template = PromptTemplateLoader.from_hub(repo_id="MoritzLaurer/closed_system_prompts", filename="claude-3-5-artifacts-leak-210624.yaml") ```

100

Moritz Laurer @moritzlaurer.bsky.social · 12/12/2024

- proposes a standard for sharing prompts (entirely locally or on the HF hub) - provides some utilities that are interoperable with the broader ecosystem Try it:

100

Moritz Laurer @moritzlaurer.bsky.social · 12/12/2024

systematically test and improve our systems. After comparing different community approaches, I think that working with modular .yaml or .json files is the best approach. The `prompt-templates` library : ...

100

Moritz Laurer @moritzlaurer.bsky.social · 12/12/2024

.yaml/.json/.jinja2 files etc. This makes sharing and working with prompt templates unnecessarily complicated. Prompt templates are currently the main hyperparameter that people tune when building complex LLM systems or agents. If we don't have a common standard for sharing them, we cannot ...

100

Moritz Laurer @moritzlaurer.bsky.social · 12/12/2024

I've been building a small library for working with prompt templates on the @huggingface.bsky.social Hub: `pip install prompt-templates`. Motivation: The community currently shares prompt templates in a wide variety of formats: in datasets, in model cards, as strings in .py files, as .txt/... 🧵

1164

Reposted by Moritz Laurer

narsilou.bsky.social @narsilou.bsky.social · 10/12/2024

Performance leap: TGI v3 is out. Processes 3x more tokens, 13x faster than vLLM on long prompts. Zero config !

1196

Reposted by Moritz Laurer

Damián Pumar @damianpumar.hf.co · 29/11/2024

🚀 We’re excited to announce Argilla v2.5.0, which includes: * Argilla webhooks, * A new design for the datasets home page. * Python 3.13 and Pydantic v2 support. 📙 Read here 👇 the full release notes github.com/argilla-io/a...

github.com

Release v2.5.0 · argilla-io/argilla

🔆 Release highlights Webhooks You can now create and manage webhooks to support your workflows! Webhooks allow you to submit real-time information to other applications whenever a specific event oc...

0172

Reposted by Moritz Laurer

Real Python @realpython.com · 24/11/2024

🐍📰 Hugging Face Transformers: Leverage Open-Source AI in Python #python realpython.com/huggingface-...

0142