So cool! transformers can now run GGUF models efficiently.
To bring the performance close to llama.cpp, it uses the underlying ggml kernels through our kernels library.
Awesome work by @marcsun.bsky.social et al.!
huggingface.co/blog/transfo...
huggingface.co
Transformers now runs llama.cpp quants
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
![Comic. Lionel Messi: World Cup Goals per game. [Graph with x axis as year and y axis as number of goals per game, ranging from 0 to 3. Data points: 2006 (~1/3), 2010 (0), 2014 (~1/2), 2018 (~1/4), 2022 (1), 2026 (3) with parenthetical (so far). Behind the line connecting the points is a grayed out curve showing the trajectory.] [caption] At this rate, by 2040 Messi will be scoring hundreds of goals per game.](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:cz73r7iyiqn26upot4jtjdhk/bafkreigq2lsyt6ad4so43vtssddjw4mvo2i4i54uf4fqves6yk3jiudko4)
