At 3 bit quant, but I guess better than nothing…
@unsloth.aiYou can now run Inkling-Small, a new 276B model by Thinking Machines.
Inkling-Small is the strongest open model for its size and runs local on 128GB RAM.
Apache-2.0 Licensed, it has image, audio + 1M context support.
Guide: unsloth.ai/docs/models/...
GGUF: huggingface.co/unsloth/Inkl...
