gab-menezes.github.io
Using the most unhinged AVX-512 instruction to make the fastest phrase search algo
Disclaimers before we start For those who don’t want to read/don’t care that much, here are the results. I hope after seeing them you are compelled to read. TL;DR: I wrote a super fast phrase search algorithm using AVX-512 and achieved wins up to 1600x the performance of Meilisearch. The source code can be found here, and here is the crate. The contents of this blog post are inspired by the wonderful idea of Doug Turnbull from the series of blog posts about Roaringish. Here we will take these ideas to an extreme, from smart algorithms to raw performance optimization. I highly recommend reading the Roaringish blog post, but if you don’t want to, there will be a recap on how it works. This project has been almost 7 months in the making, with thousands and thousands of lines of code written and rewritten, so bear with me if I sound crazy. At the moment of writing, there are almost 2.7k LOC, but I have committed around 17k LOC (let’s take a few thousand because of .lock files) (probably at
