Definitely was a community effort - shout out all the reviewers and folks who helped with the implementation. TL;DR PEXT instructions allow for really fast bit manipulations on x86 systems with a scalar fallback. Plenty of additional optimizations to be made: github.com/apache/arrow... :D
github.com
Improve performance of `filter` kernel (when `pext` instruction is not available) · Issue #11213 · apache/arrow-rs
Is your feature request related to a problem or challenge? @devanbenz / @Rich-T-kid and others updated the filter kernel to be more efficient #10136 If the CPU has the pext instruction, the compres...