Sign in

Klaus Post

@klauspost.com
194 followers 53 following 131 posts

Making Go stuff go brrrrrrrrrrrrrrrrrrr

PostsRepliesMedia
Klaus Post @klauspost.com · 30/09/2026
Back in the day Ultra Compressor II (DOS) trickled my interest in compression and data recovery as I was learning about computers. With the knowledge I have now, it was fascinating to look through it with an LLM and also quite nostalgic re-creating the UI I only vaguely remember.
110
Klaus Post @klauspost.com · 28/09/2026
zstd decompression speed... (each level decompression speed)
161
Klaus Post @klauspost.com · 28/09/2026
Working on a #golang compression benchmark comparison site. Deflate speed improvements from latest compression library release... (JSON stream)
270
Klaus Post @klauspost.com · 27/09/2026
New #golang compression release with a lot of good vibes.
1102
Klaus Post @klauspost.com · 09/09/2026
My Ad Blocker.. is now showing ads. What a timeline to live in!
010
Klaus Post @klauspost.com · 17/07/2026
Search Compressed Data for byte sequences with selective decompression using MinLZ v1.2.0 - available as cli and #golang API. Overhead is a fractional bit per byte, fully customizable and with fixed specification.
101
Klaus Post @klauspost.com · 01/07/2026
New #golang compression release. Big features include #zstd concurrent stream encoding, arm64 decompression assembly and various minor changes.
What's Changed
zstd: Add true concurrent stream encoding by @klauspost in #1136
zstd: arm64 decoder asm by @lizthegrey in #1160
zstd: avoid unused BuildDict encoder allocation by @snissn in #1147
flate: Add inflate checkpoints by @klauspost in #1154
snappy/s2: Limit length of varint in decodedLen by @eustas in #1148
gzhttp: match qvalue parameter case-insensitively (RFC 7231) by @z9z in #1149
zip: add NameDecoder callback for legacy encoding rewrite by @SAY-5 in #1150
huff0: Allow building tables from histogram. by @klauspost in #1155
huff0: Allow building table from oversized histogram. by @klauspost in #1156
s2sx: Clean symlink targets by @klauspost in #1163
1200
Klaus Post @klauspost.com · 19/05/2026
I must say the ability to test out ideas in less than an hour is so addictive. Also I feel much less "attached" to idea and it is much easier to abandon if it doesn't pan out.
140
Klaus Post @klauspost.com · 07/05/2026
000
Klaus Post @klauspost.com · 20/04/2026
Still amazed me that sometimes you can just stumble across a change that gives 200% speed.
0120
Klaus Post @klauspost.com · 21/03/2026
0120
Klaus Post @klauspost.com · 09/02/2026
New v1.18.4 release of #golang compression library adds zstd compression to gzhttp middleware. Also zstd can now have some options modified on reset. See more at: github.com/klauspost/co...
1100
Klaus Post @klauspost.com · 05/02/2026
Well, hello there follow GenX'er.
010
Klaus Post @klauspost.com · 12/01/2026
New #golang Reed-Solomon release. Contains new progressive field decoding and some significant speedups. Check it out: github.com/klauspost/re...
151
Klaus Post @klauspost.com · 19/12/2025
Vibe Coders will understand.
010
Klaus Post @klauspost.com · 01/12/2025
Compression library for #golang v1.18.2 released. If you are running v1.18.1 (specifically) please upgrade.
021
Klaus Post @klauspost.com · 27/10/2025
Now even more #golang msgp codegen in v1.5.0. Now with generics. Generates code 1000x faster than your LLM! :D Give it a spin: github.com/tinylib/msgp...
000
Klaus Post @klauspost.com · 23/10/2025
Released v1.18.1 of the #golang compression library. You know what to do! Link as always github.com/klauspost/co...
191
Klaus Post @klauspost.com · 22/07/2025
Un-archived github.com/klauspost/cr... repo after getting a bit more than 2x on IEEE CRC32 using AVX512. Will submit upstream to #golang, but if you want to test it out early check out the repo.
Benchmark of IEEE CRC32.
060
Klaus Post @klauspost.com · 14/07/2025
New #golang cpuid release. As always github.com/klauspost/cp...
010
Klaus Post @klauspost.com · 19/06/2025
🙄 #golang
011
Klaus Post @klauspost.com · 03/06/2025
This kinda implies that it is all 32 bit systems?
100
Klaus Post @klauspost.com · 17/05/2025
most are more like this....
000
Klaus Post @klauspost.com · 17/05/2025
Would a #golang MinLZ compressor level with a better "worst case" speed be useful? Overly dramatic example...
110
Klaus Post @klauspost.com · 15/05/2025
Realizing I needed to add a mutex to protect an RNG call, I almost ended up with this.... Need more coffee 🫘
000
Klaus Post @klauspost.com · 02/05/2025
Sent in in. Now I can only 🤞
030
Klaus Post @klauspost.com · 22/04/2025
Now @klauspost.com - wish that my old profile link would redirect to my new one, though. @bsky.app
000
Klaus Post @klauspost.com · 16/04/2025
cmp.Or is so neat for error handling in #golang - just remember that *all* params are evaluated in order, and first error is returned.
// serialize serializes p as json.
func (p *someThing) serialize() ([]byte, error) {
	var dst bytes.Buffer
	comp := minlz.NewWriter(&dst)
	err := cmp.Or(json.NewEncoder(comp).Encode(p), comp.Close())
	return dst.Bytes(), err
}
180
Klaus Post @klauspost.com · 08/04/2025
We need jungle, i'm afraid
110
Klaus Post @klauspost.com · 18/03/2025
Great feedback! I presume you are thinking of this. Right is better compression, so Snappy is worst here. Snappy doesn't have concurrent stream compression, so it falls behind in speed there. TBH it doesn't have too much relevance other than wide compatibility.
100
Klaus Post @klauspost.com · 19/02/2025
New #golang compression package release. Mainly focused on non-assembly performance. Now uses "unsafe" - use "nounsafe" tag to disable. As always here: github.com/klauspost/co...
changelog for package:

Add unsafe little endian loaders #1036
fix: check r.err != nil but return a nil value error err by @alingse in #1028
flate: Simplify L4-6 loading #1043
flate: Simplify matchlen (remove asm) #1045
s2: Improve small block compression speed w/o asm #1048
flate: Fix matchlen L5+L6 #1049
flate: Cleanup & reduce casts #1050
031
Klaus Post @klauspost.com · 07/02/2025
Average shopping experience on @bandcamp.com
010
Klaus Post @klauspost.com · 07/02/2025
😜
030
Klaus Post @klauspost.com · 22/01/2025
Still tweaking...
020
Klaus Post @klauspost.com · 28/12/2024
Teaser for next year :)
IT IS DONE! :tada:
I think the package is now ready for release. Spec is finalized, and implementations are done. The only thing we really need is benchmarking on some beefier HW than my home box.
There are some graphs below that shows it compared to our current compression (and Snappy, LZ4 and my Level 5 deflate), and in the overall tests there are no significant regressions. Horizontal is compression ratio. Vertically compression speed, 16 core CPU.
Next step would be pushing to a repo and make a blog post and do the benchmarks. This will be a solid improvement to on-disk compression, and it provides easy backwards compatibility.
130
Klaus Post @klauspost.com · 26/11/2024
Double negative... Most intuitive example... I *guess* this means `if (s3:if-none-match == null) == false`
            "Condition": {
                "Null": {
                    "s3:if-none-match": "false"
                }
            }
000
Klaus Post @klauspost.com · 25/11/2024
At least I am reasonably good at taking weekends off ;)
100
Klaus Post @klauspost.com · 18/11/2024
Love how javascript games are basically open source (ok, "source available" for the zealots). Let me just add some more coins.....
010
Klaus Post @klauspost.com · 14/11/2024
LOL - ChatGPT... How many mistakes can you spot ;)
000
Klaus Post @klauspost.com · 14/11/2024
This issue feels like a #golang job interview.. So I made it one :D Here if you want a serious challenge (less allocs = better): go.dev/play/p/M-O6r...
type foo struct {
	cond sync.Cond
	mu   sync.Mutex
}

// wait returns true when f.cond is triggered,
// and false if it waited longer than provided timeout.
// When returning, f.mu must always be locked.
func (f *foo) wait(timeout time.Duration) (ok bool) {
	// ...
}
110