Sign in

Vini

@vinimdocarmo.com
85 followers 464 following 55 posts

Having fun with database internals, distributed systems and trying to document my joy along the way 👋 Software Engineer @ WeTransfer

PostsRepliesMedia
Vini @vinimdocarmo.com · 24/03/2025
Well, it's now backed by S3 and I've made the code public for anyone interested: github.com/vinimdocarmo... 🦆
021
Vini @vinimdocarmo.com · 24/03/2025
Went down on a storage rabbit role in the past weeks 😁 github.com/vinimdocarmo...
github.com
GitHub - vinimdocarmo/quackfs: FUSE-based DuckDB file system 🦆
FUSE-based DuckDB file system 🦆. Contribute to vinimdocarmo/quackfs development by creating an account on GitHub.
020
Vini @vinimdocarmo.com · 19/03/2025
It is indeed! I’d only warn on the speed of the free machine types. Pretty slow depending on your use case. But I guess most workshops have small scopes anyways. Even better, if you setup your project using devcontainers people will be able to easily setup it up locally using any supported editor ✨
000
Vini @vinimdocarmo.com · 18/03/2025
What if you have an array and you want to transform it into a max-heap? Here you go:
000
Vini @vinimdocarmo.com · 13/03/2025
Can do some basic @duckdb.org operations now 🤩 🦆
010
Vini @vinimdocarmo.com · 06/03/2025
Here's the AI generated implementation by the way: gist.github.com/vinimdocarmo...
000
Vini @vinimdocarmo.com · 06/03/2025
That's usually how I like to use AI for coding: to do things I don't want to do myself. Letting it do (i.e. to try to) everything is really unrewarding for me 🫠 I keep the resiliency high and still try and experiment with LLMs quite often though
110
Vini @vinimdocarmo.com · 06/03/2025
I was solving a DP LeetCode problem in Python and I really wanted to visualize my decision tree structure in the terminal. I'm not a Python expert nor I wanted to waste my energy implementing a "pretty print" function. Copilot was disabled, but I turned it on and did what one must (ask the AI):
110
Vini @vinimdocarmo.com · 28/02/2025
Had some progress today! I could write to a “virtual” file backed by Postgres. Since I’m using FUSE to simulate my file system ops, the file content is being persisted into Postgres instead of my OS file system 🎉
210
Vini @vinimdocarmo.com · 28/02/2025
I mind less when it gives the precise, correct answer. But it's even more frustrating when it just spits nonsense.
000
Vini @vinimdocarmo.com · 28/02/2025
My attention span is pretty short and every time Copilot takes much time to "think" I lose interest. I'm not finding working with AI much rewarding when it comes to coding. Maybe I should do some changes on my workflow? Not sure. I'll keep trying.
100
Vini @vinimdocarmo.com · 28/02/2025
I didn't want to setup my Mac machine to make FUSE work, so I've decided to setup a dev container for VS Code and that's been giving me some headaches. Almost finished though. I only need to finish setting up Postgres 🤞
100
Vini @vinimdocarmo.com · 28/02/2025
Today is Play Day for me and I've started building a Differential Storage using GoLang, something similar to what @motherduck.com does under the hood: motherduck.com/blog/differe...
motherduck.com
Differential Storage: A Key Building Block For A DuckDB-Based Data Warehouse - MotherDuck Blog
Differential Storage: A Key Building Block For A DuckDB-Based Data Warehouse | Reading time: 11 min read
120
Reposted by Vini
Kay Lack @neoeno.bsky.social · 15/01/2025
new video! this datatype → [1, 2, 3] ← is a bit of a marvel. it can store items of any size, grow to any length, rarely causes performance issues, and almost every modern language has one. what is it, really? and how does it work?
1203
Vini @vinimdocarmo.com · 13/02/2025
I see. The reason I asked it was because we tried dedup in a block-level at WeTransfer, but it didn’t really yield good results since the data set is usually unrelated.
100
Vini @vinimdocarmo.com · 13/02/2025
Great article! I see how dedup can help in a repo-level. But does it yield similar benefits between different, unrelated repos?
110
Vini @vinimdocarmo.com · 06/02/2025
Even though the position and company was a great fit, I put my and my partner’s needs and long-term goals in first place. I’m relieved but at the same time I feel a bit of grief.
000
Vini @vinimdocarmo.com · 06/02/2025
I passed the interviews, but unfortunately they couldn’t hire me in Amsterdam anymore due to policy changes. They still tried and I was aware London would be the second option. The offer turned out to be London-based and after much deliberation with my partner we decided to stay in Amsterdam.
100
Vini @vinimdocarmo.com · 23/01/2025
No worries! Hope you're having fun 🛝
010
Vini @vinimdocarmo.com · 23/01/2025
Care to share a bit more of what you're working on? 👐
110
Vini @vinimdocarmo.com · 23/01/2025
I'm at this point learning more about that and thinking about implementing a naive custom allocator in C++
010
Vini @vinimdocarmo.com · 07/01/2025
I’ve passed to final stage: a 30min call with an Executive. That’s one stage further compared to last time, so that’s a win already 🥳 Godspeed 🤞
100
Vini @vinimdocarmo.com · 01/01/2025
Today I saw this expression for the first time while learning C++ `std::this_thread::sleep_for(200ms);`. I was suspicious. How is `200ms` defined? Is that dynamic? What?! 💡 TIL: you can define your own literals en.cppreference.com/w/cpp/langua...
en.cppreference.com
User-defined literals (since C++11) - cppreference.com
020
Reposted by Vini
Loris Cro ⚡ @kristoff.it · 27/12/2024
Ghostty is out of private beta! #ziglang ghostty.org
ghostty.org
Ghostty
Ghostty is a fast, feature-rich, and cross-platform terminal emulator that uses platform-native UI and GPU acceleration.
06412
Vini @vinimdocarmo.com · 25/12/2024
That’s how I best learn things 🎨
Doodle drawn by hand explaining pipes in Linux
020
Vini @vinimdocarmo.com · 17/12/2024
Well, we didn't spend time confirming that iCloud Drive was loosing files, so that's still an assumption. But getting that folder out of iCloud Drive sync stopped the issue 🤷‍♂️
000
Vini @vinimdocarmo.com · 17/12/2024
He uses a MacBook and his projects folder was being synced with iCloud Drive. So every time any file was changed, iCloud Drive sync'ed them with remote. Problem is, `.git` and `node_modules` folders change quite often and iCloud Drive started to loose some files between changes. What a nightmare!
100
Vini @vinimdocarmo.com · 17/12/2024
It was a simple React app, but often times the git history and remote config would just go crazy. npm start would also just stop working with some weird timeout errors reading files. It was like this for weeks, until I noticed something funny...
100
Vini @vinimdocarmo.com · 17/12/2024
I'm mentoring a good friend of mine - which also happened to be my manager and CEO of a startup I worked for - and we experienced some real, weird bugs using node modules and git locally! I've never experienced those myself, so it was really hard to find out what was going on 👻
120
Vini @vinimdocarmo.com · 16/12/2024
I was way more in ease in the second interview process. The first time I created so much expectations and it led to big frustration in the end. I'm glad I didn't give up though. Regardless of how it'll pan out, I've learned a lot!
100
Vini @vinimdocarmo.com · 16/12/2024
4 days later I saw another interesting position for another unrelated team. Applied again, joined the process and last week I did the last round of interviews. I'm now waiting on their final decision. Hope it's a better one this time 🤞
110
Vini @vinimdocarmo.com · 16/12/2024
In October I interviewed for a big tech company. I passed all the interview stages, but in the end they decided to hire someone else with more experience. That sucked big time
100
Vini @vinimdocarmo.com · 16/12/2024
max-heap* 🙃
000
Vini @vinimdocarmo.com · 05/12/2024
A @duckdb.org Conference in Amsterdam?! I'm definitely going 🦆🌷 duckdb.org/events/2025/...
duckdb.org
DuckCon #6 in Amsterdam
DuckDB is an in-process SQL database management system focused on analytical query processing. It is designed to be easy to install and easy to use. DuckDB has no external dependencies. DuckDB has bin...
000
Vini @vinimdocarmo.com · 03/12/2024
What if you have an array and you want to transform it into a max-head? I’ll soon share that too!
110
Vini @vinimdocarmo.com · 20/11/2024
Nice! For those using ClickHouse and want to read all rows from all the parquet files at once, the following query will give you a glimpse on how to achieve that (see the usage of /* at the end of the S3 path):
040
Vini @vinimdocarmo.com · 20/11/2024
Indeed. At wetransfer.com we also use HTTP rage requests when delivering the zip file download (containing all the user's transfer files). This helps the browser's download clients to retry the requests if needed without restarting it over.
010
Vini @vinimdocarmo.com · 15/11/2024
It sorts employees by department, then salary, then hire date within each department
000
Vini @vinimdocarmo.com · 15/11/2024
TIL: window functions work in ORDER BY! Pretty useful for complex sorting. Check this out
100
Vini @vinimdocarmo.com · 15/11/2024
We got a dog 💜
000
Vini @vinimdocarmo.com · 15/11/2024
I've just learned that pgai 🐘 (vectorizer) by TimescaleDB implements its queue using a db table. Blog post here: www.timescale.com/blog/how-we-...
timescale.com
How We Designed a Resilient Vector Embedding Creation System for PostgreSQL Data
Learn the design decisions and trade-offs behind our system for creating and storing vector embeddings for data in PostgreSQL, PgVectorizer.
000
Vini @vinimdocarmo.com · 15/11/2024
🙋‍♂️
010
Vini @vinimdocarmo.com · 15/11/2024
Ultimately my goal is to work on database internals, but I know I don't have previous work experience to show to those companies. In spite of that I'll be focusing some of my time to close that gap 💪
010
Vini @vinimdocarmo.com · 15/11/2024
In various projects I had to go a bit lower down the stack to help us improve our storage workloads. I've always liked databases, but now I'm just loving it to a point where I decided to move my career to this area of expertise.
110
Vini @vinimdocarmo.com · 15/11/2024
4 years ago I decided to focus my career on backend development instead of fullstack development. While working at WeTransfer I'm made my name when it comes to databases (specially MySQL which is what we use).
110
Vini @vinimdocarmo.com · 10/11/2024
Interesting. Do you have any initial ideas of how could that look like?
100
Vini @vinimdocarmo.com · 06/11/2024
What about DeleteObjects? I was under the impression it was also free and you can pass up to 1k objects to be deleted. docs.aws.amazon.com/AmazonS3/lat...
200
Vini @vinimdocarmo.com · 05/11/2024
I love databases.
010
Vini @vinimdocarmo.com · 03/11/2024
If a node is breaking the heap property push it down by swapping places with smaller children recursively.
Personal notes explaining how to maintain the max-heap property.
100
Vini @vinimdocarmo.com · 03/11/2024
Another use case I find valuable: use it to explain things to my teammates. I plug it to Mac, mirror screen to QuickTime app and share its window on the meeting
000