iiinova — where your digital self keeps evolving

Anders @dataders.bsky.social · 01/04/2025

new rule: talk about Iceberg without mentioning Hive or ACID properties. the vast majority of SQL users don't care (and shouldn't)! it's like explaining Dropbox starting w/ bringing up libfuse If you're new to iceberg lmk if you get something from this! roundup.getdbt.com/p/iceberg-gi... #databs

roundup.getdbt.com

Iceberg?? Give it a REST!

The new abstraction that changes nothing... and everything

092

Anders @dataders.bsky.social · 24/03/2025

lots of juicy looking papers in the agenda for EDBT/ICDT 2025 happening this year in Barcelona! definitely going to be diving in more later today edbticdt2025.upc.edu?contents=det... #databs #edbc2025 #icdt2025

edbticdt2025.upc.edu

EDBT/ICDT 2025 Joint Conference - 25th March - 28th March, 2025 - Barcelona, Spain

020

Anders @dataders.bsky.social · 17/03/2025

this is the clearest case for Arrow that I've ever seen. I love that it's high-level but also doesn't shy away from details when it's important. this should be a #databs canon text imho. thanks @ianmcook.bsky.social !

181

Anders @dataders.bsky.social · 13/03/2025

TIL about relationalplayground.com makes relational algebra accessible to a SQL monkey like me. much better than in a dry textbook where it's normally found. #databs

relationalplayground.com

Relational Playground

An exploration of relational algebra. Compare SQL queries with relational algebra expressions along with intermediate results.

040

Anders @dataders.bsky.social · 03/03/2025

imho, the most clear case Databricks has made in public on their Iceberg future post-Tabular acquisition. I agree with this vision of the future and it's nice to see DBRX sharing how they see themselves participating in it. worth clicking through the deck! #databs speakerdeck.com/databricksja...

speakerdeck.com

Iceberg Meetup Japan #1 : Iceberg and Databricks

2月21日に開催されたIceberg Meetup #1で使用した資料になります。 DatabricksとIcebergを使用する際のカタログについてご紹介しています。

030

Anders @dataders.bsky.social · 03/03/2025

1) my top-level takeaways on DeepSeek's smallpond: a distributed data processing framework used for training LLMs #databs

182

Reposted by Anders

Internal Tech Emails @techemails.bsky.social · 08/02/2025

Mark Zuckerberg messages Facebook engineer April 5, 2012

Mark Zuckerberg
Around?

Facebook engineer
Yeah

Mark Zuckerberg
If you could buy one of either Instagram, Foursquare or Pinterest, which would you buy?

1776

Reposted by Anders

Bijil Subhash @bijilsubhash.bsky.social · 01/02/2025

Just finished watching the webinar on introducing SDF by dbt team. After seeing SDF in action, I have to admit that I am really looking forward to the future of dbt engine. I was wondering when dbt was going to bring in notable changes to the developer experience and this might be it. #databs

031

Reposted by Anders

Sung Kim @sungkim.bsky.social · 26/01/2025

Building Query Compilers by Guido Moerkotte (695 pages) Note: This is repost of @emresevinc.bsky.social's on X Link: pi3.informatik.uni-mannheim.de/~moer/queryc...

061

Anders @dataders.bsky.social · 24/01/2025

post two! The key technologies behind SQL Comprehension #databs, do not fear compiler concepts -- embrace them and the new world order they enable for us! read the great blog (& pretty diagrams!) @daveconnors3.bsky.social, truly a masterpiece docs.getdbt.com/blog/sql-com...

Bernie Sanders meme labelled with Dave Connors's name: I am once again asking you to move up the stack and learn how database compilers work

041

Anders @dataders.bsky.social · 23/01/2025

post one of a new series kicking off today whose larger thrust is effectively: understanding SQL, not just a job for the database! Post 1 lays out what the levels of understanding docs.getdbt.com/blog/the-lev... #databs

141

Anders @dataders.bsky.social · 04/01/2025

Look forward to these every year. Always love Andy's candor and insight on our little corner of the world. #databs

060

Anders @dataders.bsky.social · 27/12/2024

#ghostty dropping today is the GitHub equivalent of a Beyoncé album. My whole feed is everyone following it. Now I gotta at least try it right? github.com/ghostty-org/...

040

Anders @dataders.bsky.social · 19/12/2024

my experience installing MSFT's ODBC driver on M-series macbook re-stokes my long-burning ire for Simba drivers and the company behind them. One might think MSFT is to blame for a less than medoicre database driver, but in a way they're victims here too, held captive by Simba. #databs

020

Anders @dataders.bsky.social · 18/12/2024

does anyone use the 1Password CLI? I feel it has potential to be very valuable for managing environment variables (esp for teams). but i'm hesitant to adopt bc it requires that every command you run consume the output of `op` developer.1password.com/docs/cli/get...

developer.1password.com

Get started with 1Password CLI | 1Password Developer

Learn how to install and sign in to 1Password CLI, then get started with commands and scripts to manage users, vaults, and items on the command line.

210

Anders @dataders.bsky.social · 17/12/2024

admittedly, I'm triggered by SFTP, but it's a good analogy! What if there was an SFTP where you didn't have to parse random-delimited text files, but could just query the tables inside?

010

Anders @dataders.bsky.social · 09/12/2024

art history major = business stakeholder? 😜 #databs

I must study anaytics eng, so that our sons have liberty to study data analytics and reporting. Our sons ought to study data analytics, reporting and solving problems for the business in order to give their children a right to say: "hey -- quick question"

040

Reposted by Anders

Kit Menke @kitmenke.com · 06/12/2024

Great breakdown of the new S3 Tables feature that leverages Apache Iceberg. Including an explanation of the costs... which are complicated. #dataBS bigdata.2minutestreaming.com/p/meet-your-...

bigdata.2minutestreaming.com

meet your new data lakehouse: S3 Iceberg Tables

S3 Tables and S3 Metadata are two brutal new features that compete with common Apache Iceberg Lakehouse architectures

061

Reposted by Anders

Nikhil Benesch @benesch.bsky.social · 04/12/2024

S3 (Iceberg) Tables is everything I dreamt of, and more. I blogged some long-form thoughts: meltware.com/2024/12/04/s... I think we're about to see an explosion of data tools (@materialize.com, @clickhouse.com, @duckdb.org, et al.) learn to write Iceberg tables via S3 table buckets. #databs

meltware.com

A First Look at S3 (Iceberg) Tables

AWS announced S3 Tables today, which brings native support for Apache Iceberg to S3. It’s hard to overstate how exciting this is for the data analytics ecosystem. This post is a quick rundown of my th...

1510637

Reposted by Anders

Camille Fournier @skamille.themanagerswrath.com · 26/11/2024

Need a holiday gift for the nerd in your life? How about a copy of my new book, Platform Engineering: A Guide for Technical, Product, and People Leaders! amzn.to/4eUz5zB

amzn.to

Platform Engineering: A Guide for Technical, Product, and People Leaders

Amazon.com: Platform Engineering: A Guide for Technical, Product, and People Leaders eBook : Fournier, Camille, Nowland, Ian: Kindle Store

45915

Anders @dataders.bsky.social · 26/11/2024

anyone have a good term to collectively refer to the data "languages" like #SQL, #polars, #pandas, #dplyr, #malloy, #preql #numpy etc? "data transformation APIs" (DTAPIs) is the best I've got so far but I'm dreaming of a better word #databs

9171

Reposted by Anders

Hasan Geren @hgeren.bsky.social · 19/11/2024

Hey #dataBS and #datasky folks, Our new post about "how understanding Big O Notation & Execution Plans can optimize SQL queries" has just been posted. Check it out if you're interested, and we'd love to hear your thoughts! @hopefanhe.bsky.social open.substack.com/pub/pipeline...

open.substack.com

SQL Behind the Curtain: How Are Queries Executed?

Explore the journey of your SQL query guided by execution plans

182

Reposted by Anders

SofritoGuy 🇵🇷 @sofritoguy.com · 19/11/2024

Oh, we just full table scanning huh #databs

162

Reposted by Anders

Mike Driscoll @medriscoll.com · 14/11/2024

Welcome to #dataBS the creator of Apache Calcite & Mondrian, architect @ Looker, mentor to many @ ASF, and wizard of SQL: @julianhyde.bsky.social One of Julian’s most popular recent initiatives is bringing the metrics layer to SQL: hydromatic.net/hyde2024meas...

hydromatic.net

1276

Anders @dataders.bsky.social · 12/11/2024

must read, imho. high-level and clear on the problems solved. but also not a free lunch! "the creation of Iceberg at Netflix['s goal was] to define a better table abstraction on top of blob storage. […] Suddenly, many operations become a lot easier. […] Yes, we also re-invented the DBA role."

071

Anders @dataders.bsky.social · 11/11/2024

parquet's not good enough?! two new file formats: BtrBlocks -- claims better performance on cloud object store. Vortex -- claims order(s) of magnitude better scans and reads than parquet. conflicted bc file formats are stickier than table formats, so hard to forsee uptake but... novelty! #dataBS

100

Anders @dataders.bsky.social · 08/11/2024

early days still for #iceberg, but you can't ignore a PR that shows me the end of the table format wars is nigh. Delta-only deletion vectors to land in v3 spec Incredible to see SWEs from DBRX, SNOW, GCP, and more chime in. Nature is healing. Iceberg is real! #databs github.com/apache/icebe...

github.com

Puffin: Add delete-vector-v1 blob type by rdblue · Pull Request #11238 · apache/iceberg

This adds a blob type to the Puffin spec that can store a Roaring bitmap delete vector. This is in support of the row-level delete improvements proposed for Iceberg v3.

0183

Anders @dataders.bsky.social · 04/11/2024

I'm so jazzed on my team's operating model that I don't think I could ever not work like this. check out @jasnonaz.bsky.social's write up on how it works at a high-level?

010

Anders @dataders.bsky.social · 31/10/2024

wish: if we take as inevitable that LLMs take the throne of ads monetization from search engines. I propose that SEO be supplanted by: RAGU (RAG ¿utilization?) honestly idc what the word is as long as it starts with U

130

Reposted by Anders

RJ Andrews @infowetrust.com · 30/10/2024

THE SIMPSONS PARADOX

524883