Neelesh Salian @neelesh.bsky.social · 15/03/2025Some open-source Spark drama if you want some lists.apache.org/thread/0558x...lists.apache.org 020
Neelesh Salian @neelesh.bsky.social · 13/02/2025Mentally I’m still at JDK 8 and 11. 23! Image borrowed from LinkedIn 070
Neelesh Salian @neelesh.bsky.social · 04/02/2025What would it take for people on LinkedIn to stop posting about Deepseek? I think we should shut it down for a few days and see if people care about it. Most are chasing the trend and spewing random posts. 030
Neelesh Salian @neelesh.bsky.social · 30/01/2025How would this integrate? Perhaps enhancing the Dynamic tables? 140
Neelesh Salian @neelesh.bsky.social · 30/01/2025When they invite the Engineer to present to the C Suite 120
Neelesh Salian @neelesh.bsky.social · 27/01/2025It’s telling how AI as an industry is progressing when you see the people freaking out about Deepseek are mostly VCs. 030
Neelesh Salian @neelesh.bsky.social · 25/01/2025Found this comprehensive guide to AI and how to make sense of the various concepts. www.leverage.to/learn/dev/ai...leverage.toThe 6 AI Engineering Patterns In 2025Learning how to build AI powered products 010
Neelesh Salian @neelesh.bsky.social · 17/01/2025Tomorrow is restart your Apple TV+ subscription for Severance Season 2 day. Also, maybe catch Silo. And re watch Slow Horses. It’s going to be a long work day sigh. 130
Neelesh Salian @neelesh.bsky.social · 16/01/2025Onehouse with their Compute Runtime across table formats. It seems like a good direction but I’m curious if those benchmarks are accurate. 30x query perf, 10x faster writes, I wish there was a demo or something to showcase these. www.onehouse.ai/blog/introdu...onehouse.aiIntroducing Onehouse Compute Runtime to Accelerate Lakehouse Workloads Across All EnginesLearn how Onehouse Compute Runtime delivers an independent, universal foundation to accelerate queries 2-30x and reduce cloud infrastructure bills 20-80% with adaptive workload optimizations, serverle... 220
Neelesh Salian @neelesh.bsky.social · 15/01/2025Tobiko acquired Quary: tobikodata.com/tobiko-acqui... The transformation game is back on againtobikodata.comTobiko - Tobiko Acquires Quary: Evolving the Gold Standard for Data Engineers 010
Neelesh Salian @neelesh.bsky.social · 14/01/2025To whomsoever it may concern: Acquisitions are great. Integrations are hard. Good luck. 0120
Neelesh Salian @neelesh.bsky.social · 13/01/2025Worth reading this blog about Arrow: arrow.apache.org/blog/2025/01...arrow.apache.orgHow the Apache Arrow Format Accelerates Query Result TransferArrow speeds up query result transfer by slashing (de)serialization overheads. We outline five key attributes of the Arrow format that enable this. 020
Neelesh Salian @neelesh.bsky.social · 11/01/2025How AI is messing up things across the board Also, some things could be a Slack message than an email 110
Neelesh Salian @neelesh.bsky.social · 11/01/2025Anytime I open LinkedIn it’s either - Random person with a take on AI, usually hyping - Someone talking about random topics like why is Test Driven Development - At least one reference to Clean Code I can’t imagine how that platform is useful to anyone who wants to get an idea of the industry. 030
Neelesh Salian @neelesh.bsky.social · 10/01/2025Most companies need a better backend API than anything close to AI or LLMs. Just do the basic thing right first. You don’t need to jump to the fancy objects. 040
Neelesh Salian @neelesh.bsky.social · 02/01/20252025 is the year of realizing that hourly batch or micro batch is sufficient over continuous processing. The trick is to ask what your stakeholders actually expect to happen and then build accordingly. 040
Neelesh Salian @neelesh.bsky.social · 01/01/2025Happy New Notebook Day. My first MUJI notebook. The pen has been great! Got these on my trip to Seoul. 130
Neelesh Salian @neelesh.bsky.social · 29/12/2024My 2025 New Year’s Resolution is to tell everyone in the Data and AI space that you can’t build good AI without quality data. Focus on fundamentals rather than fancy. 061
Neelesh Salian @neelesh.bsky.social · 16/12/2024Looks like PySpark, Ray, and Dask have a competitor - Bodo. Has anyone tried it yet? This is the claimed execution time. github.com/bodo-ai/Bodo 060
Neelesh Salian @neelesh.bsky.social · 12/12/2024I'd be surprised if it gets practical usage in the next 10 years. 210
Neelesh Salian @neelesh.bsky.social · 10/12/2024I find it strange that people still think Hudi and Delta have a chance despite the fact that Iceberg has clearly won the table format race. If all major vendors and platforms are putting their bets on one format, why wouldn't an org move to it or stay in it? 110
Neelesh Salian @neelesh.bsky.social · 10/12/2024And just like that, we all stopped doing Advent of Code this year. 020
Neelesh Salian @neelesh.bsky.social · 06/12/2024The most important task to do after going home from a conference: Unsubscribe to all the vendor emails that you are getting because you scanned your badge for socks. 030
Neelesh Salian @neelesh.bsky.social · 05/12/2024There’s Generative AI and then there Adversarial AI. Anyone have a good source / reference to understand and grasp the latter? #AWSreInvent 010
Neelesh Salian @neelesh.bsky.social · 05/12/2024This article is a really good summary and detail about S3 Tables. dataengineeringcentral.substack.com/p/aws-s3-tab...dataengineeringcentral.substack.com 042
Neelesh Salian @neelesh.bsky.social · 03/12/2024More reason to bank on Iceberg. AWS now has S3 tables which are geared specifically for Iceberg tables. If I can read this correctly, it looks like an integrated Rest API catalog behind the scenes. Can integrate well with Glue too. aws.amazon.com/blogs/aws/ne...aws.amazon.comNew Amazon S3 Tables: Storage optimized for analytics workloads | Amazon Web ServicesAmazon S3 Tables optimize tabular data storage (like transactions and sensor readings) in Apache Iceberg, enabling high-performance, low-cost queries using Athena, EMR, and Spark. 050
Neelesh Salian @neelesh.bsky.social · 03/12/2024Walk metrics from Day 1 at AWS ReInvent. Most of this was expo hall walking and to and fro between hotels. #AWSreInvent2024 000
Neelesh Salian @neelesh.bsky.social · 19/11/2024Remember, if your Engineering team isn't fixing that bug in this week, it's a 2025 problem. Holidays are upon us. 120
Neelesh Salian @neelesh.bsky.social · 15/11/2024I am curious how the Databricks IPO will look like and its impact on the data industry as a whole 100
Neelesh Salian @neelesh.bsky.social · 12/11/2024AI is not replacing people. People can't even sell it to other people without, more people. 140
Neelesh Salian @neelesh.bsky.social · 11/11/20242025 will be the year of batch jobs. Just like 2024, 23, …. 020
Neelesh Salian @neelesh.bsky.social · 09/11/2024If there’s one language that could dethrone Python, it would be Golang, imo. Ship a go binary in a docker versus a Py library and tell me you see it. But, existing investment in developing Python libraries and tooling and the time it might take to build it in another language are large barriers. 440
Neelesh Salian @neelesh.bsky.social · 08/11/2024Is anyone else in Data BS attending AWS reInvent next month? I'm headed there for the first time. Mostly talks and other events. 220
Neelesh Salian @neelesh.bsky.social · 08/11/2024Don’t know how true this is but a rumor is that Databricks will IPO Q1 2025. Anyone know more? 210
Neelesh Salian @neelesh.bsky.social · 08/11/2024Are data companies that raised capital during ZIRP doing ok? I don’t think most would justify the valuation that they raised at. Either lost valuation multiples or shut down. Possibly acquired. Any one know of cases? 440
Neelesh Salian @neelesh.bsky.social · 08/11/2024Data Contracts, Data Mesh, Data Fabric still a thing or we’ve moved on? 310
Neelesh Salian @neelesh.bsky.social · 06/11/2024The feeling you are going to feel for a bit is like when you wanted Daenerys to get the throne but they ultimately made Bran king. 020
Neelesh Salian @neelesh.bsky.social · 06/11/2024Is there a single source of truth for election data? Can I download a bunch of csvs somewhere? 000
Neelesh Salian @neelesh.bsky.social · 06/11/2024The real winner tonight are the dashboards we’ve been staring at collectively as a planet 260
Neelesh Salian @neelesh.bsky.social · 05/11/2024We ran 80000 simulations today. Took a few seconds since ran it on DuckDB 050
