Neelesh Salian @neelesh.bsky.social · 15/03/2025Some open-source Spark drama if you want some lists.apache.org/thread/0558x...lists.apache.org 020
Neelesh Salian @neelesh.bsky.social · 13/02/2025Mentally I’m still at JDK 8 and 11. 23! Image borrowed from LinkedIn 070
Neelesh Salian @neelesh.bsky.social · 04/02/2025What would it take for people on LinkedIn to stop posting about Deepseek? I think we should shut it down for a few days and see if people care about it. Most are chasing the trend and spewing random posts. 030
Neelesh Salian @neelesh.bsky.social · 04/02/2025I learn more about American culture daily. Thanks Bluesky. 000
Neelesh Salian @neelesh.bsky.social · 01/02/2025Why bother being a coffee shop. 6:00 am or 7:00am. Anything after is unacceptable. 010
Neelesh Salian @neelesh.bsky.social · 30/01/2025How would this integrate? Perhaps enhancing the Dynamic tables? 140
Neelesh Salian @neelesh.bsky.social · 30/01/2025When they invite the Engineer to present to the C Suite 120
Neelesh Salian @neelesh.bsky.social · 27/01/2025It’s telling how AI as an industry is progressing when you see the people freaking out about Deepseek are mostly VCs. 030
Neelesh Salian @neelesh.bsky.social · 25/01/2025Found this comprehensive guide to AI and how to make sense of the various concepts. www.leverage.to/learn/dev/ai...leverage.toThe 6 AI Engineering Patterns In 2025Learning how to build AI powered products 010
Neelesh Salian @neelesh.bsky.social · 17/01/2025Tomorrow is restart your Apple TV+ subscription for Severance Season 2 day. Also, maybe catch Silo. And re watch Slow Horses. It’s going to be a long work day sigh. 130
Neelesh Salian @neelesh.bsky.social · 16/01/2025Was the quary move an acqui-hire? I heard it was a couple of engineers a few months ago to help with speed ups and the news was released to coincide with the dbt news. 120
Neelesh Salian @neelesh.bsky.social · 16/01/2025@dipankartnt.bsky.social if I know him well enough he might have something in the works too 010
Neelesh Salian @neelesh.bsky.social · 16/01/2025Onehouse with their Compute Runtime across table formats. It seems like a good direction but I’m curious if those benchmarks are accurate. 30x query perf, 10x faster writes, I wish there was a demo or something to showcase these. www.onehouse.ai/blog/introdu...onehouse.aiIntroducing Onehouse Compute Runtime to Accelerate Lakehouse Workloads Across All EnginesLearn how Onehouse Compute Runtime delivers an independent, universal foundation to accelerate queries 2-30x and reduce cloud infrastructure bills 20-80% with adaptive workload optimizations, serverle... 220
Neelesh Salian @neelesh.bsky.social · 16/01/2025A way to store my data so it has all the info I need and allows me to access it cleanly 130
Neelesh Salian @neelesh.bsky.social · 16/01/2025Makes me want to go back to my Breaking Spark talk and say, "When it doesn't Spark joy" 030
Neelesh Salian @neelesh.bsky.social · 15/01/2025I still use sculpt. Been on it for years. I have the Kinesis advantage but never made the shift. 000
Neelesh Salian @neelesh.bsky.social · 15/01/2025Tobiko acquired Quary: tobikodata.com/tobiko-acqui... The transformation game is back on againtobikodata.comTobiko - Tobiko Acquires Quary: Evolving the Gold Standard for Data Engineers 010
Neelesh Salian @neelesh.bsky.social · 15/01/2025Didn’t read it correct and thought it was about wine 120
Neelesh Salian @neelesh.bsky.social · 15/01/2025Some Cloud vendors barely have much of a moat but tiny VC funded startups think they have a moat because of one project. Funny. 020
Neelesh Salian @neelesh.bsky.social · 14/01/2025Honestly after learning it, Scala isn’t bad. Used to build Spark apps on Scala for a bit. 110
Neelesh Salian @neelesh.bsky.social · 14/01/2025I remember a time when Spark used to ignore developing in Python. All dev was in Scala and Java, Python was an afterthought. That changed as the Python data ecosystem evolved. 120
Neelesh Salian @neelesh.bsky.social · 14/01/2025To whomsoever it may concern: Acquisitions are great. Integrations are hard. Good luck. 0120
Neelesh Salian @neelesh.bsky.social · 14/01/2025So they together form SQLmesh? @captaintobs.bsky.social 010
Neelesh Salian @neelesh.bsky.social · 14/01/2025Scala is dead? I thought it was around due to Spark to some extent. Not everyone is on Pyspark 140
Neelesh Salian @neelesh.bsky.social · 14/01/2025I learned a while ago that it is ok to put down a book. Not every book is meant for you. 010
Neelesh Salian @neelesh.bsky.social · 13/01/2025Worth reading this blog about Arrow: arrow.apache.org/blog/2025/01...arrow.apache.orgHow the Apache Arrow Format Accelerates Query Result TransferArrow speeds up query result transfer by slashing (de)serialization overheads. We outline five key attributes of the Arrow format that enable this. 020
Neelesh Salian @neelesh.bsky.social · 13/01/2025Mostly at home from my desk and maybe move to the dining table or the couch. Separately I head to a nearby Starbucks once every two weeks if I need a change of scenery. 010
Neelesh Salian @neelesh.bsky.social · 12/01/2025This is a good blog on it: www.anthropic.com/research/bui...anthropic.comBuilding effective agentsA post for developers with advice and workflows for building effective AI agents 081
Neelesh Salian @neelesh.bsky.social · 11/01/2025How AI is messing up things across the board Also, some things could be a Slack message than an email 110
Neelesh Salian @neelesh.bsky.social · 11/01/2025Anytime I open LinkedIn it’s either - Random person with a take on AI, usually hyping - Someone talking about random topics like why is Test Driven Development - At least one reference to Clean Code I can’t imagine how that platform is useful to anyone who wants to get an idea of the industry. 030
Neelesh Salian @neelesh.bsky.social · 10/01/2025Most companies need a better backend API than anything close to AI or LLMs. Just do the basic thing right first. You don’t need to jump to the fancy objects. 040
Neelesh Salian @neelesh.bsky.social · 02/01/20252025 is the year of realizing that hourly batch or micro batch is sufficient over continuous processing. The trick is to ask what your stakeholders actually expect to happen and then build accordingly. 040
Neelesh Salian @neelesh.bsky.social · 02/01/2025@dipankartnt.bsky.social should have been in that author list too. But they didn’t do right by him. 010
Neelesh Salian @neelesh.bsky.social · 01/01/2025Have the same one in green. I use it for journaling. Best paper quality in my opinion. 010
Neelesh Salian @neelesh.bsky.social · 01/01/2025Yeah I don’t see the reason for the hate. Redis is pretty solid. It’s not designed to be the way he is describing and that’s ok. 000
Neelesh Salian @neelesh.bsky.social · 01/01/2025Happy New Notebook Day. My first MUJI notebook. The pen has been great! Got these on my trip to Seoul. 130
Neelesh Salian @neelesh.bsky.social · 31/12/2024It's dynamic, and you can rely on Iceberg to do it that by looking at the partition spec. In Hive metastore, it's not automatic, as far as I remember. I had scala code that wrote out those locations (in s3) based on the partitions that Spark would write out. Then update the HMS. 010
