Sign in

Arik Friedman

@arikf.net
75 followers 89 following 51 posts

Data Scientist @ Atlassian Spent some time looking into software teams' effectiveness, and how software teams can use data to reflect and improve how they work. These days I'm trying to learn more about Statistical Process Control.

PostsRepliesMedia
Arik Friedman @arikf.net · 30/04/2026
Back in MeasureCamp Sydney I had the opportunity to reflect on lessons I picked up throughout my tenure as a data scientist at Atlassian. This led to a fun discussion at the APH on what it actually means to "get things right" in data analysis.
020
Arik Friedman @arikf.net · 14/10/2025
Oh, that's mean! 🤔
030
Reposted by Arik Friedman
Emily Riederer @emilyriederer.bsky.social · 27/01/2025
I was tagged the other day by someone kindly sharing my Python Rgonomics post and realized that my thesis of "tooling keeps improving" held up too well and some of the recos have changed Hence, here's the 2025 update: www.emilyriederer.com/post/py-rgo-... Now feat uv (vs pyenv, pdm) and Positron
emilyriederer.com
Python Rgonomics - 2025 Update | Emily Riederer
Switching languages is about switching mindsets - not just syntax. New developments in python data science toolings, like polars and seaborn’s object interface, can capture the ‘feel’ that converts fr...
58417
Arik Friedman @arikf.net · 28/12/2024
Not a silly idea. Reminds of this arxiv.org/abs/1102.4374
arxiv.org
Link Prediction by De-anonymization: How We Won the Kaggle Social Network Challenge
This paper describes the winning entry to the IJCNN 2011 Social Network Challenge run by Kaggle.com. The goal of the contest was to promote research on real-world link prediction, and the dataset was ...
120
Arik Friedman @arikf.net · 21/12/2024
Interviewer: can you explain this gap in your resume? Data scientist: it's a confidence interval.
070
Reposted by Arik Friedman
Vicki @vickiboykis.com · 13/12/2024
KNN + topic detection getting a big glow-up www.anthropic.com/research/clio
anthropic.com
Clio: Privacy-preserving insights into real-world AI use
A blog post describing Anthropic’s new system, Clio, for analyzing how people use AI while maintaining their privacy
3519
Arik Friedman @arikf.net · 30/11/2024
Converting SQL code to python equivalent would have been just another route towards spending hours on this 🫠 But yes, if it was only a single cell or a little amount of SQL code I would have just done it all in python and be done with it.
000
Arik Friedman @arikf.net · 30/11/2024
I tried passing it through the spark context with spark.conf.set() but the SQL cell didn't pick it up. Maybe I used the wrong commands?
000
Arik Friedman @arikf.net · 30/11/2024
Spent hours yesterday to get a SQL cell in a databricks job use a variable that was set in a prior python cell. Amazing how something that seems so trivial can be so convoluted. Is that what "business impact" looks like?
010
Arik Friedman @arikf.net · 29/11/2024
Good. Now do the rest of the internet.
010
Arik Friedman @arikf.net · 24/11/2024
I heard that in at least some of the cases it was due to commercial and film rights reasons www.npr.org/2023/06/07/1...
npr.org
Sony and Marvel and the Amazing Spider-Man Films Rights Saga : Planet Money
(Note: This episode originally ran back in 2022.)This past weekend, Spider-Man: Across the Spider-Verse had the second largest domestic opening of 2023, netting (or should we say webbing?) over $120 million in its opening weekend in the U.S. and Canada. But the story leading up to this latest Spider-Man movie has been its own epic saga.When Marvel licensed the Spider-Man film rights to Sony Pictures in the 1990s, the deal made sense — Marvel didn't make movies yet, and their business was mainly about making comic books and toys. Years later, though, the deal would come back to haunt Marvel, and it would start a long tug of war between Sony and Marvel over who should have creative cinematic control of Marvel's most popular superhero. Today, we break down all of the off-screen drama that has become just as entertaining as the movies themselves.This episode was originally produced by Nick Fountain with help from Taylor Washington and Dave Blanchard. It was engineered by Isaac Rodrigues. It was edited by Jess Jiang. The update was produced by Emma Peaslee, with engineering by Maggie Luthar. It was edited by Keith Romer. Help support Planet Money and get bonus episodes by subscribing to Planet Money+ in Apple Podcasts or at plus.npr.org/planetmoney.
020
Arik Friedman @arikf.net · 24/11/2024
Dark Matter is based on a book by Blake Crouch.
110
Reposted by Arik Friedman
Taras Kaduk @taraskaduk.com · 23/11/2024
0224
Reposted by Arik Friedman
Nicolay Gerold @nicolaygerold.com · 22/11/2024
How did I just find this? wiki.postgresql.org/wiki/Don%27t... #databs
wiki.postgresql.org
Don't Do This - PostgreSQL wiki
1326
Arik Friedman @arikf.net · 21/11/2024
Great stuff, and an excuse to recommend the book Storytelling with Data by Cole Nussbaumer Knaflic for anyone who wants to learn more on this.
000
Reposted by Arik Friedman
Arik Friedman @arikf.net · 16/11/2024
One of my favourite pieces on the role of the data analyst comes from @rdpeng.org , who in turn quotes Tukey's "The Future of Data Analysis" from 1962 (!) simplystatistics.org/posts/2019-0... #databs
simplystatistics.org
Simply Statistics: Tukey, Design Thinking, and Better Questions
242
Arik Friedman @arikf.net · 18/11/2024
Definitely different setup to anything I experienced. I think that the aspects of team composition and targeted scope would still play a bigger role. Examples I've seen (www.synq.io/blog/data-te... or towardsdatascience.com/data-to-engi...) indicate a wide spread, so the context likely matters more.
synq.io
Data team as % of workforce: A deep dive into 100 tech scaleups
What’s the right ratio of data roles in scaleups and why it should probably be higher than you think.
000
Arik Friedman @arikf.net · 18/11/2024
Absolutely. Maybe we just have different notions of what data storytelling is? I see data storytelling as the competency of communicating effectively with data. That is different to using narratives as a substitute for data sense making.
010
Arik Friedman @arikf.net · 18/11/2024
Yes, I agree on that point, storytelling can be abused. But I don't think it means it can't be useful or applicable within a process control worldview.
110
Arik Friedman @arikf.net · 18/11/2024
Btw, from what I've seen in Donald Wheeler's books so far, I got the impression he'd classify top-down goal setting and metric tracking (which he calls "voice of the customer") as belonging to the Fantasy genre.
110
Arik Friedman @arikf.net · 18/11/2024
Storytelling is a communication competency. From my exposure to process control so far, its application can help you verify your stories are of the non-fiction variety.
220
Arik Friedman @arikf.net · 18/11/2024
I usually call it "data"
010
Arik Friedman @arikf.net · 17/11/2024
Adversarial and malicious are common terms in computer security.
010
Arik Friedman @arikf.net · 17/11/2024
Have you seen hamel.dev/blog/posts/l... ?
hamel.dev
Creating a LLM-as-a-Judge That Drives Business Results –
A step-by-step guide with my learnings from 30+ AI implementations.
1130
Reposted by Arik Friedman
Tim but Spooky @timburga.com · 16/11/2024
one of the many things to like about atproto is that everything is authenticated and public you don't have to trust bluesky, you can read from the relay yourself it's not an API... more like a spinal tap into the central feed and this is an amazing example of what you can do with that power
skyzoo.blue
Sky Zoo
Stats on Bluesky, At Protocol, ...
0297
Arik Friedman @arikf.net · 17/11/2024
Another critical factor is their remit. Focusing attention on a few areas that can be served well is more sustainable than trying to spread thin a few data scientists to cover as much ground as possible. My experience was that being embedded in a team gives more leverage than providing a service.
100
Arik Friedman @arikf.net · 17/11/2024
Reference numbers can always provide some extra context, but I've seen other factors can play a much more critical role. For example, how much support does DS get from other functions (data platform eng, data engineers, ...) - this can have a huge impact on how data scientists spend their time.
100
Arik Friedman @arikf.net · 17/11/2024
What do you mean by data team? Data scientists? Data platform engineers? Data engineers? Machine learning engineers? What's the org context you're referring to? Startup? Corporate? I have some considerations in mind from a data scientist perspective, but I'm an IC, so take this with a grain of salt.
100
Arik Friedman @arikf.net · 16/11/2024
Should we pay attention to how AI is changing our work? Sure! But staying on top of changes and evolutions in the field is part of the job, and AI is only one of many things driving this constant shift. bsky.app/profile/apre...
010
Arik Friedman @arikf.net · 16/11/2024
As data scientists, our greatest impact isn't in delivering more precise answers - it's in improving the quality of the questions being asked. That's why I'm not worried about AI "taking our jobs," though it can definitely help us do them better.
The image shows a graph plotting "Quality of question" vs "Strength of evidence." It features three paths: an ideal but unrealistic path starting from high quality question, a path showing statistical/ML progression from a poor question to a poor question with a precise answer, and the "job of the data scientist" path that progresses along both question quality and strength of evidence.
120
Arik Friedman @arikf.net · 16/11/2024
One of my favourite pieces on the role of the data analyst comes from @rdpeng.org , who in turn quotes Tukey's "The Future of Data Analysis" from 1962 (!) simplystatistics.org/posts/2019-0... #databs
simplystatistics.org
Simply Statistics: Tukey, Design Thinking, and Better Questions
242
Reposted by Arik Friedman
Bluesky @bsky.app · 15/11/2024
A number of artists and creators have made their home on Bluesky, and we hear their concerns with other platforms training on their data. We do not use any of your content to train generative AI, and have no intention of doing so.
95410231927056
Arik Friedman @arikf.net · 15/11/2024
I have a warm spot for Shadow of the day, it was the first song my drum teacher gave me. But What I've Done is my favourite.
000
Reposted by Arik Friedman
Vicki @vickiboykis.com · 15/11/2024
This is probably the one of the clearest explanations I’ve seen of zero-shot, etc arxiv.org/abs/2309.024...
Chart of zero-shot prompting, RAG, etc
411616
Arik Friedman @arikf.net · 12/11/2024
Can confirm
000
Arik Friedman @arikf.net · 10/11/2024
Nice! One of my favourite examples is the Datasaurus Dozen r-causal.github.io/quartets/
r-causal.github.io
Datasets to Help Teach Statistics
In the spirit of Anscombes quartet, this package includes datasets that demonstrate the importance of visualizing your data, the importance of not relying on statistical summary measures alone,...
110
Arik Friedman @arikf.net · 09/11/2024
tbh I don't know 😂 I usually try to scroll past and not pay attention to anything that resembles an ad. I need to unlearn this habit now.
010
Arik Friedman @arikf.net · 09/11/2024
I noticed my app habits changed over the last couple of weeks. I now head to bluesky for #dataBS, and head to X when I want to see temu ads.
2431
Reposted by Arik Friedman
Taylor Murphy @tayloramurphy.dad · 04/11/2024
This is a fantastic read. But depressing, but fantastic. hellooperator.substack.com/p/what-it-me...
hellooperator.substack.com
What It Means to Be Data-Driven
Stop hiding behind the numbers and start using them instead.
2257
Arik Friedman @arikf.net · 02/11/2024
Have you seen vickiboykis.com/what_are_emb... from @vickiboykis.com ?
vickiboykis.com
What are embeddings?
A deep-dive into machine learning embeddings.
130
Arik Friedman @arikf.net · 02/11/2024
I mean, it's great if you happen to be selling those boxes, but it can sometimes feel jarring when you are the box.
010
Arik Friedman @arikf.net · 02/11/2024
This hits me hard.
100
Reposted by Arik Friedman
Jeremiah Lowin @jlowin.dev · 01/11/2024
Yeah, I think the immune response of "AI will replace us" was just so misguided. "AI will help us" is so much healthier. I write more code because of Cursor, not less.
191
Arik Friedman @arikf.net · 02/11/2024
I forgot to hashtag my #dataBS. Apparently this site doesn't have an edit button.
030
Arik Friedman @arikf.net · 02/11/2024
This doesn't cover why I think LLMs won't replace analysts - that's for another thread. But before that’s even on the table, they still lack the reliability for complex tasks. They're useful, especially for pros, but we're not at the point where they can fully replace humans in coding or analysis.
000
Arik Friedman @arikf.net · 02/11/2024
Simple code operations like merging arrays or renaming columns trip me because I don't code frequently enough to memorize the syntax. LLMs save my time, and it's trivial to verify correctness. However, orchestrating these steps is where LLMs can go wrong and human oversight is still critical.
100
Arik Friedman @arikf.net · 02/11/2024
Errors in test code are much lower risk compared to your main code logic. In fact, LLMs can help you improve your main code because they can generate tests for edge cases that you might have not considered yourself.
110
Arik Friedman @arikf.net · 02/11/2024
Test and validation code: my sense is that data scientists don't spend enough time on that, but it’s a good software engineering practice to adopt. LLMs are a great way to generate such code, and you can prompt them to target specific scenarios.
100