iiinova — where your digital self keeps evolving

Reposted by Norm Matloff (你有冇諗清楚呀?)

Paul Pharoah @paulpharoah.bsky.social · 28/05/2026

#statsky #episky I'm reviewing a paper and a link is provided to a github with the analysis code. There are several R scripts (none annotated) and the main, complex analyses clearly not included in any of them. The data are available "on reasonable request". Is this a red flag? Does it matter?

352

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 22/05/2026

Yes, the assumptions underlying 'mice' are unrealistic, even for continuous variables.

030

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 18/05/2026

Looks excellent, very versatile and cleanly designed, but some links and the MNIST example seem broken.

120

Reposted by Norm Matloff (你有冇諗清楚呀?)

Frank Harrell @f2harrell.bsky.social · 16/05/2026

Computing tool of the day: I've needed to have an elegant way to stitch together pieces of Quarto books to make a unified html file for teaching. In collaboration with Claude I now have a great yaml-driven tool for this. #RStats

Yaml markup for specifying which sections of which books to extract or exclude, for building a course html presentation file.

4333

Reposted by Norm Matloff (你有冇諗清楚呀?)

Christos Argyropoulos MD PhD @christosargyrop.mastodon.social.ap.brid.gy · 11/05/2026

I stand behind this #hatespeech #rstats #clang blog.r-project.org/2019/03/28/use-o… "A summary of the recommendation would be: don’t use C++ to interface with R. If you need to implement some computation in native code, use C (or perhaps Fortran), not C++"

blog.r-project.org

Use of C++ in Packages - The R Blog

032

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 09/05/2026

Yes, indeed you did. Sorry. But the reason I didn't remember it is that you had no specific comments. Can you comment on at least one of the case studies?

000

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 08/05/2026

Still waiting for your comments on matloff.github.io/WackyLinearAlgebra/Powered-by-Linear-Algebra.pdf.

matloff.github.io

100

Reposted by Norm Matloff (你有冇諗清楚呀?)

Christos Argyropoulos MD, PhD, FASN 🇺🇸 0kale/acc @christosargyrop.bsky.social · 04/05/2026

And even if you send, I will use a chatbot to rewrite in base R and data.table as God intended

041

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 06/05/2026

I don't think I've ever said that base-R is easier to READ; I say that it is easier to WRITE (for beginners).

210

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 04/05/2026

I suppose it's possible that your brain is wired to crave complexity. :-) I'd be curious to see your take on matloff.github.io/TidyverseSkeptic.

matloff.github.io

Redirect to Skeptic.html

120

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 03/05/2026

If you've ever wanted to learn data.table, this looks excellent.

010

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 25/04/2026

Nice history, but the Unix roots probably should be clarified here. Unix pipes are mainly intended for functions with a single input and a single output, and already debugged. R pipes go well beyond that.

010

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 20/04/2026

Is this really true? If so, it's depressing news.

010

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 19/04/2026

Thanks, any feedback would be appreciated. We are currently converting it to published form with Taylor and Francis (with free HTML access), with my daughter, Laura Matloff, as coauthor.

031

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 19/04/2026

Yes, you can view it as an operator, but the main thing to know is that "+" is a function, e.g. > "+"(3,5) [1] 8 Similarly, > x <- c(6,1,1,4,5) > "["(x,3) [1] 1 This epitomizes the fact that R is a functional language.

170

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 18/04/2026

Even though I do not use IDEs, I did install it and gave it a run a few weeks ago. I was quite impressed. The authors have clearly given very careful thought to what an IDE should do.

010

Reposted by Norm Matloff (你有冇諗清楚呀?)

sducrocq.bsky.social @sducrocq.bsky.social · 15/04/2026

Happy 20th Anniversary to #rdatatable! 🎂 Since 2006, DT[i, j, by] has been the gold standard for high-performance data manipulation in R. Speed, efficiency, and elegance! A huge thanks to Matt Dowle, Arun Srinivasan, and all the contributors! @rdatatable.bsky.social 🥂 #rstats #rdatatable

0134

Reposted by Norm Matloff (你有冇諗清楚呀?)

Adrian Barnett @aidybarnett.bsky.social · 15/04/2026

Outstanding work by @alexdgibson.bsky.social to uncover the use of highly dubious data sets from @kaggle.com being used in hundreds of research papers and potentially even informing clinical practice. If you're re-using data, take the time to confirm that it's real. www.nature.com/articles/d41....

nature.com

Dozens of AI disease-prediction models were trained on dubious data

The models are designed to predict someone’s risk of diabetes or stroke. A few might already have been used on patients.

02311

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 11/04/2026

The special issue of the Australian and New Zealand J. of Statistics commemorating the 25th anniversary of R (the language was developed in NZ) is finally out, with free access for the next 90 days (yes, an irony for an open source language), onlinelibrary.wiley.com/doi/toc/10.1...

onlinelibrary.wiley.com

25 YeaRs: Australian & New Zealand Journal of Statistics

<em>Australian & New Zealand Journal of Statistics</em> is an international statistics journal covering statistical theory, methodology, applications and computing.

0101

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 31/03/2026

Frank, are you familiar with the controversy over a 2022 paper by Pekar, udinh Bayesian analysis to claim Covid had a zoonotic genesis? The critics claim errors in his analysis of the chance of 2 zoonotic causes, close in time, but even without errors, should an issue like this be analyzed by Bayes?

110

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 31/03/2026

I think is not talking about nested calls; she's doing nested definitions, it seems.

110

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 27/03/2026

One of the "drawbacks" that Thomas cites is side effects. But that is precisely the reason why I use macros! (Which I have been doing for a long time using his gtools package.) What I want is something that is basically freestanding code but has the look and feel of a function.

000

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 27/03/2026

Macros in gtools have been around for a lot longer than that.

000

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 27/03/2026

That blog post is a rare example of a package author giving a clear and convincing account of their design decisions, very nice.

031

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 27/03/2026

The failure of the LLMs (the ones I tried) to ask whether the return value of loess() has a component like 'fitted' is quite telling, IMO. Any competent R programmer would check this as a first resort.

000

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 27/03/2026

Here are the 2 graphs, hopefully now showing as white background on your screens. As I said, my post was just meant to make a couple of small points: a. LLMs can produce poor code even in very simple apps. b. The curves' color change can be very useful, here showing federal law.

100

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 27/03/2026

Hmm. In the file-save function I wrote for images long ago and have been using ever since, I see that I have a call png(filename,bg='white') Just now, I tried adding bg='white' to my call to plot() in qepc, still have the same problem. Thanks anyway.

010

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 27/03/2026

Nice! I have my own Vim macros for this kind of thing, but your system is more versatile.

021

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 27/03/2026

US census data uses "top coding." Here it meant that any wage about $350K was truncated to that value. Now remember, this is Year 2000 dollars. Those data points created an ugly bar at the top of the plot, and had the effect of flattening the curves. So I deleted any row having wage > $200K. 8/8

000

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 27/03/2026

My call for the Economist-style graph was qepc(svcensus[-rich,], 'age','wageinc','occ',prePost=TRUE,prePostChangePt=40) Here qepc is the modified qePlotCurves, which I will add to qeML. What about that -rich? 7/

100

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 27/03/2026

The LLMs did something similar, but did not take advantage of the $fitted component of the object returned by loess(). Instead, they regenerated the fitted values by calling predict() on x. Not a big sin, but it does show that LLMs may not do well writing even very simple code. 6/

100

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 27/03/2026

Now, an interesting point re implementation. Easy, but the LLMs that I tried seemed to miss something. The qePlotCurves function uses loess() to smooth the data. The return value includes the x and fitted values. I called order() on x, and then called lines() twice, for the before and after. 5/

100

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 27/03/2026

Ah, the pesky black background again! Again, the original is white. Sorry. Unfortunately, I don't have the time to hunt that down. Anyone knows what Bluesky mistake I made? There is a vertical line at age = 40, hard to see, but one can see the change in curve color to gray at that point. 4/

200

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 27/03/2026

(Not sure where that black background came from, not in the original .png.) So I added the feature to qePlotCurves, and got this graph below. I chose age 40 for the dividing line, as it is the cutoff in federal law on age discrimination. Seems to match the data! 3/

100

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 27/03/2026

I thought it would be a nice enhancement of this plot made by qeML::qePlotCurves on a census dataset on Silicon Valley wages in 2000. Wages initially go up, but start to come down as one grows older. Different curves are for different occupations, but all are in the tech area. 2/

100

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 27/03/2026

Here's a quick little project that I hope some will find interesting in misc. ways. I saw the below graph in The Economist, showing pre- and post-data, here 2025. thread 1/

220

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 25/03/2026

I have always thought the word "concentration" instead of "major" is an affectation, like Stanford's referring to its Fall term as Autumn. Is there actually a difference?

010

Reposted by Norm Matloff (你有冇諗清楚呀?)

Gilles Colling @gilles-colling.bsky.social · 24/03/2026

200 treated patients. 800 controls. Five covariates to match on. How do you pair them? Propensity scores collapse those five dimensions into one number and match on that. Clean and well-understood. But you can also solve the pairing directly in the original covariate space. 1/4

111

Norm Matloff (你有冇諗清楚呀?) @matloff.bsky.social · 23/03/2026

(Re-do.) I'm enjoying your posts. Nice package! But, a suggestion: I think many readers would like to copy and paste your example code, but you make that impossible by placing code and output in one image. Why not separate them?

220