Sign in

Ben Bolker

@bbolker.bsky.social
2.5K followers 650 following 1K posts

Ecology, evolution, epidemiology, statistics (mixed models). McMaster University, Hamilton, Ontario math.mcmaster.ca/bolker

PostsRepliesMedia
Ben Bolker @bbolker.bsky.social · 13/10/2026
she's a good person, and a badass. From "Infectious disease : a very short introduction" (Wayne & Bolker 2023, OUP):
One interesting sidelight of the discovery of Bd is the experience of Joyce Longcore, a chytrid expert. After keeping house and raising children for 20 years, Longcore went back to graduate school, receiving a PhD in mycology (the study of fungi) in 1991. As of 1997, just before the discovery of Bd, she looked set for a quiet career studying an obscure family of fungi. When Bd suddenly exploded in importance, she was the go-to person for information about the biology of chytrid fungi, and her career took off like a rocket. Since 1998 she has co-authored 87 papers with more than 7,000 total citations, making her a scientific star. Her story demonstrates the value of having the right knowledge at the right time. It also shows that seemingly arcane biological knowledge can suddenly become vital to understanding a novel ecological situation.
010 0
Ben Bolker @bbolker.bsky.social · 12/10/2026
fun photo of 5 MB of data on punch cards (from www.lanl.gov/media/public... , via this article on programmer Mary Tsingou en.wikipedia.org/wiki/Mary_Ts... : www.nytimes.com/2026/10/10/s... )
black and white photo of a woman in a sweater and skirt standing next to a pile of punch cards that reaches shoulder height. Caption: "IBM’s punched cards were standardized in the 1920s and remained in regular use for more than half a century. These 62,000 punched cards equal roughly five megabytes of data—that’s about one iPhone photo. Los Alamos National Laboratory"
22913 0
Ben Bolker @bbolker.bsky.social · 12/10/2026
@jdushoff.bsky.social your instinct that the constant was very large was correct ... (also, ObXKCD: xkcd.com/3026/ )
So why is this new FFT result interesting? Well, note that in big-O notation, it is technically asymptotically faster than the Cooley-Tukey algorithm because $n \log(n)^{1-10^{-13}} < n \log n$

I mean, sure. But given the constants, the algorithm runs faster only when n is impossibly large. Roughly, you need $n \ge 10^{10^{10^15}}$
an XKCD cartoon

function LinearSort(list):
   StartTime = Time()
   MergeSort(list)
   Sleep(1e6*length(list) - (Time() - StartTime))
   return

"how to sort a list in linear time"
1200 0
Ben Bolker @bbolker.bsky.social · 09/10/2026
With alt text!
1. All languages contain terms for black and white.
2. If a language contains three terms then it contains a term for red.
3. If a language contains four terms then it contains a term for either green or yellow (not both).
4. If a language contains five terms then it contains a term for both green and yellow.
5. If a language contains six terms then it contains a term for blue.
6. If a language contains seven terms then it contains a term for brown.
7. If a language contains eight or more terms then it contains a term for purple, pink, orange, veiled,
grey,
castrated,
foaming,
alert,
heavenstruck,
Kantian,
careless,
gloveless,
bridal,
rattling,
clinging,
neoKantian,
shoved out,
riddled,
dolphin lured,
inflected,
in droves,
unnameable,
exasperated,
strangely not,
the death of us.

ANNE CARSON
000 0
Ben Bolker @bbolker.bsky.social · 05/10/2026
"Never buy dice from a wombat", from @nathanwpyle.bsky.social
a happy (?) looking wombat sitting behind a counter next to a pyramid of dice and in front of a can of white paint with a brush in it. Behind it is a sign saying "All sales final"; below it is a "NO REFUNDS" sign.
1184 0
Ben Bolker @bbolker.bsky.social · 01/10/2026
github.com/bbolker/bbmi... (lazy-coded with help from Claude)
plot of monthly visits to US from Mexico/Canada/UK/Japan/France, log scale. Giant pandemic drop. Only Canada and France are trending down recently
110 0
Ben Bolker @bbolker.bsky.social · 28/09/2026
Yup, that's like these from Agresti. (I should try the Agresti-Coull Beta(2,2) prior ...) I could try again with smaller baseline prob ...
from Agresti and Coull Am Stat 1998: figures of binomial coverage probability for n=5 (top row), 10 (bottom row) for Wald, Agresti-Coull ("Adjusted Wald"), Clopper-WIlson ("Exact")Figure 1.3 from Agresti _Categorical Data Analysis_: Clopper-Pearson, Score, Wald binomial confidence interval coverage for pi = 0-1.
100 0
Ben Bolker @bbolker.bsky.social · 24/09/2026
a lot of scaling to do here ...
The chips can operate for about 15 minutes in space before needing to be shut down so they can cool off, said Travis Beals, Google’s senior director of product management for Project Suncatcher. During that time, the chips can process short queries so Google’s A.I., known as Gemini, can respond.
150 0
Ben Bolker @bbolker.bsky.social · 26/08/2026
Probably an eastern black walnut (_Juglans nigra_)? www.inaturalist.org/observations... (c) Sandy Wolkenberg – some rights reserved (CC BY)
five eastern black walnuts, in their outer casings, slightly decayed, sitting on a bed of brown leaves with some green leaves in the foreground
110 0
Ben Bolker @bbolker.bsky.social · 02/08/2026
Nice. There's a deeper issue w/ CIs for curves (e.g. epidemic trajectories), known for a while in functional data analysis: see e.g. Juul+ (2021) doi.org/10.1038/s415... . (Our commentary was rejected both by Nature Physics & PeerJ ...) math.mcmaster.ca/~bolker/misc... github.com/aligharouni/...
A curve showing an ensemble of trajectories ("number newly hospitalized") from a stochastic epidemic model (black lines), as well as central regions computed in various ways. 

Figure 1. Comparison of 90% central regions computed with different functional boxplot methods,
using epidemic curve ensembles from Juul et al.. FBD = functional band distance; J = number of curves
used for centrality calculation. Curve with J = 2 computed via the roahd package (Ieva et al., 2019):
curve with J = 50 used our own implementation of the functional band distance algorithm described by
Juul et al., curve with Mahalanobis (5 features) used Mahalanobis distance on features of interest
including the peak value of incidence, the time at which the peak occurs, the initial growth rate, epidemic
duration, and final size of the epidemic.
180 0
Ben Bolker @bbolker.bsky.social · 28/07/2026
I might be being a bonehead/reading too quickly, but is the second line here (and in the analogous equation farther down) supposed to be "s ~ ..." ? (i.e., effectively a half-normal prior on s, but making it full-normal is harmless because we're squaring s anyway) ?
an equation $\hat \theta | \theta ~ Normal(\theta, s^2), \theta ~ Normal(\mu, \tau^2)$
100 0
Ben Bolker @bbolker.bsky.social · 28/07/2026
Not sure quite what you're looking for, but iNaturalist has a huge collection - you could target particular known-to-be spotted species, e.g. panthers www.inaturalist.org/observations... There's also an API api.inaturalist.org/v2/docs/ which you could use to (vibe-??) code something ...
part of a panel of photos from iNaturalist showing pictures of leopards (Panthera pardus) in a variety of habitats
120 0
Ben Bolker @bbolker.bsky.social · 20/07/2026
Yup. (1) didn't try to see what happens if I include the redundant term. (2) Need to remember that glmmTMB defaults to ML, lmer to REML (and to use homcs() for glmmTMB, cs(..., hom = TRUE) for lmer)
set.seed(101)
library(glmmTMB)
library(lme4)
dd <- expand.grid(f = factor(rep(LETTERS[1:4])),
                  g = factor(1:25),
                  rep = 1:10)
betavec <- rnorm(4)
sdvec <- rnorm(4)
corvec <- rnorm(choose(4,2))
dd$y <- simulate_new(~f + us(f|g),
                     family = gaussian,
                     newparams = list(beta = betavec,
                                      betadisp = -2,
                                      theta = c(sdvec, corvec)),
                     newdata = dd)[[1]]

m1 <- glmmTMB(y ~ f + us(f|g), data = dd, family = gaussian)
## FIXME: warn if additional args are specified?
m2 <- glmmTMB(y ~ f + homcs(0+f|g), data = dd, family = gaussian)
m3 <- lmer(y ~ f + cs(0+f|g, hom = TRUE), data = dd, REML = FALSE)
stopifnot(all.equal(logLik(m2), logLik(m3)))

m4 <- glmmTMB(y ~ f + (1|g/f), data = dd, family = gaussian)
m5 <- lmer(y ~ f + (1|g/f), data = dd, REML = FALSE)
stopifnot(all.equal(logLik(m2), logLik(m4)))
stopifnot(all.equal(logLik(m5), logLik(m4)))
111 0
Ben Bolker @bbolker.bsky.social · 15/07/2026
I don't know if this is paywalled or not, but e.g.: Jones, Edgar. 2014. “Terror Weapons: The British Experience of Gas and Its Treatment in the First World War.” War in History 21 (3): 355–75. doi.org/10.1177/0968....
If worn correctly, the small box respirator, introduced by the British late in 1916 (though not fully issued until January 1917),41 provided reliable protection against chlorine and phosgene.42 Largely for this reason, in July 1917 the Germans introduced mustard gas, a vesicant that burned the skin. Known as ‘yellow cross’ from the shell markings, the toxin immediately produced a flood of admissions: 160,970 soldiers presented at CCSs between 21 July and 23 November 1917, of whom 1,859 died.43 It was estimated that 75 per cent had been exposed to mustard gas.44 Deaths initially occurred because the toxin had delayed effects.45 The smell of the gas was ‘not very noticeable and… the immediate effect on troops exposed to it’, Brigadier K. Wigram reported, was ‘only a slight irritation of the nose and throat’, so that most troops had been slow to realize that they had been poisoned.46 Mustard gas achieved its greatest effect in the months immediately after its introduction because ‘the novelty of the condition, the multiplicity of the symptoms, and the entire absence of knowledge as to possible after-effects naturally led to the condition being overestimated’.47 Fear inspired by mustard gas was heightened by its capacity to cause loss of sight. Although most blinded servicemen recovered, the acute photophobia, conjunctivitis, and oedema of the eyelids forced soldiers to close their eyes, ‘so much so in fact that when some of the milder cases were evacuated each man had to be led like a blind man by an orderly to the ambulance car’.48 Such images struck a cultural chord and remain among the iconic symbols of the First World War.49
120 0
Ben Bolker @bbolker.bsky.social · 01/07/2026
(sorry)
To put things less delicately, Chernoff faces are an exam-
ple of an idea that is widely acknowledged to be bad, that
is used by essentially nobody in the real world, and yet is
nonetheless widely circulated in academia. Contrast this
with rainbow colormaps and 3D pie charts, which have also
been generally complained about, but are still seen in the
wild with some frequency.
020 0
Ben Bolker @bbolker.bsky.social · 01/07/2026
Correll, Michael. 2018. “Ross-Chernoff Glyphs Or: How Do We Kill Bad Ideas in Visualization?” doi.org/10.1145/3170....
side-by-side plots: (A) a traditional linegraph of bitcoin prices (a jagged, random-walk looking line); (B) automatically restyled as a painting, with white mountains matching the trajectory of the line, with a blue sky above (with stars added) and a gray-green foreground. Caption: "Figure 7: Using a neural net to automatically restyle a time series,
from Moritz Klack [14]. This would be a useful way of generating
custom Ross-Chernoff plots, and also allows me to include “deep
learning” in the author keywords"
120 0
Ben Bolker @bbolker.bsky.social · 30/06/2026
Hmm, Kobo daily deals is offering _Brave New World_ and _The Stranger_ as YA books ... ? I know these are often on high school reading lists, but seems weird. (Not sure CS Lewis would be happy being listed adjacent to Huxley and Camus ...??)
Kobo's "young adult books on sale" showing thumbnails for _The Chronicles of Narnia_, _Brave New World_, _Mission Improbable_, and _The Stranger_
010 0
Ben Bolker @bbolker.bsky.social · 31/05/2026
I like this a lot, and it's impossible to please everyone ... but ... I would like it if at the end we could remind people that if/when they remove predictors to remove instability in estimates, they are **imposing (potentially large) additional assumptions** about which factors are operating ...
Rather than automatically removing predictors, focus on whether multicollinearity prevents you from answering your specific research question, and whether the instability in coefficient estimates is acceptable given your goals.
140 0
Ben Bolker @bbolker.bsky.social · 19/05/2026
(2/n) e.g. why would we expect the effects of (say) income on risk to follow an accumulating-hazard dynamic? Phenomenologically, would we expect an inverse link function that approaches 1 *very* rapidly? (The cutoff above x>3.5 is R's way of saying "too close to 1 to distinguish").
A line plot of four (inverse) link functions [inverse-](cauchit, cloglog, logit, probit) for x from -5 to 5. Probability is represented on the logit scale, so the (inverse)logit is linear by definition. Cauchit is flat and symmetric. Probit has a slightly different scale and is also curvy in the tails. Cloglog increases very rapidly above x=0, and is capped at 1.0 for values > 3.  Code:

library(tidyverse); theme_set(theme_bw())
xvec <- seq(-5, 5, length = 101)
links <- c("logit", "probit", "cauchit", "cloglog")
dd <- setNames(links, links) |>
  purrr::map_dfr(~ data.frame(x = xvec, y = make.link(.)$linkinv(xvec)),
                 .id = "link")
brkvec <- c(1e-5, 0.5, 0.9, 0.999999)
ggplot(dd, aes(x, y, color = link)) +
  geom_line() +
  scale_colour_brewer(palette = "Dark2") +
  scale_y_continuous(trans = "logit",
                     breaks = brkvec,
                     name = "probability (logit scale)")
ggsave("links.png")
120 0
Ben Bolker @bbolker.bsky.social · 04/05/2026
The other (reasonable) point that someone made on CBC news this morning is that the incubation period is long, so that people could have been infected before they started the cruise, or during an excursion off the boat ... www.canada.ca/en/public-he...
INCUBATION PERIOD: 2 to 4 weeks (range from few days to 2 months) for HFRS; and 14-17 days for HPS Footnote 3Footnote 4.

COMMUNICABILITY: Person-to-person transmission is very rare, but has been observed for Andes virus during an outbreak in Southern Argentina Footnote 1Footnote 7.
100 0
Ben Bolker @bbolker.bsky.social · 01/05/2026
Thanks -- apparently it's not originally Rogers -- goes back to an anonymous source in 1932: quoteinvestigator.com/2017/02/23/j... . Can get just enough context from Newspapers.com to see the original source (without paying for a subscription): thestarpress.newspapers.com/search/resul...
A snippet of the Muncie Evening Press (p. 4) from Weds Feb 17 1932. We can see the words "... mostly on experience and ... experience usually comes from ... judgment"
120 0
Ben Bolker @bbolker.bsky.social · 28/04/2026
obAbstruseGoose: github.com/s-macke/Abst... (alt-text modified/corrected from what ChatGPT gave me)
Panel 1
Caption: “SO THE THREE HAPPY FRIENDS WERE JUST STANDING THERE, MINDING THEIR OWN BUSINESS…”
Three simple figures stand side by side. Their torsos are labeled, from left to right: “eˣ,” “eʸ,” and “yˣ.”
Panel 2
Caption: “…WHEN SUDDENLY… OH NO… HERE COMES THE ∂/∂x MONSTER!”
A large dinosaur‑like creature labeled “∂/∂x” approaches from the left toward the three figures, who look alarmed.
Panel 3
Caption: “OMG! HE’S EATING MISTER eˣ!”
The derivative monster bites and swallows the figure labeled “eˣ.” The figures labeled “eʸ” and “yˣ” watch with horrified expressions.
Panel 4
Caption: “BUT WAIT… IT’S OK. HE’S JUST GOING TO POOP HIM RIGHT BACK OUT UNHARMED, THANK GOD.”
The monster crouches and poops out the figure labeled “eˣ”, who bounces once and lands on the floor, smiling.
Panel 5
Caption: “BUT, OH NO, HERE HE COMES AGAIN! RUN! RUN AWAY!… DAMN, IT’S TOO LATE. HE’S GOT MR eˣ!”
The monster lunges back toward the “eˣ” figure and grabs it. Mr. "yˣ" looks on in horror.
Panel 6
Caption: “I CAN’T BEAR TO WATCH THIS. HE’S MAULING MR eˣ TO DEATH! SWEET FANCY MOSES!”
The monster lifts and chews the “eˣ” figure. Mr. "yˣ" watches.
Panel 7
Caption: “THERE’S eˣ BLOOD AND GUTS SPEWING ALL OVER THE PLACE! OH, THE HUMANITY!!!”
Exaggerated gore—detached figure parts and splashes—erupts as the monster continues.
Panel 8
Caption: “RUN, MR yˣ, RUN! THERE’S NO TELLIN’ WHAT THAT ∂/∂x BASTARD IS GONNA DO TO YOUR SORRY ASS…”
The “yˣ” figure runs to the right. A speech bubble from off‑panel says: “OK, BILLY, THAT’S ENOUGH. SIT DOWN.”
Panel 9
The scene is revealed to be a classroom. Billy is working on a problem on the board while his teacher (female presenting) looks on. He says: "BUT I'M NOT FINISHED YET." She says: "YES YOU ARE". 
Chalkboard text (visible):
“∂/∂x ( eˣ + eʸ + yˣ )”
“= eˣ + 0 + …”
041 0
Ben Bolker @bbolker.bsky.social · 28/04/2026
FWIW from www.thebulwark.com/p/exclusive-... this is not universal: 'the State Department is planning a “limited run” of 25,000 Trump-emblazoned passports'. From the Fox story, available by request www.foxnews.com/politics/exc... (I bet some people would be willing to pay extra $$$ for this ...)
A State Department official told Fox News Digital that the new designs will be available for "any American citizen" who applies for a passport when the rollout happens and will continue for as long as there is availability. The passports will only be available at the Washington Passport Agency, the official detailed.
000 0
Ben Bolker @bbolker.bsky.social · 07/04/2026
PS:
same picture as before, but with overseas territories circled in red (two in the Pacific, at left edge; four in South Atlantic; two in north Atlantic [Bermuda, Gibraltar])library(maps)
library(mapdata)
m <- map("worldHires", "uk")
m2 <- map("worldHires", c("UK:Great Britain", "UK:Northern Island",
                          "UK:Scotland",
                          "UK:Guernsey", "UK:Jersey"))

others <- data.frame(x = m$x, y=m$y) |>
  subset(x<m2$range[1] | x>m2$range[2] | y<m2$range[3] | y > m2$range[4])

png("uk.png")
map("worldHires", "uk")
par(xpd=NA)
points(y~x, data = others, col = "red", cex = 2)
dev.off()
000 0
Ben Bolker @bbolker.bsky.social · 07/04/2026
reminds me of this `library(maps); library(mapdata); map("worldHires", "uk")` (hint: in this old database "UK" regions included Gibraltar, Henderson, Pitcairn, Ascension, Gough, Tristan da Cunha, Saint Helena, and Bermuda, in addition to GB/NI/Channel Islands ...)
a map of the UK that includes a bunch of overseas possession in the Atlantic and Pacific, so Great Britain is a very small blob in the upper right hand corner.
120 0
Ben Bolker @bbolker.bsky.social · 04/04/2026
transcribe AMR (sound recorder) file to ABC automatically they said ... it'll be easy, they said ... (AMR →WAV [ffmpeg] → midi [ waon kichiki.github.io/waon/ ] → abc [abc2midi] → PostScript [abcm2ps] → PDF [ps2pdf]. Unsurprisingly, WAV → midi is the weak link. (Should try a simpler round trip next)
A page of complicated sheet music that is more or less gibberish
360 0
Ben Bolker @bbolker.bsky.social · 29/03/2026
Here you go. gist.github.com/bbolker/ce64... Uncertainty estimates are ±2 SE from the `mgcv` prediction for 2026 ...
image showing dates of past NSERC Discovery Grant announcements, with three GAMs (cubic spline, Gaussian process, thin plate). x-axis years 2013-2026 (data up to 2025), y-axis announcement date (March 15 - April 22). 2013 date is around March 22, curve peaks in 202 around April 17. Prediction for GP in 2026 is March 30 (2 SE = 22 March - 7 April). uncertainty estimates for cubic spline, thin plate spline are wider (March 15 - April 15)
191 0
Ben Bolker @bbolker.bsky.social · 18/03/2026
a picture of a white cat peering from behind a computer screen. The screen is showing part of a Firefox window; only the left half of the cat's face is visible.
010 0
Ben Bolker @bbolker.bsky.social · 15/03/2026
I have a very sketchy WIP that plots prior predictive curves and allows the user to select a subset that they want to keep ... github.com/bbolker/bbmi...
A Shiny app showing a subset of lines selected from a prior sample
152 0
Ben Bolker @bbolker.bsky.social · 14/03/2026
I believe the junco and possibly the killdeer. The meadowlark is probably (??) a starling mimic, and the bald eagle *might* have been flying overhead and vocalizing at just the right moment?
A screenshot from the Merlin app showing Bald Eagle, Eastern Meadowlark, Killdeer, and Dark-eyed Junco
000 0
Ben Bolker @bbolker.bsky.social · 12/03/2026
@vdveenb.bsky.social where is the best venue for asking `gllvm` questions? Wanted to see if I could use the devel version to fit a multitype model equivalent to the following MCMCglmm specification (i.e. two different 'rows' grouping variables, each with random slopes) @idworkin.bsky.social ...
## fit multi-type model (don't scale count var)
form_MMLM2  <- cbind(femur_s, tibia_s, tarsus_s, SCT) ~
    trait:(genotype*temp) - 1

m2 <- MCMCglmm(form_MMLM2,
         random=~ us(trait):line, 
         rcov=~ us(trait):units,
         data= as.data.frame(dll_data), 
         family = rep(c("gaussian", "poisson"), times = c(3, 1)),
         nitt= 10000, burnin= 2000, thin=10)
231 0
Ben Bolker @bbolker.bsky.social · 11/03/2026
badge from web site: "CRAN synchronization  failing"GitHub actions failure log, ending  with:

Error in main() : 
  Failed to create branches for 1 package(s): ‘usearchlite’
Execution halted
Error: Process completed with exit code 1.
010 0
Ben Bolker @bbolker.bsky.social · 09/03/2026
Meanwhile in Toronto 🇨🇦 highparknaturecentre.com/cherry-bloss...
The Sakura Cherry Blossoms in High Park are firmly closed in brown buds. It is still too early to make a definitive prediction on when peak bloom will occur.

Note: Sakura Cherry Blossoms in High Park typically reach peak bloom anytime between late April to early May.
110 0
Ben Bolker @bbolker.bsky.social · 08/03/2026
Oops, it's more complicated (sorry). life+70 is for publications after 1978. Otherwise ... maybe have to ask someone who has this stuff at their fingertips ...
For works first published prior to 1978, the term will vary depending on several factors. To determine the length of copyright protection for a particular work, consult chapter 3 of the Copyright Act (title 17 of the United States Code). More information on the term of copyright can be found in Circular 15a, Duration of Copyright, and Circular 1, Copyright Basics.(b) Copyrights in Their Renewal Term at the Time of the Effective Date of the Sonny Bono Copyright Term Extension Act7—Any copyright still in its renewal term at the time that the Sonny Bono Copyright Term Extension Act becomes effective shall have a copyright term of 95 years from the date copyright was originally secured.
120 0
Ben Bolker @bbolker.bsky.social · 03/03/2026
This is decision-theoretic rather than stats only, but: Ludwig, Donald. “Uncertainty and the Assessment of Extinction Probabilities.” Ecological Applications 6, no. 4 (1996): 1067–76. doi.org/10.2307/2269....
The Bayesian results imply much higher risk of early collapse than maximum likelihood methods. This difference is due to large probabilities of early collapse for certain parameter values that are plausible in light of the data. Because of simplifying assumptions, these results are not directly applicable to assessment. Nevertheless they imply that maximum likelihood and similar methods based upon point parameter estimates will grossly underestimate the risk of early collapse.
000 0
Ben Bolker @bbolker.bsky.social · 01/03/2026
Quote is attributed from Amundson R (2005) The Changing role of the embryo in evolutionary thought: Roots of Evo-Devo. Cambridge University Press, Cambridge (Google books screenshot → Harrison, Ross G. 1937. “Embryology and Its Relations.” Science 85 (2207): 369–74 (JSTOR screenshot)
"Embryologist E. E. Just complained that genetics and selection could explain why populations of flies had more or fewer bristles on their backs, but it couldn't explain how a fly constructed its back in the first place (Harrison 1937: 372; Gilbert et al 1996: 361)
241 0
Ben Bolker @bbolker.bsky.social · 28/02/2026
argh.
results of `apropos("R.?[Vv]ersion", ignore.case=FALSE)`: c("getRversion", "R.version", "R.Version", "R.version.string")
130 0
Ben Bolker @bbolker.bsky.social · 25/02/2026
www.burns-stat.com/pages/Tutor/...
Circle 4
Over-Vectorizing
We skirted past Plutus, the fierce wolf with a swollen face, down into the fourth Circle. Here we found the lustful.

It is a good thing to want to vectorize when there is no effective way to do so. It is a bad thing to attempt it anyway. A common reflex is to use a function in the apply family. This is not vectorization, it is loop-hiding. The apply function has a for loop in its definition. The lapply function buries the loop, but execution times tend to be roughly equal to an explicit for loop. (Confusion over this is understandable, as there
is a significant difference in execution speed with at least some versions of S+.) Table 4.1 summarizes the uses of the apply family of functions.
Base your decision of using an apply function on Uwe’s Maxim (page 20).
The issue is of human time rather than silicon chip time. Human time can be wasted by taking longer to write the code, and (often much more importantly) by taking more time to understand subsequently what it does
0151 0
Ben Bolker @bbolker.bsky.social · 13/02/2026
Did you know there was some confusion during the Medieval and Renaissance periods about which end of the Minotaur was bullish and which was manly? scifi.stackexchange.com/a/294815/35859
Image from a 14th c. Italian manuscript of the Inferno in the Bodleian library at Oxford https://digital.bodleian.ox.ac.uk/objects/ab35e336-a471-4cf0-a9a7-592dbb8695d8/surfaces/cba2ff2c-ecd7-44e3-907b-1bd6d0376daf/ : Dante and Virgil are at the right of the image, facing left towards the minotaur who is running down a hill. In this version the head and torso are a man's, the body is the bull's.
240 0
Ben Bolker @bbolker.bsky.social · 24/01/2026
BTW, I got confused: the picture in the thread above is @hadley.nz's original R version. I made this Python version at some point.
Left panel, a salt shaker containing multiple pepper packets, labeled 'x'. Middle, shaker containing one packet, labeled 'x[0:1]'. Right, top, one pepper packet, labeled 'x[0]'. Right, bottom, pile of pepper grains, labeled 'x[0][0]'
010 0
Ben Bolker @bbolker.bsky.social · 23/01/2026
MS Copilot (ugh) gives a useful answer here, if you ask it carefully:
What’s happening?


dd[["x"]] and dd$x return the vector integer64, so sum() dispatches to bit64’s method for the Summary group generics (sum.integer64), which knows how to add 64‑bit integers correctly. Result: 55 ✅


dd["x"] returns a one-column data.frame (i.e., a list). When you call sum(dd["x"]), you’re now calling sum() on a data.frame, not on an integer64 vector.
For data.frames, base R uses the Summary group generic method Summary.data.frame (in base). Internally, this method coerces the data frame to something vector-like (e.g., via as.matrix() or unlist() pathways) before applying the sum generic. In doing so, it strips the integer64 class, and the remaining underlying storage is misinterpreted.


integer64 (from bit64) is not a native R type; it’s a classed vector with an underlying storage that does not map 1:1 to base numeric types. When the class is stripped by the data.frame method, the bytes get reinterpreted (e.g., as doubles) instead of being properly converted. That’s why you see a tiny, nonsensical denormal-like value such as 2.717361e-322 — it’s the result of interpreting the 64‑bit integer bits as an IEEE-754 double, not the sum of actual numbers.
140 0
Ben Bolker @bbolker.bsky.social · 23/01/2026
(1/2) It's definitely the interaction of single-brackets and integer64. Single square brackets return a list, double square brackets (generally safer/best practice!) return a vector. (Left-hand image is a modified version of a Python original by @hadley.nz, I think.).
> library(bit64)

> dd <- data.frame(x = as.integer64(1:10), y = as.integer(1:10))

> sum(dd$x)
integer64
[1] 55

> sum(dd["x"])
[1] 2.717361e-322

> sum(dd[["x"]])
integer64
[1] 55

> sum(dd$y)
[1] 55

> sum(dd["y"])
[1] 55

> sum(dd[["y"]])
[1] 55
> A multi-panel figure. Left: 'x'; a pepper-shaker containing a collection of paper packets of pepper. Middle: 'x[1]': a pepper-shaker containing a single pepper packet; Right, top: 'x[[1]]': a single pepper packet; Right, bottom: 'x[[1]][[1]]': a pile of grains of pepper
350 0
Ben Bolker @bbolker.bsky.social · 20/01/2026
annals of yak shaving: troubling remembering the name of the amazing github.com/cynkra/const... pkg (human-readable `dput()`). I remember (1) I have it installed (2) the names starts with "c" and (3) is an English word ...
grepv("^c", rownames(installed.packages())) |> Filter(f = \(x) x %in% qdap::GradyAugmented)
010 0
Ben Bolker @bbolker.bsky.social · 12/01/2026
Hmm, I'm not seeing much difference. What am I missing?
R code

library(microbenchmark); library(ggplot2)
set.seed(1001)
n <- runif(1e6, 0, 2)
m1 <- microbenchmark(
  pmax(n,1),
  pmax.int(n,1),
  {n[n<1] <- 1}
)
autoplot(m1)
A 'microbenchmark' plot of pmax(), pmax.int(), and n[n<1] <- 1 as in the previous code. medians are pretty close together, bulk of the results are between 1.5 and 3 milliseconds for all three methods. maxima are larger (longer tail) for pmax.int and sub-assignment
100 0
Ben Bolker @bbolker.bsky.social · 12/01/2026
Nice. Related to a WIP with @jdushoff.bsky.social : bbolker.github.io/powertargets... : basically, extending power analyses to see which bin we'll fall in (I do agree that CIs are better!)
A line graph with sample size per group on the x axis, proportion of simulations on the y-axis (empirical estimate of probability). Lines disaggregate outcomes: "clearly large and positive" starts at about 0.08 for n = 5, grows to near 1.0 for n>100. "Clearly positive, maybe large" peaks around n=10. Not both (large and negative) declines from 0.3 to nearly 0 at n=20. Other categories are very low prob in this case.
2124 0
Ben Bolker @bbolker.bsky.social · 08/01/2026
Hey @visnut.bsky.social @hadley.nz were you aware of this prior art on graphical diagnostics à la github.com/dicook/nulla... ? Schruben 1980 doi.org/10.1177/0037... (didn't see it in refs of Buja+ 2009; found ref in Brillinger+ 2008 projecteuclid.org/euclid.imsc/...; they call it a 'Turing test')
A flow chart from Schruben 1980 showing (top) 'exogenous inputs'. These flow to 'real system' (left) and 'simulated system' (right).  Both the real and simulated systems flow to a report; the two reports then flow into a 'shuffle' step, which then goes to 'manager', and finally (bottom) splits to two boxes labeled 'real' and 'bogus'.
120 0
Ben Bolker @bbolker.bsky.social · 06/01/2026
probably from observed testing trends e.g. www.publichealthontario.ca/en/Data-and-... (hospitalizations haven't peaked, but those would tend to be lagging indicators ...)
screenshot from the Ontario respiratory virus dashboard showing % positivity [COVID-19, influenza, RSV]; cases [COVID-19, influenza]; outbreaks [all three]; hospital admissions [all 3]. Focusing on influenza, the top three indicators (% positivity, cases, outbreaks) all appear to have peaked recently
130 0
Ben Bolker @bbolker.bsky.social · 06/01/2026
plotly use cases: (1) 3D plots where appropriate - ggplotly has worked better for this purpose (for me) than rayshader, and more automatically (mapping aesthetics etc.) than rgl ... (2) hover/mouseover popup labels are nice instead of static labels for crowded plots (ggiraph would work for that)
a 3D surface plot showing (x) prey size, (y) prey density, (z) proportion predated
110 0
Ben Bolker @bbolker.bsky.social · 30/12/2025
Based on the pattern this looks like straight-up metadata mangling rather than LLM hallucination?
Cytokine-mediated CAR T therapy resistance in AML

    Anand S. Bhagwat, Leonel Torres, Olga Shestova, Maksim Shestov, Patrick W. Mellors, Hansen R. Fisher, Saamia N. Farooki, Benjamin F. Frost, Michael R. Loken, Avery L. Gaymon, Diane Frazee, Walter Rogal, Noelle Frey, Elizabeth O. Hexner, Selina M. Luger, Alison W. Loren, Mary Ellen Martin, Shannon R. McCurdy, Alexander E. Perl, Edward A. Stadtmauer, Jennifer L. Brogdon, Joseph A. Fraietta, Wei-Ting Hwang, Don L. Siegel, Gabriela Plesa, Richard Aplenc, David L. Porter, Carl H. June & Saar I. Gill
120 0
Ben Bolker @bbolker.bsky.social · 11/12/2025
(1/2) Yours is prettier than mine: gist.github.com/bbolker/1bb9... (our earliest sunset was Dec 7 at 16:24:03, although that only differed by 3.9 seconds from the sunset on Dec 8 ...) --
110 0