"[Variants of R-hat and the effective sample size] flag failures of convergence, but they are not designed to characterise the structure of that failure."
Excellent work that clearly identifies practical situations where this matters, proposes a diagnostic, explains how to use it and how *not* to.
![Although this has been thoroughly discussed in the FDA guidelines [1] and elsewhere in literature for non-collapsible measures (for example, [4]), the survey result suggests that many respondents were still not aware that covariate adjusted or stratified analyses may estimate different quantities in a non-linear model. The results are summarized in Table 1. Among the 122 respondents, 61.5% respondents think that a stratified analysis and unstratified analysis target the same estimand in non-linear models; if one is chosen as the primary analysis, the other could be a "sensitivity analysis”, (which, by definition, targets the same estimand as the primary analysis). Similarly, 56.6% of respondents indicated that covariate-adjusted analysis and covariate-unadjusted analysis target the same estimand in non-linear models, and 57.4% respondents believe removing or pooling strata on an ad-hoc basis for interim analysis does not change the pre-specified estimand. The result is quite consistent across the three questions, which demonstrates the gap in the understanding of different statistical analysis models targeting different estimands for non-collapsible measures, and the critical need of further clarification and training on this topic. Retrospectively, we acknowledge that the survey question in relation to unstratified analyses may have been open to interpretation, since this can refer to an unadjusted analysis (which would always target a marginal estimand, or an adjusted analysis (which may target a conditional or marginal estimand in non-linear models depending on how it is approached in estimation). For clarity, our recommendations and conclusions insinuate that the unstratified analyses referred to unadjusted models. However, we highlight this as a limitation of the responses to the question in reference to unstratified analyses.](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:3zyo4iakhqs47bttjalqgbk7/bafkreihtyofpvpgmwmkfpnihxmba7xfclhpd7bv7tphuh34rp3b7txaiye)
