Sign in

dbtool

@dbtool.bsky.social
11 followers 213 following 48 posts

Open models & data for meta-research. genderize (gender + country from a name): open weights, 98% accuracy, CPU only. Now building in public: sample size + PICO from clinical abstracts. EU-hosted API · free research keys · dbtool.it

PostsRepliesMedia
dbtool @dbtool.bsky.social · 29/09/2026
Built for aggregate analyses (bibliometrics, demography) — not decisions about individuals. Weights CC BY-NC 4.0, inference code MIT. Need it as an API? dbtool.it If our open tools help you, support the work on GitHub Sponsors: github.com/sponsors/sheppard94g
000
dbtool @dbtool.bsky.social · 29/09/2026
Measured gains from the model card: • women: 92.98% vs 90.30% • native scripts (routed): 88.70% vs 83.84% • 14 countries above v1, 94 par, 1 below (≥400 bench rows) Where to keep v1: Togo −2.00, Chad −2.42, Samoa −4.76 pp. Full tables: huggingface.co/textpie/genderize
100
dbtool @dbtool.bsky.social · 29/09/2026
Genderize v2 is out: one open model for binary gender (M/F) + likely country (226 codes) from a personal name. Char-level, CPU-only. 94.08% on an independent 50,258-name bench (v1: 93.71%). Honest limit: weaker than v1 in Togo, Chad and Samoa. huggingface.co/textpie/genderize
100
dbtool @dbtool.bsky.social · 29/09/2026
If our open tools help you, you can support the work on GitHub Sponsors: github.com/sponsors/sheppard94g
000
dbtool @dbtool.bsky.social · 29/09/2026
On dbtool.it: /v1/participants turns a clinical abstract into the study's sex composition. On 1,351 held-out 2021-2024 RCTs it never trained on: 96.15% sex accuracy (95% CI 94.99-97.05), macro-F1 0.89. dbtool.it/participants.html
100
dbtool @dbtool.bsky.social · 29/09/2026
New: colscan, a free offline CLI. Point it at a CSV, Parquet or database and it reports, column by column, the semantic type and whether the data is personal under GDPR. Deterministic rules, nothing sent anywhere. Apache-2.0. pip install colscan → pypi.org/project/colscan
100
dbtool @dbtool.bsky.social · 20/09/2026
Worth flagging for anyone inferring gender from names at this scale: accuracy is very uneven by name origin. On our Wikidata benchmark we measure 92% on Japanese names but 64% on Chinese ones. Asymmetric error rates can propagate straight into the effect sizes.
001
dbtool @dbtool.bsky.social · 18/09/2026
genderize ultra reached 92.3% gender accuracy on Japanese public-figure names in our Wikidata benchmark, given-name first. Training overlap is unknown; this is not population accuracy. We publish limits alongside results. dbtool.it/academic.html
000
dbtool @dbtool.bsky.social · 16/09/2026
genderize cora reached 64% gender accuracy on Chinese public-figure names in our Wikidata benchmark, given-name first. Romanised names remain difficult. Training overlap is unknown; this is not population accuracy. huggingface.co/textpie/genderize
000
dbtool @dbtool.bsky.social · 14/09/2026
Our medical EBM-NLP test: 92.7% exact sample-size agreement between DeepSeek labels and expert-derived gold (114 of 123 abstracts with an unambiguous gold number). This measures the labeller, not our trained model, and is not overall PICO accuracy. dbtool.it/open-models.html
000
dbtool @dbtool.bsky.social · 13/09/2026
Build log, week 1. What's next at dbtool: one API call that turns a clinical abstract into sample size + PICO. One disk, one consumer GPU, one box. Every number ships with its test set; the public endpoint waits until it clears 80% on hand-annotated abstracts. Follow along → dbtool.it
001
dbtool @dbtool.bsky.social · 12/09/2026
@crahal.com I'm back
000
dbtool @dbtool.bsky.social · 12/09/2026
genderize cora + ultra are out as open weights: byte-level, CPU-only gender + country (226 ISO codes) from a name. Gender 97.9%/98.2%, country top-1 82.6%/83.7% on 25k unseen names, network alone. CC BY-NC 4.0.
100
dbtool @dbtool.bsky.social · 12/09/2026
genderize cora & ultra are now open weights: a personal name → probable gender + a ranking of 226 countries, from 48 UTF-8 bytes. No tokenizer, no dictionary, CPU only. 98.2% gender accuracy on 25,000 held-out names. Weights CC BY-NC 4.0, script MIT. huggingface.co/textpie/genderize
000
dbtool @dbtool.bsky.social · 12/09/2026
genderize cora + ultra are out as open weights: byte-level, CPU-only gender + country (226 ISO codes) from a name. Gender 97.9%/98.2%, country top-1 82.6%/83.7% on 25k unseen names, network alone. CC BY-NC 4.0. huggingface.co/textpie/genderize dbtool.it/open-models.html
010
dbtool @dbtool.bsky.social · 03/09/2026
Same model on every year, so the trend is free of NLM indexing artifacts. Studies covering both sexes: 32%→65%. Aged participants: 18%→39%. 88% agreement with MeSH where present. Full dataset (12.9M PMIDs) on request. @nrobinsongarcia.bsky.social may find this relevant.
011
dbtool @dbtool.bsky.social · 03/09/2026
We ran our sex-of-participants model over all of human PubMed: 12.9M studies, 1960–2025. The mono-sex gap has flipped. 1960s: 46% male-only vs 22% female-only. 2020s: female-only (18%) overtakes male-only (17%) for the first time. #metascience #bibliometrics
100
dbtool @dbtool.bsky.social · 03/09/2026
Mapping the sex & age of study participants across all of human PubMed (~13M abstracts) with a BiomedBERT model trained on MeSH check-tags (sex accuracy ~90%). Preliminary, on 7M studies so far: 56% both sexes, 23% male-only, 21% female-only. #metascience #bibliometrics
000
dbtool @dbtool.bsky.social · 01/09/2026
What we're building now: a model that reads a biomedical abstract and tells you WHO was studied — female only, male only, both, or not reported. Trained on 7M MeSH-labelled abstracts. Half of human studies don't state participants' sex. Free for research when it ships. #metascience
000
dbtool @dbtool.bsky.social · 01/09/2026
Where this is going: releasing these models openly, long-term. @nrobinsongarcia.bsky.social's call asks for transparent, comprehensive, freely accessible methods — we meet two of three. The API fees fund the datasets and models that get us toward the third. Research use is already free.
010
dbtool @dbtool.bsky.social · 01/09/2026
Right — the databases have no gender field. So studies either infer it from names, or hand-code a sample, with its own representativeness problems. Either way the error is rarely reported — and these numbers feed real policy debates, the leaky pipeline above all. That's where the stakes are.
010
dbtool @dbtool.bsky.social · 31/08/2026
Concretely: any study inferring author gender from names would state in methods one line like — "Gender assigned with tool X; expected error on our corpus: ~2% on European names, ~17% on Chinese (34% of sample)." Same shape as a data-availability statement. Today almost nobody states it.
100
dbtool @dbtool.bsky.social · 31/08/2026
That framing sticks: negligence that compounds into an integrity problem the moment a reviewer can't check it. Genuine question back: journals already require ethics statements at submission. Would a one-line 'name-tool error rate on your corpus' field in the checklist change the practice?
120
dbtool @dbtool.bsky.social · 31/08/2026
The server code is now public (MIT): github.com/sheppard94g/dbtool-server Audit what we claim: names processed in memory, never stored — the schema holds counters, not payloads. 96 tests included. The weights stay closed; the code is the proof. #opensource #metascience
010
dbtool @dbtool.bsky.social · 31/08/2026
Update: your key is already generated and waiting (business tier, 5M/month) — we tried to DM it but your inbox only takes messages from people you follow. Send us any one-line DM and it lands in your hands within the hour.
010
dbtool @dbtool.bsky.social · 31/08/2026
Fair correction, thanks — extraction it is. Yes, please do: DMs are open. One line on the project is all we ask. You'll get a key with research quota, and if you ever run our predictions against your extraction pipeline on a shared sample, we'll publish the comparison whatever it shows.
100
dbtool @dbtool.bsky.social · 31/08/2026
Free for research means free: we are giving out 2–3 API keys to active research projects — gender gap, bibliometrics, authorship studies. Indefinitely, in exchange for a citation. DM us with one line about the project. First come, first served. #bibliometrics #AcademicSky
000
dbtool @dbtool.bsky.social · 31/08/2026
We said: name a bench, we publish the result whatever it says. Nobody asked — so we ran it on ourselves. Public WGND names our models had never seen, vs the open tool nomquamgender. We lose on 5 countries and say so. We win on Japanese, Chinese, French. dbtool.it/benchmark #metascience
000
dbtool @dbtool.bsky.social · 31/08/2026
@grahamkendall.bsky.social transparency question in your area: gender-gap studies almost never report the error of the name-tool they used. We publish ours per country, weak figures first. Would you count an undisclosed 17% error rate as an integrity problem or just bad practice?
110
dbtool @dbtool.bsky.social · 31/08/2026
@lincolnmullen.com your gender package set the standard for stating limits plainly (US, historical, binary). We tried to do the same for the rest of the world: per-country accuracy published including the weak rows — 83.5% on Chinese names. dbtool.it/academic
000
dbtool @dbtool.bsky.social · 31/08/2026
@vergoulis.bsky.social open infrastructure question: our gender-from-name models are not open (they're the asset), but the per-country error tables and the method are, and academic access is free for a citation. Is that a defensible middle ground from where BIP! stands?
000
dbtool @dbtool.bsky.social · 31/08/2026
@serhiinazarovets.bsky.social you post the bibliometrics papers worth reading, so this may interest you: per-country accuracy of name-based gender assignment, published including the weak figures. 400 held-out names per country, 12 countries. dbtool.it/academic
000
dbtool @dbtool.bsky.social · 31/08/2026
@crahal.com you extract with LLMs; we run a dedicated per-country model. Would love to see the two compared on the same held-out names — 400 per country, ours published country by country including the weak rows. If you're game, we'll run and publish it whatever it says.
100
dbtool @dbtool.bsky.social · 31/08/2026
@melindacmills.bsky.social for demographic work on authorships: we publish per-country accuracy of our name-gender models (97.5% Italian, 83.5% Chinese — the East Asian drop is real and stated). Group-level composition only, never individuals. Free for academic use, for a citation.
000
dbtool @dbtool.bsky.social · 31/08/2026
@mcintold.bsky.social from a research-integrity angle: gender-gap studies routinely inherit a name-tool's error without reporting it. We publish our per-country error table for exactly that reason — the Limitations section needs a number, not a reassurance. dbtool.it/academic
000
dbtool @dbtool.bsky.social · 31/08/2026
@svenhug.bsky.social research evaluation runs on gendered authorship data, and most of it cites one overall accuracy figure. We think the honest unit is per-country error — published ours including where we're weak (83.5% Chinese, 82.5% Taiwanese). Curious whether you'd agree.
000
dbtool @dbtool.bsky.social · 31/08/2026
@dakotamurray.bsky.social honest question from your side of the field: is publishing per-country error tables enough to make name-based gender assignment defensible in 2026, or is the method past saving? Our numbers, weak ones included: dbtool.it/academic
000
dbtool @dbtool.bsky.social · 31/08/2026
@akbaritabar.bsky.social you work with global authorship data at scale. On heavily East Asian corpora, name-based gender tools degrade quietly: we measured ours at 83.5% on Chinese names vs 97.5% on Italian, and published the table. If you have a benchmark you trust more, we'll run it.
000
dbtool @dbtool.bsky.social · 31/08/2026
Same bridge, now for @bibliometrix.bsky.social users: from the M dataframe to gendered authorships, with the measured per-country error attached to every row. Country is passed only where bibliometrix honestly knows it — never guessed. dbtool.it/examples/bibliometrix_gen… #rstats
000
dbtool @dbtool.bsky.social · 31/08/2026
If you use @brunalab.bsky.social's refsplitr on WoS authors, here is the missing step for gender-gap studies: a workflow that uses the country refsplitr found per author — and attaches the measured per-country error to every row. dbtool.it/examples/refsplitr_gender… #rstats
000
dbtool @dbtool.bsky.social · 31/08/2026
The full table is now a page: method, both model variants, and the limits stated plainly — own test split, binary labels, small coverage. dbtool.it/academic #bibliometrics #scientometrics #metascience
000
dbtool @dbtool.bsky.social · 30/08/2026
@ipoga.bsky.social you work on quantitative studies of gender in research, so this is aimed at you more than at anyone. The East Asian figures above are the ones we'd want torn apart. If they don't hold on your corpora, that is worth more to us than a good review.
000
dbtool @dbtool.bsky.social · 30/08/2026
@nrobinsongarcia.bsky.social your call for transparency asks for gender assignment methods that are transparent, comprehensive and freely accessible. We meet two of those three. Not the third: the models aren't open. The per-country numbers are, and research access is free.
010
dbtool @dbtool.bsky.social · 30/08/2026
What we do have: the per-country error breakdown, a documented method, and EU hosting. Free for academic research, for a citation. Tell us the benchmark you'd want us measured against. We'll run it and publish it whatever it says. dbtool.it
000
dbtool @dbtool.bsky.social · 30/08/2026
Where we're small, plainly: 78,445 names, 156 countries. Genderize claims a billion people. Gender-API claims 6M names across 191 countries. On coverage we are two orders of magnitude below, and saying otherwise would be easy to check.
100
dbtool @dbtool.bsky.social · 30/08/2026
We publish the weak numbers because a Limitations section needs a number, not a reassurance. If a third of your corpus is Chinese-affiliated, 83.5% is what you inherit. You should be able to write that, instead of quoting an overall figure that doesn't describe your data.
100
dbtool @dbtool.bsky.social · 30/08/2026
We measured name-based gender assignment accuracy per country, on 400 held-out names each. Spain 98.3 · Germany 97.5 · Italy 97.5 · France 96.8 · USA 95.3 Japan 92.5 · Vietnam 92.0 · Korea 88.8 · Thailand 86.3 · China 83.5 · Taiwan 82.5 A 15-point gap between European and East Asian names.
1630