“Search 100M+ papers to find the strongest evidence for and against, for any scientific claim” paperclip.gxl.ai/evidence-api
paperclip.gxl.ai
Paperclip
Paperclip — search, read, and analyze 11M+ biomedical papers, regulatory documents, clinical trials, and biological databases (UniProt, PDB, ChEMBL) from the command line.
![Rovibronic coupling
Rovibronic coupling, also known as rotation/vibration-electron coupling, denotes the simultaneous interactions between rotational, vibrational, and electronic degrees of freedom in a molecule.[1] When a rovibronic transition occurs, the rotational, vibrational, and electronic states change simultaneously, unlike in rovibrational coupling. The coupling can be observed using spectroscopy, and is most easily seen in the Renner–Teller effect in which a linear polyatomic molecule is in a degenerate electronic state and bending vibrations will cause a large rovibronic coupling.](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:onk4xy2h4k2wenfbdaelpcqc/bafkreihmdie5izxbsjbdc34weqh2m7dccit26gmobwogszz5e3svkgusfi)
![Claude chat excerpt:
**The change:** Instead of reading every column from the claims Parquet file with Polars' streaming engine (`file_refs`), they switched to reading only the **"needed leaves"** with PyArrow — the specific nested fields that `file_refs` actually uses:
- `property`
- `datavalue.id`
- `datavalue.unit`
(These are accessed at any depth in the main snak, qualifiers, and references.)
**The gain:**
- **Time:** 48 s → **30 s** (1.6× faster)
- **Peak RSS:** 21.4 GiB → **17.4 GiB** (almost 4 GiB lower)
- Same 871,948 refs produced, verified
**Why it works:** PyArrow's `read(columns=)` can project nested columns/leaves from Parquet at decode time — you only decompress and convert to Arrow what you actually need, rather than decoding the entire structure and then selecting fields afterward. The row explosions (snaks into refs) still dominate memory usage, but the payload to decode is far smaller.
The benchmark script (`refs_bench.py`) in the journal compares the two methods side-by-side and is now integrated into the pipeline for `file_refs` on the main set.
(can polars not do that?)
Not efficiently for nested structures. Polars can select top-level columns via `scan_parquet(columns=...)`, but it doesn't expose PyArrow's fine-grained **leaf-level column projection** for nested Parquet fields.
From the journal, when they tried `pl.scan_parquet(path).slice(start, n)`, it "apparently [read] far more than the row group, three times (main snaks, qualifiers, references)" — Polars' streaming engine was still decoding entire nested structures even when only a few leaves were needed.
PyArrow's `read(columns=["main.property", "datavalue.id", ...])` is a native Parquet feature: the file format supports reading just those leaf paths without touching sibling fields. PyArrow exposes this directly. Polars, built on PyArrow underneath, doesn't expose that capability at the Polars API level — you have to drop down to PyArrow to use it.](https://cdn.bsky.app/img/feed_thumbnail/plain/did:plc:onk4xy2h4k2wenfbdaelpcqc/bafkreidus3fjkafpieq62gaj3vlzokupg4udz6cudt5mmno5xampvvwxr4)
