Note that a PDF also exists: https://arxiv.org/pdf/2606.15991 Comments

The DWARF parsers used to allocate lots of HEAP memory, including thrown away objects, lots of strings (most never used), and build large matrix tables. It could quickly fill GC memory that would n...

Historical GitHub uptime reconstructed from archived status data.

The Software Stewardship Lab launches today, a Scottish non-profit set up to do applied research on the open source ecosystem, and I’m one of its directors. Vlad-Stefan Harbuz, who runs the Open Source Pledge, is the executive director and did the bulk of the work getting it incorporated. I’ve been trying to do this kind of research for about a decade, starting with Libraries.io in 2015 as a side project indexing every package registry I could find, which was acquired when the hosting bills outgrew what I could cover on my own. ecosyste.ms is the second attempt at the same idea, kept running by a sequence of grants and contracts that each cover a year or two of server costs, with most of my own time on it either unpaid or subsidised by other contracting work. The organisations set up to address open source sustainability have the same funding problem as the maintainers they are trying to help, often a worse version of it because a maintainer at least produces software that people want to use, and almost everyone I know doing this kind of research is patching together an income the same way I am. Grant programmes fund a project for a fixed term and then stop, which is also roughly how the academic contracts work for researchers studying maintainer burnout, while the tools built to measure the ecosystem need servers and bandwidth that someone has to keep paying for year after year regardless. The funding pattern also pushes the research towards one-off snapshots rather than systems that keep collecting and reporting, so each new grant redoes much of the same data collection from scratch, and a large part of why ecosyste.ms exists is to be a continuously running layer underneath that later work can build on. I’ve watched a fair number of good efforts wind down when a grant ended or a sponsor changed priorities, and the questions they were working on are all still open. The Lab is an attempt to give that work an institutional home with a horizon longer than a single grant cycle, structured as a research lab that contracts researchers, publishes open-access papers and open datasets, and builds tools that stay open source, on the basis that instruments for monitoring critical infrastructure are themselves critical infrastructure and shouldn’t sit behind a licence fee. Work is also under way with solicitors to register it as a charity, which if granted would make it the first UK charity working on the sustainability of the open source ecosystem, and would mean a regulator accepting that as a charitable purpose in its own right. The initial research themes are supply chain security, maintainer wellbeing, funding models, and governance, which overlap heavily with what I’ve been writing about here for the past year, and a chunk of that writing is already syndicated on the Lab’s site alongside work from the others. I’ve been reading and citing the rest of the board for years, which is a large part of why I said yes: Miranda Heath, a psychologist at the University of Edinburgh, wrote the most thorough study of burnout in open source I’ve come across, and Dawn Foster brings a PhD in software metrics along with board seats at CHAOSS and OpenUK. Daniel Roe leads Nuxt, Matias Capeletto is a core Vite developer, and the two of them are behind npmx, which I wrote about back in April, so the board includes people who maintain a sizeable share of the modern JavaScript toolchain and know first-hand what being on the receiving end of sustainability advice is like. Mike McQuaid, who I worked with at GitHub and now maintain Homebrew alongside, is advising, and I’m lucky to be working with that group on this. The stated long-term goal in the pitch document is to raise enough recurring funding to offer open-ended stipends to people doing this research, so the work can outlast whichever grant happens to be paying for it in a given year, and after a decade without anything like that available it’s the part I most want to see exist. That funding will come from company sponsorships and individual contributions through Open Collective, and there’s a Discord open to anyone working on these problems who wants to compare notes.

The 2026 Ruby on Rails Community Survey is now open. Add your voice to the community data. Brought to you by Planet Argon.

Comments

Building tools that talk to package registries means making thousands of requests to npm, RubyGems, PyPI, and crates.io while you iterate, and at some point you start to feel bad about it. Recorded HTTP fixtures go stale, mocking sixteen different registry protocols by hand is its own project, and pointing the test suite at the real thing means every red-green cycle is a few hundred more requests to infrastructure other people are paying for. I wanted something I could point git-pkgs and brief at locally that would answer like the real registries do but only fetch from upstream once. proxy is a single Go binary that speaks the wire protocols of npm, PyPI, RubyGems, Cargo, Go modules, Maven, NuGet, Composer, Hex, pub.dev, Conan, Conda, CRAN, Debian, RPM, and the OCI container registry. Start it, point a package manager at `localhost:8080`, and the first install fetches from upstream and writes the artifact to local storage; every install after that is served from the cache. Metadata responses are rewritten on the way through so tarball URLs point back at the proxy rather than the origin, which is the part most simple HTTP caches get wrong. proxy & npm_config_registry=http://localhost:8080/npm/ npm install GOPROXY=http://localhost:8080/go,direct go build pip install --index-url http://localhost:8080/pypi/simple/ requests That’s all I originally wanted from it, a local set of registry endpoints I could hammer from tests without bothering anyone. It’s still a side project for me, mostly a test bed where I can try package-registry experiments against real protocol handlers without first convincing sixteen upstream teams to ship them. But sitting between a package manager and its registry turns out to be a useful place to stand, and once those handlers existed it was also most of the way to being a free, single-binary alternative to Artifactory or Nexus for people who just want a caching mirror without the rest of the platform attached. ### CI The dependency caching built into GitHub Actions and its equivalents works at the filesystem level: tar up `~/.npm` or `~/.cargo`, key it on a hash of the lockfile, restore it next time. That’s fine until the lockfile changes by one package and the whole cache key misses, or a matrix build spreads the same dependency set across six OS and runtime combinations that each get their own tarball. A registry-protocol cache sits one level up, keyed on package coordinates rather than filesystem layout, so `lodash-4.17.21.tgz` is stored once and served to every job that wants it regardless of what else changed in the lockfile or which runner is asking. Point it at S3 or Postgres instead of the default SQLite-and-local-disk and it can be shared across runners. Longer term I’d like to see this wired directly into something like Forgejo’s CI runner, so every job on an instance gets a shared package cache and a cooldown policy by default rather than every repo having to configure it. ### Mirroring The proxy can be told to fetch packages before anything asks for them. `proxy mirror` takes PURLs, or a CycloneDX or SPDX SBOM, and pulls every listed artifact into the cache: proxy mirror pkg:npm/[email protected] pkg:cargo/[email protected] proxy mirror --sbom sbom.cdx.json Feed it the SBOM for a repository and you have an offline mirror of exactly that project’s dependency tree, which is the shape you want for air-gapped builds or for warming a cache before a CI fleet starts pulling. The same operation is exposed as `POST /api/mirror` on the running server for driving it from a pipeline. ### Cooldowns Because the proxy is rewriting metadata responses anyway, it can also edit them. The `cooldown` setting strips any version younger than a configured age from the version lists it returns: cooldown: default: "3d" ecosystems: npm: "7d" packages: "pkg:npm/lodash": "0" With that config, a version published to npm an hour ago doesn’t exist as far as anything behind the proxy is concerned, and won’t for a week. I wrote about why I think cooldowns are the single most effective supply-chain control most projects aren’t using; the short version is that almost every malicious-package incident is caught within a day or two of publish, so a build that can’t see anything younger than three days was never exposed in the first place. A few package managers have grown a native setting for this since I wrote that post, but doing it at the proxy means one config covers every ecosystem at once, including the ones that haven’t. ### Web UI There’s a web UI on `/` for browsing what’s in the cache: packages by ecosystem, hit counts, size, a source browser for reading files inside cached tarballs without extracting them, and a diff view for comparing two cached versions of the same package file by file. The enrichment API behind it (`/api/package/{ecosystem}/{name}/{version}`) returns licence, publish date, latest version, and any OSV advisories for a given coordinate, which is the same lookup git-pkgs needs, so the two share that code. There’s a lot more I want the UI to do, most of it on the list of things worth stealing from npmx: bundle composition, install-size sunbursts, typosquat warnings. A registry frontend that only shows you packages you’ve actually installed is a slightly different design problem from one that fronts all four million packages on npm, and I haven’t fully worked out what that should look like yet. ### Next Upstream merging would put an internal index and the public registry behind one URL, with the internal names shadowing the public ones, which is the dependency-confusion defence that pip in particular has no native answer for. I’d also like the proxy to enforce dependency policy more broadly than just cooldowns (licence allowlists, blocked package names, version floors), though there’s no shared format for writing those policies down, so whatever config the proxy grows will be yet another one, and I’d rather that problem got solved upstream of any individual tool. The experiment I’m most looking forward to is a wrapper mode, `proxy npm install express`, where the binary finds or starts a server, sets `NPM_CONFIG_REGISTRY` or `GOPROXY` or `PIP_INDEX_URL` as appropriate, and execs the underlying command. Add `alias npm="proxy npm"` to your shell and every install on your machine is cached and cooled down without ever touching an `.npmrc`. Most of the heavy lifting is in modules shared with the rest of the git-pkgs tooling: the manifest and lockfile parsers, the PURL handling, the SBOM readers, the OSV client. Each new registry backend is a few hundred lines of protocol handler on top of that, plus a config snippet for the install page. Contributions are very welcome on this one, particularly protocol handlers for the registries still unticked in the README (Helm, Swift, Alpine, Arch) and anything that makes the browse UI more useful. Tell me what you’d want it to do on Mastodon or the issue tracker. `brew install git-pkgs/git-pkgs/proxy` / github.com/git-pkgs/proxy

Apply for a Debugger Engineer job at Apple. Read about the role and find out if it’s right for you.

Recently, the application of AI tools to Erdos problems passed a milestone: an Erdos problem (#728 https://www.erdosproblems.com/728) was solved more or less autonomously by AI (after some feedback from an initial attempt), in the spirit of the problem (as reconstructed by the Erdos problem website community), with the result (to the best of our knowledge) not replicated in existing literature (although similar results proven by similar methods were located). This is a demonstration of the genuine increase in capability of these tools in recent months, and is largely consistent with other recent demonstrations of AI using existing methods to resolve Erdos problems, although in most previous cases a solution to these problems was later located in the literature, as discussed in https://mathstodon.xyz/deck/@tao/115788262274999408 . This particular case was unusual in that the problem as stated by Erdos was misformulated, with a reconstruction of the problem in the intended spirit only obtained in the last few months, which helps explain the lack of prior literature on the problem. However, I would like to talk here about another aspect of the story which I find more interesting than the solution itself, which is the emerging AI-powered capability to rapidly write and rewrite expositions of the solution. (1/5)

Debunking an alarmingly common HTML misconception.

Comments

A lean mean io_uring machine. Contribute to digital-fabric/uringmachine development by creating an account on GitHub.

Der Vortrag gibt einen Einblick in die verschiedenen Generationen von Flippern und deren Technik. Angefangen von elektromechanischen Geräten aus den frühen Sechzigern, über erste Prozessorsteuerungen, bis hin zu modernsten computergesteuerten Auto...

Git Source Code Mirror - This is a publish-only repository but pull requests can be turned into patches to the mailing list via GitGitGadget (https://gitgitgadget.github.io/). Please follow Documen...

This specification defines the QUERY method for HTTP. A QUERY requests that the request target process the enclosed content in a safe and idempotent manner and then respond with the result of that processing. This is similar to POST requests but can be automatically repeated or restarted without concern for partial state changes.