3 posts tagged with "CI"

A monorepo of forty packages that each report their own results produces a specific kind of unusable CI. The check is green and nobody believes it. The logs are thorough and nobody reads them. And the one question a reviewer actually has — "did this branch break that test, or was it already broken?" — has no answer anywhere in the run, because answering it needs a comparison nobody made.

Blong's answer is one document. Per package it carries the counts, the duration, the coverage and the change against the base branch; for the run it carries the tests that did not run, the slowest test, and every failure with its provenance. That same document is the artefact and the pull-request comment, so there is nothing to reconcile.

There is a class of bug that no amount of testing discipline catches, because it is not in your code. MySQL under concurrent writes takes a lock nobody expected; a broker rebalances while you are consuming; a token refresh races the request that needed it. A mock will never find these, and a simulator will never find these — they exist only where two real systems meet.

The usual answer is a shared staging environment, which is either stale, contended, or both. This is how Blong runs those tests instead: provision the real back ends into a cluster that lives exactly as long as the job, publish their ports to the runner, and let the suites connect as if the services were local.

Releasing a framework is a different job from releasing an application. An application ships once and the version belongs to a deployment; a framework is consumed by other people's builds, so every published version is a promise that has to hold for strangers, and the promise has to be attributable when it breaks. The repository has to answer three questions without a human remembering anything: what changed, what version does that make, and can a consumer prove the artifact came from this commit.

In January we replaced a manual, change-file-driven release with a pipeline where the version is a consequence of the merge and the artifacts are attested. The commit that did it is called fix: prepare for publishing, and it deleted more than it added.