How to recover when one registry upload fails mid release#

Your release job published the npm, PyPI, and RubyGems packages for 1.4.0, then crates.io answered 503 and the job died. The tag went out first, so Go and PHP consumers already resolve a version that is missing from one registry. The rerun fails sooner than the first run: npm refuses to publish over an existing version, so the job stops at its first step.

What you get

You will end up with a publish step that asks each registry what it holds, skips matching uploads, stops at a real failure, and tags last. This is for you if one release feeds several registries and a rerun is your recovery plan.

Short answer

Make each publish step idempotent. Ask the registry whether the version is there, compare its digest with the artifact you built, and skip it when they match. An “already published” refusal means read and compare again, never success. Give each registry its own CI job so a rerun repeats only the failed one, and push the tag last. A wrong artifact that landed cannot be replaced, so burn the version and ship the next patch everywhere.

You will need

Node 22 or later, a release that sends one version to more than one registry, and credentials for each. Verified 2026-10-21 against Node 22.22.0. The sample runs against a stand-in for six registries on 127.0.0.1, which answers with the paths, status codes, and refusal texts of the real ones, so it publishes nothing. The upload itself stays with the tool you already run, npm publish, twine upload, gem push, or cargo publish; the sample replaces the loop around it.

Approaches compared

The skip check and the per-registry jobs are not rivals: the jobs limit how much a rerun repeats, and the check makes whatever it repeats harmless. The Central Portal bundle costs you its scope, one registry out of six. Burning the version costs a number consumers see but cannot install, and it is the only choice left once wrong bytes have landed, because no registry here replaces them.

Ask the registry before you upload

Every registry here has a read API that says whether a version exists, and four of the six also say which bytes it holds. registries.mjs records the question and the answer for each one: the path to ask, the digest field, the matching local digest, and the refusal sent for a repeat.

// alreadyThere is true when the answer to an upload means "this version is held". None
// of them says whose bytes are held, so release.mjs reads again and compares digests.
export const REGISTRIES = {
  npm: {
    kind: 'upload',
    // GET /<name>/<version> is 200 with a dist object, or 404 "version not found".
    existsPath: (pkg, version) => `/${encodeURIComponent(pkg.name)}/${version}`,
    read: (status, body) => (status === 200 ? { exists: true, digest: JSON.parse(body).dist.integrity } : { exists: false }),
    localDigest: sri512,
    uploadMethod: 'PUT',
    uploadPath: (pkg) => `/${encodeURIComponent(pkg.name)}`,
    // A second publish of a version is refused with 403, whatever its bytes.
    alreadyThere: (status, body) => status === 403 && /previously published/.test(body),
  },
  pypi: {
    kind: 'upload',
    // GET /pypi/<project>/<version>/json is 200 with urls[].digests, or 404. A release
    // can hold several files, so one that lacks this file has not got it yet.
    existsPath: (pkg, version) => `/pypi/${pkg.name}/${version}/json`,
    read: (status, body, pkg) => {
      if (status !== 200) return { exists: false }
      const file = JSON.parse(body).urls.find((u) => u.filename === pkg.artifact)
      return file ? { exists: true, digest: file.digests.sha256 } : { exists: false }
    },
    localDigest: sha256,
    uploadMethod: 'POST',
    uploadPath: () => '/legacy/',
    // Warehouse answers an identical file with 200 and the same file name with other
    // bytes with 400, putting the message in the status line, where twine reads it.
    alreadyThere: (status, body, reason) => status === 400 && /already exist/i.test(`${reason} ${body}`),
  },

The npm registry API answers GET /<name>/<version> with that version’s dist object, and npm publish sends a sha512 integrity field with every tarball, so the comparison is an SRI string. PyPI’s JSON API lists each file of a release with its sha256. The RubyGems.org API returns one version with its sha, and the crates.io sparse index returns one JSON line per version with a cksum of the .crate file. The two tag-driven registries answer yes or no: the Go module proxy serves @v/<version>.info once it has seen the tag, and Packagist lists every tagged version in p2/<vendor>/<package>.json.

Each refusal says the version is held without saying whose bytes it holds. npm answers any repeat with 403, crates.io with a 400 carrying an errors array. Warehouse, the code behind PyPI, accepts an identical file with 200 and refuses the same name with other bytes with a 400 whose message rides in the HTTP status line.¹ twine’s --skip-existing skips on that 400, so it skips the one upload whose bytes differ. RubyGems.org answers 200 when the same key pushes the same bytes, and 409 otherwise.²

Skip what landed, stop at what failed, tag last

release.mjs is the loop a CI job runs. For each upload target it probes, compares, and either skips or uploads. A refusal meaning already there sends it back to read and compare. Any other failure ends the run, a failed read included, since only a 404 or 410 says absent. The tag goes last, because for Go and Packagist the tag is the publication.

    const checkBytes = (there) => {
      if (there.digest !== spec.localDigest(bytes)) {
        throw new ReleaseError(`${pkg.registry} holds ${pkg.name} ${plan.version} with different bytes`, { burn: true })
      }
    }

    const there = await probe(spec, bases[pkg.registry].read, pkg, plan.version, fetch)
    if (there.exists) {
      checkBytes(there)
      log(row(pkg, plan.version, 'already there, skipped'))
      continue
    }

    const url = `${bases[pkg.registry].write}${spec.uploadPath(pkg)}?version=${plan.version}&file=${pkg.artifact}`
    const res = await fetch(url, { method: spec.uploadMethod, body: bytes })
    const body = await res.text()
    if (spec.alreadyThere(res.status, body, res.statusText)) {
      // The refusal says the version is held, not whose bytes it holds. Ask again.
      const again = await probe(spec, bases[pkg.registry].read, pkg, plan.version, fetch)
      if (!again.exists) {
        throw new ReleaseError(`${pkg.registry} says ${pkg.name} ${plan.version} is there, but its read API does not show it yet; rerun once it does`)
      }
      checkBytes(again)
      log(row(pkg, plan.version, `already there (${res.status} on upload), same bytes`))
      continue
    }
    if (!res.ok) throw new ReleaseError(`${pkg.registry}: ${pkg.name} ${plan.version} failed, ${res.status} ${body.split('\n')[0]}`)
    log(row(pkg, plan.version, 'published'))
  }

  if (!tagFirst) await pushTag()

Run it twice against the stand-in. The first run is told to fail at crates.io once.

node demo.mjs

run 1: release 1.4.0, crates.io refuses its upload once
  npm       @acme/sdk 1.4.0           published
  pypi      acme-sdk 1.4.0            published
  rubygems  acme_sdk 1.4.0            published
  stopped: crates: acme-sdk 1.4.0 failed, 503 Service Unavailable
  serving 1.4.0: npm yes, pypi yes, rubygems yes, crates no, goproxy no, packagist no
run 2: the same release again
  npm       @acme/sdk 1.4.0           already there, skipped
  pypi      acme-sdk 1.4.0            already there, skipped
  rubygems  acme_sdk 1.4.0            already there, skipped
  crates    acme-sdk 1.4.0            published
  tag v1.4.0 pushed: goproxy and packagist now serve 1.4.0
  serving 1.4.0: npm yes, pypi yes, rubygems yes, crates yes, goproxy yes, packagist yes
upload attempts: npm 1, pypi 1, rubygems 1, crates 2 (one refused)

The first run stopped before the tag, so the two tag-driven registries served nothing and no consumer saw a half release. The second run made one read per landed package and uploaded to crates.io alone. The attempt counts on the last line are the property to assert: no registry received a second successful upload.

Give each registry its own job

The skip check makes a rerun harmless, and per-registry jobs make it short. release.yml builds once, hands the artifacts to a matrix of publish jobs, and tags in a job that needs all of them.

  publish:
    needs: build
    runs-on: ubuntu-latest
    permissions:
      contents: read
      id-token: write
    strategy:
      fail-fast: false
      matrix:
        registry: [npm, pypi, rubygems, crates]
    steps:
      - uses: actions/checkout@v6
        with:
          persist-credentials: false
      - uses: actions/download-artifact@v5
        with:
          name: dist
          path: dist
      - run: node release/publish.mjs ${{ matrix.registry }} "$(cat VERSION)"

  tag:
    needs: publish
    runs-on: ubuntu-latest
    permissions:
      contents: write
    steps:
      - uses: actions/checkout@v6
      - run: |
          git tag "v$(cat VERSION)"
          git push origin "v$(cat VERSION)"

needs holds the tag job back, because a job that fails or is skipped skips every job that needs it. fail-fast: false matters because the default cancels every in-progress job in the matrix when one fails, which turns one failed upload into several interrupted ones. The artifact from build is what every publish job uploads, so a rerun publishes the bytes the first run built rather than a rebuild. GitHub re-runs the failed jobs at the same commit; the skip check covers a full re-run.

The workflow grants the token nothing by default and each job asks for what it uses: only the publish jobs mint OIDC tokens, and only the tag job pushes. The build runs every dependency’s scripts with neither, and the checkouts that never push set persist-credentials: false.

Finish the version or burn it

A rerun that skips a landed artifact trusts it, and the digest comparison earns that trust. When the registry holds 1.4.0 with other bytes, the run stops and says so.

node burn.mjs

stopped: npm holds @acme/sdk 1.4.0 with different bytes
decision: burn 1.4.0, release 1.4.1 to all six
upload attempts after the stop: npm 0, pypi 0, rubygems 0, crates 0

Finish the version when every landed artifact matches what you built, because the registries allow nothing else. npm says a published name and version can never be used again, and PyPI does not allow a filename to be reused. Cargo calls a publish generally permanent, and Maven Central says a published component cannot be changed. Burn the version when any landed artifact is wrong, or when you cannot tell. Burning means 1.4.1 on all six registries, including the three that hold a correct 1.4.0, because consumers of a lockstep SDK expect one number everywhere.

Check it worked

Run the suite. Fourteen tests pin the two runs, the burn, the tag, each refusal while a read lags, and the workflow’s permissions.

node --test release.test.mjs

1..14
# tests 14
# suites 0
# pass 14
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 375.184785

# pass 14 is the line to look for. The fourth test is the one to keep if you cut the rest: the upload meets PyPI’s refusal for other bytes, and the run stops without the tag.

When it goes wrong

The tag went out before the uploads. Go and PHP consumers resolve the version the moment the tag exists, so a failure after the tag leaves them with a release the other registries deny:

node pitfall.mjs

tag first, then the uploads
  tag v1.4.0 pushed: goproxy and packagist now serve 1.4.0
  npm       @acme/sdk 1.4.0           published
  pypi      acme-sdk 1.4.0            published
  rubygems  acme_sdk 1.4.0            published
  stopped: crates: acme-sdk 1.4.0 failed, 503 Service Unavailable
  serving 1.4.0: npm yes, pypi yes, rubygems yes, crates no, goproxy yes, packagist yes
  v1.4.0 exists: true

Push the tag from a job that needs every publish job, as release.yml does. The module mirror’s own FAQ says a version requested before its tag was pushed can take up to 30 minutes to appear, because the mirror caches the miss.³

The probe says absent, but the upload says already there. Packagist’s API documentation admits that mirroring delays can hand you an outdated file, and other registries cache their read paths too. On PyPI that refusal means the bytes differ, so release.mjs reads again and compares, and stops without the tag while the read still lags.

The rerun rebuilt the artifacts and the digests differ. Tarballs, wheels, and gems carry file timestamps, so a rebuild on another day produces different bytes for the same source, and the burn verdict fires on a release that is fine. Publish the artifacts the first run built, as the shared dist artifact does, or make the build reproducible; SOURCE_DATE_EPOCH is the convention for pinning the timestamps a build embeds.

When not to do this

Do not run this against Maven Central as though it were one of the other five. The Central Portal takes one bundle, validates it as a unit, and either publishes all of it or marks the deployment FAILED, so a rerun there is a fresh deployment. The states its Publisher API exposes, PENDING, VALIDATING, VALIDATED, PUBLISHING, PUBLISHED, and FAILED, are the transaction the other registries lack.

Do not treat the skip check as permission to move a tag. For Go and Packagist a tag is the release, so a moved tag is a second release wearing the first one’s name.

Do not burn a version to avoid the digest comparison. A burned number stays in every registry’s history without ever becoming installable, and it needs a changelog entry that says so.

Do not skip on existence alone. Skipping without the digest is how a wrong tarball survives the rerun meant to fix the release.

Related how-tos

• Choose a release automation tool for an SDK repository

• Make queue consumer writes safe under redelivery

Last verified

Verified 2026-10-21 against Node 22.22.0. Every output block is what the command preceding it printed. The six registries are a stand-in on 127.0.0.1 whose paths, status codes, and refusal texts follow the documentation and source cited on this page, so no request reached a real registry. release.yml was read as a file rather than run on GitHub Actions, and the stand-in’s npm refusal text is the message the npm client prints, which no npm document states. The stand-in’s RubyGems.org keeps no API keys, so it answers every identical repeat as one long-lived key would.