How to batch writes without losing per-item error detail#

A client posts fifty order lines to your batch endpoint. Forty-eight are written, one has a negative quantity, and one repeats a line sent an hour ago. The endpoint has one status code and one body for all fifty, and whatever it answers, the client either reports success for the bad line or sends the forty-eight good ones again.

What you get

You will end up with a POST /items/batch handler that answers one result per input under an errors flag, and a client that retries only what did not land. This is for you if your API accepts an array of writes and callers must learn which ones failed.

Short answer

Return one result per input item, in input order. Each carries the id the client sent, an HTTP status of its own, and either the created resource’s location or an RFC 9457 problem. Put an errors flag at the top so a client knows whether to read on. Keep the batch non-atomic by default, offer atomic as a request option, and have the client resend only the items the server did not attempt, by id.

You will need

Node 22 or later, an endpoint that creates one item, and an error format for a single failure, here RFC 9457 problem details. Verified 2026-10-21 against Node 22.22.0. The sample is a node:http server and a fetch client in one process, with tests on the Node test runner, and it installs no package. The store is a Map, which keeps the batch semantics visible and the database out of the way.

Approaches compared

The two families differ in what a client does after a failure. Under partial success, the client reads the results, fixes the failed items, sends the unattempted ones again, and the rest stay written. The cost is a body every client must parse, which Elasticsearch and Graph leave to the reader.¹ Under all or nothing, one bad item sends the whole batch back, which JSON:API requires and DynamoDB offers as a transaction. The cost is throughput: a batch of fifty with one bad row lands no rows until somebody fixes it.

Shape one result per input, in input order

The handler validates each item on its own, writes the good ones, and answers in the order it was asked, so position and id both identify a result. Every result carries the client’s id, an HTTP status for that item, and either a location or a problem.

  // Partial success: every item gets the outcome the same write would earn on its own.
  function partial(items) {
    let attempted = 0
    return items.map((item) => {
      const bad = validate(item) ?? conflict(item)
      if (bad) return { id: typeof item?.id === 'string' ? item.id : null, status: bad.status, problem: bad }
      if (store.has(item.id)) return exists(item)
      if (attempted >= budget) return notAttempted(item)
      attempted++
      return write(item)
    })
  }

The per-item status is the one the same write would earn as a single request. A new row earns 201, a replay of a stored item earns 200, and an item that needs fixing earns 422 with a problem. A stored id sent with different content earns 409, because a 200 would tell the client that its changed content landed. The last outcome, 503 with a not-attempted problem, is DynamoDB’s UnprocessedItems in HTTP clothing: the server ran out of budget for this request and wants the item back unchanged. A client has to tell that apart from a 422, because one is resent as it is and the other is not resent until a person changes it.

    const items = body?.items
    if (!Array.isArray(items) || items.length === 0) return refuse(res, problem('items-required', 'items must be a non-empty array', 422))
    if (items.length > cap) return refuse(res, problem('batch-too-large', `A batch holds at most ${cap} items`, 422, { max: cap, received: items.length }))
    const repeats = repeatedIds(items)
    if (repeats.length) return refuse(res, problem('repeated-id', 'An id appears more than once in this batch', 422, { repeats }))
    const results = body.atomic === true ? atomic(items) : partial(items)
    send(res, statusFor(results), { errors: results.some((r) => r.status >= 400), results })

Three things refuse the whole request before any item is looked at: a missing or empty array, more items than the cap, and an id that appears twice. The repeated id is refused rather than reported per item because a result keyed by that id would be ambiguous, which is the rule Graph applies with a 400.² Everything else is answered item by item, under an errors flag that says whether anything is not 2xx.

/** 200 when every item landed, 207 when some did, otherwise the one status that fits them all. */
export function statusFor(results) {
  const landed = results.filter((r) => r.status < 300).length
  if (landed === results.length) return 200
  if (landed > 0) return 207
  return results.every((r) => r.status === 503) ? 503 : 422
}

The batch status is derived from the items, the rule the page on reporting partial failure argues at length. It is 200 when every item landed, 207 when some did, and a 4xx or 5xx when none did. 207 is in the 2xx class, so res.ok is true for it,³ and the flag exists for the client that reads nothing else.

node demo.mjs

a batch of six: four new, one invalid, one already stored, to a store that writes 3 a request
  POST /items/batch -> 207, errors true, 6 items
  POST /items/batch -> 200, errors false, 1 item
  ln_a  201  /items/it_2
  ln_b  422  qty must be a positive integer (item ln_b sent qty -2)
  ln_c  201  /items/it_3
  ln_d  200  /items/it_1
  ln_e  201  /items/it_4
  ln_f  201  /items/it_5
  created 4, already stored 1, failed 1, not attempted 0
  came back 503 and was resent unchanged: ln_f

Six inputs, six results in input order. ln_d was stored before the batch and answers 200 with the location it already had, so a replay costs no second row. ln_f fell past the budget of three writes, came back 503, and landed on the second request without anyone touching it.

Resend only what did not land

The client chunks to the cap, reads every result, and sorts them into bins by status.

/** Every item sent gets one result under its own id. A repeated or invented id hides a lost item. */
function checkIds(chunk, results) {
  // The server answers an item without a string id under null, so null stands in for it.
  const missing = chunk.map((item) => (typeof item?.id === 'string' ? item.id : null))
  const unexpected = []
  for (const result of results) {
    const at = missing.indexOf(result?.id)
    if (at === -1) unexpected.push(result?.id)
    else missing.splice(at, 1)
  }
  if (missing.length || unexpected.length) {
    const ids = (list) => list.map((id) => JSON.stringify(id) ?? 'no id').join(', ') || 'none'
    throw new Error(`results do not match the ids sent: missing ${ids(missing)}, unexpected ${ids(unexpected)}`)
  }
}

export async function writeAll(base, items, { cap = 100, attempts = 4, sleep = (ms) => new Promise((r) => setTimeout(r, ms)) } = {}) {
  // The server refuses an id repeated in one request, but chunking can put two copies in two
  // requests, where the second would read as already stored. Refuse it before anything is sent.
  const byId = new Map()
  for (const item of items) {
    if (typeof item?.id === 'string' && byId.has(item.id)) throw new Error(`id ${item.id} appears more than once`)
    byId.set(item?.id, item)
  }
  const outcome = { created: [], exists: [], failed: [], unprocessed: [], retried: [], requests: [] }
  let pending = [...items]
  for (let attempt = 1; pending.length > 0; attempt++) {
    const retry = []
    for (const chunk of chunks(pending, cap)) {
      const res = await fetch(`${base}/items/batch`, {
        method: 'POST',
        headers: { 'content-type': 'application/json' },
        body: JSON.stringify({ items: chunk }),
      })
      const body = await res.json()
      if (!Array.isArray(body.results)) throw new BatchRefused(res.status, body)
      // One result per input: a short answer would drop items without a trace.
      if (body.results.length !== chunk.length) throw new Error(`sent ${chunk.length} items and got ${body.results.length} results`)
      // The right count is not enough: results for a and a, after sending a and b, lose b.
      checkIds(chunk, body.results)
      outcome.requests.push({ status: res.status, errors: body.errors, items: chunk.length })
      for (const result of body.results) {
        if (result.status === 201) outcome.created.push(result)
        else if (result.status === 200) outcome.exists.push(result)
        // Only a result with an id names an item to resend; one without stays with the failures.
        else if ((result.status === 503 || result.status === 429) && typeof result.id === 'string') {
          retry.push(byId.get(result.id))
          if (!outcome.retried.includes(result.id)) outcome.retried.push(result.id)
        }
        else outcome.failed.push(result)
      }
    }
    pending = retry
    if (pending.length === 0 || attempt >= attempts) break
    await sleep(backoff(attempt))
  }
  outcome.unprocessed = pending
  return outcome
}

A 503 goes back on the pile unchanged for the next attempt, after a doubling wait. A 422 goes to the caller with its problem, because sending it again unchanged would fail again. byId is what the client-supplied id buys: the result names the item, so the client can find and resend it without position bookkeeping. A whole-batch refusal arrives as a problem document with no results, and the client raises rather than guessing. It also raises when a response holds fewer results than the chunk had items. It refuses an id that appears twice in its input before sending anything, because chunking could put the two copies in different requests.

The right count under the wrong ids loses items too: results for a and a, after sending a and b, drop b and resend a twice. checkIds matches the results to the chunk’s ids, one each, and raises an error naming the missing and unexpected ids before the client sorts any result or sends anything again. A 503 or 429 without an id stays with the failures, because the client cannot tell which item to resend.

node demo.mjs

fix the failed item and resend it alone, by its id
  POST /items/batch -> 200, errors false, 1 item
  ln_b  201  /items/it_6
  store holds 6 rows after 6 writes: it_1 it_2 it_3 it_4 it_5 it_6

The failed line is resent alone after its quantity is corrected, and the store ends with six rows after six writes. Nothing landed twice, which is the property a batch client has to prove rather than assume.

Offer atomic as an option, not the default

Some callers need all or nothing: two lines that must exist together, or a store with a transaction to spend. atomic: true validates everything first and writes nothing if any item fails.

  // All or nothing: validate everything first, write nothing if any item fails, and tell the
  // good items why they were not applied, the way Graph answers a dependent with 424.
  function atomic(items) {
    const problems = items.map((item) => validate(item) ?? conflict(item))
    if (problems.some(Boolean)) {
      return items.map((item, i) => problems[i]
        ? { id: typeof item?.id === 'string' ? item.id : null, status: problems[i].status, problem: problems[i] }
        : { id: item.id, status: 424, problem: problem('not-applied', 'Not applied because another item in this batch failed', 424) })
    }
    const fresh = items.filter((item) => !store.has(item.id))
    if (fresh.length > budget) return items.map(notAttempted)
    return items.map((item) => (store.has(item.id) ? exists(item) : write(item)))
  }

The good items are told why with a 424. RFC 4918 defines that code for a request that failed because another one did, and Graph uses it for a dependent of a failed request. DynamoDB’s TransactWriteItems answers the same situation with one CancellationReasons entry per item.⁴ Either way the caller learns which item sank the batch.

node demo.mjs

an atomic batch with the same bad item
  POST /items/batch -> 422, errors true
  ln_g  424  Not applied because another item in this batch failed
  ln_h  422  qty must be a positive integer (item ln_h sent qty 0)
  store still holds 6 rows

Check it worked

The tests count the server’s writes: results can be misread, rows in the store cannot.

node --test batch.test.mjs

1..16
# tests 16
# suites 0
# pass 16
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 340.608041

# pass 16 is the line to look for. The sixth test pins the resend. A cap of four and a budget of three against six items make three requests, [207, 4], [200, 2], and [200, 1], and the store counts six writes, so nothing was written twice.

When it goes wrong

The client reported success and a row is missing. The client checked res.ok and stopped, and res.ok is true for a 207.

node pitfall.mjs

a client that reads res.ok and stops
  status 207, res.ok true, so it reports: all 3 written
  body.errors true, results not 2xx: b 422
  store holds 2 of 3

Read errors before anything else, and when it is true, walk the results. The status code cannot carry per-item truth once some items have landed, because any status that admits success is a 2xx.

The batch endpoint is slow for everyone. The server answers all or nothing by default, so a batch with one bad row lands no rows and the client has to pull the row out and send the batch again.

node pitfall.mjs

1000 items, 10 invalid, batches of 50
  partial success  requests 20  landed after round 1: 990  failed: 10  resent after removing them: 0  landed in the end: 990
  all-or-nothing   requests 30  landed after round 1: 500  failed: 10  resent after removing them: 490  landed in the end: 990

Ten bad rows in a thousand cost the atomic shape ten extra requests and held back 490 good rows for a round trip. The partial shape landed the same 990 rows in one round and reported the ten by id.

The client retries a 422 three times and gets three more. Only a 503 or a 429 means resend unchanged; a 4xx means the item is wrong and will stay wrong until something about it changes. Sort by status before deciding what to resend, as writeAll does.

When not to do this

Do not make atomic the default for independent items. A thousand order lines with one bad quantity are not a transaction, and treating them as one holds back 999 good rows until a person fixes a number. Offer atomic to the callers who ask for it, and make them ask per request.

Do not answer 200 with an errors flag alone to clients you do not control. The flag is the right signal for a client that reads it, and the status code is the only signal for a client that does not. Send both: 207 when some items landed and a 4xx when none did.

Do not retry a 4xx item unchanged. The problem document says what is wrong with it, and a retry loop that treats every non-2xx the same way turns a validation error into three validation errors and a wait. Reserve the retry pile for the statuses that mean the server did not try.

Related how-tos

• Report partial failure in a batch endpoint

• Return validation errors a client can map to a form

• Retry fetch calls with exponential backoff in Node.js

• Import a CSV file through a bulk ingest job from a script

Last verified

Verified 2026-10-21 against Node 22.22.0. Every output block is what the command preceding it printed, against the server in the code directory on 127.0.0.1. The Elasticsearch, Graph, JSON:API, and DynamoDB behaviors are read from their documentation, linked where each is named, and were not exercised against those services.