How to load API data into Observable Framework with a data loader#

Your team wants the orders dashboard to show yesterday’s numbers from the billing API. The API wants a bearer key on every call, and the dashboard is a static site. The first draft put the key in a fetch call on the page, where anyone with the view-source shortcut can read it. The second draft took the key out, and the browser reports a CORS error for every call instead.

What you get

You will end up with a data loader that pulls a paginated API at build time, with a key the browser never sees, and a page that reads it as a file. This is for you if your dashboard’s API needs a key.

Short answer

Put a Node script at src/data/orders.json.js that reads the API URL and key from process.env, follows the pagination, logs to stderr, and writes one JSON document to stdout. Reference it from the page as FileAttachment("data/orders.json").json(). The build runs the loader, hashes the output into dist/_file, and ships the page without the key. A fetch in the page itself is always fresh, but every visitor receives the key.

You will need

Node 20 or later, Observable Framework 1.13, and an API key readable from an environment variable. Verified 2026-10-21 against Node 22.22.0 and @observablehq/framework 1.13.4. The sample pins that version in its package.json, so npm ci installs the one dependency, and it builds against a fake orders API on 127.0.0.1 that wants a bearer key and pages its results. The key is read with process.env, which the loader inherits from whatever shell or CI job runs the build.

Approaches compared

Freshness and secrecy pull in opposite directions. The loader runs where the key lives and produces a file, so the browser gets data without a credential, at the price of data that is as old as the last build. A fetch in the page is as fresh as the last reload, and whatever it needs to authenticate ships with the page. The notebook suits finding out what the data looks like,¹ and stops suiting the moment the file is shared with a key in it.

Write the loader where the key lives

A data loader is an ordinary program whose stdout becomes a file. This one reads the API URL and the key from the environment, follows the pagination, and writes the orders as one JSON array.

const base = process.env.API_URL;
const token = process.env.API_TOKEN;
if (!base || !token) {
  console.error("orders.json.js: API_URL and API_TOKEN must be set");
  process.exit(1);
}

const orders = [];
let next = "/orders?page=1";
while (next) {
  const response = await fetch(base + next, { headers: { authorization: `Bearer ${token}` } });
  if (!response.ok) {
    console.error(`orders.json.js: ${next} answered ${response.status}`);
    process.exit(1);
  }
  const page = await response.json();
  orders.push(...page.data);
  console.error(`orders.json.js: ${next} gave ${page.data.length} orders`);
  next = page.next;
}

process.stdout.write(JSON.stringify(orders));

Two rules shape it. Every log line goes to console.error, which is stderr, because Framework’s data loader documentation says that anything logged to stdout is included in the output file and sent to the client. And every failure ends in process.exit(1). The same page says a loader that fails must return a non-zero exit code, and that empty output is not by itself treated as an error. A loader that swallows a 401 and exits 0 publishes an empty dashboard with a green build beside it.

node demo.mjs

the loader on its own
  exit 0, stdout 527 bytes, 1 line
  stdout parses as JSON: 7 orders
  stderr: orders.json.js: /orders?page=1 gave 3 orders
  stderr: orders.json.js: /orders?page=2 gave 3 orders
  stderr: orders.json.js: /orders?page=3 gave 1 orders

Run on its own, the loader is a script like any other: three requests, one line of JSON on stdout, and the progress on stderr where the build log will show it.

Reference the file and let the build run the loader

The page asks for a file that does not exist in the source tree.

const orders = FileAttachment("data/orders.json").json();

Framework resolves the path relative to the page, finds no data/orders.json, and looks for a file of the same name with a second extension, which is how orders.json.js becomes the loader for orders.json.² The argument has to be a string literal, because the build finds loaders by reading the page rather than running it,³ and only files a page references this way are built. The config states the source root, which stops the build warning that root is missing, and names the output directory beside it.⁴

export default {
  root: "src",
  output: "dist",
  title: "Orders",
};

The demo clears the cache and the output, sets API_URL and API_TOKEN, and runs observable build as a child process with the build’s own log sent to stderr.

node demo.mjs

observable build
  pages: index.html, live.html
  data file: dist/_file/data/orders.79f00478.json, 7 orders
  index.html references _file/data/orders.79f00478.json: true
  cache written: true
  requests the API saw during the build: 3

The loader ran once, under the build, against the API: three requests, the whole of the build’s traffic to the API. Its output landed in dist/_file/data/ under a name that carries a content hash, so a changed snapshot gets a changed URL and a visitor’s cache cannot serve the old one. The same output was written to src/.observablehq/cache/. A second build would read it from there without running the loader, which the pitfall below turns on.

Prove the key is in neither the page nor the file

A second page in the sample does what the first draft did and fetches from the page itself, with a placeholder where a key would be.

const token = "sk_live_PLACEHOLDER_do_not_do_this";
const orders = await fetch("https://api.example.com/orders", {
  headers: { authorization: `Bearer ${token}` },
}).then((response) => response.json());

Framework compiles a page’s code blocks into the page’s HTML, so a literal in a block is a literal in dist/live.html. The loader’s key, by contrast, exists only in the process that ran the build.

node demo.mjs

where the token is
  "secret-orders-token" in dist/index.html: 0, in the data file: 0
  "secret-orders-token" in all 7 files under dist/: 0
  "sk_live_PLACEHOLDER" in dist/live.html: 1

The count is the verification. The key the build used appears nowhere in the output. The key the page used appears exactly where every visitor can read it. Pointed at the fake API, the second page would also meet CORS in a browser, because the fake sends no Access-Control-Allow-Origin header, and most internal APIs send none either. The loader runs under Node, where CORS does not apply.

Check it worked

The fourth test runs a clean build and asserts on the output tree rather than on the loader alone.

node --test loader.test.mjs

1..4
# tests 4
# suites 0
# pass 4
# fail 0
# cancelled 0
# skipped 0
# todo 0
# duration_ms 3388.53926

# pass 4 is the line to look for. The fourth test checks that dist/_file/data/ holds an orders.<hash>.json with all seven orders and that the cache was written. It then checks that the build’s token is in none of the files under dist and the placeholder is in live.html once.

When it goes wrong

The dashboard shows nothing and the build was green. A console.log in the loader went to stdout, which is the file.

node pitfall.mjs

console.error for progress (src/data/orders.json.js)
  stdout starts: "[{\"id\":\"ord_1001\",\"placed\":\"2026-03-02\","
  JSON.parse(stdout): 7 orders
console.log for progress (chatty-loader.mjs)
  stdout starts: "fetched /orders?page=1: 3 orders\nfetched"
  JSON.parse(stdout): SyntaxError: Unexpected token 'e'

The chatty loader exits 0, so the build caches and serves its output, and the page’s .json() call fails in the browser instead. Log to stderr, and test the loader’s stdout with JSON.parse before the build does.

The dashboard shows last week’s numbers after a build. Framework’s cache under src/.observablehq/cache is read during a build whenever the output is present, with no check of modification times. A build on a machine that kept the cache never runs the loader. Delete the cache before a build that must refresh, as build.mjs does, or keep the cache out of CI.

The build fails on a file the project never mentions. Inside a larger repository, the bundler walks up from the project looking for a tsconfig.json. This sample’s first build died on the site’s own, which extends a package the sample does not install. A tsconfig.json of the project’s own, even a small one, stops the walk there.

When not to do this

Do not use a loader for data that must be live or that differs per visitor. The file is a snapshot taken at build time, the same for everyone. A dashboard that promises the current minute needs a fetch at load time against an API designed to be called from a browser, with its own short-lived credentials.

Do not put a key in a page block because the API allowed your origin. CORS decides which pages may call the API; it does nothing to hide what those pages contain, and the key is in the HTML of every visitor who loads the page.

Do not let the notebook become the publication. The HTML notebook format is made for editing and sharing, and a cell that fetched with a key during exploration carries that key into every copy. Move the fetch into a loader before the notebook leaves your machine.

Do not let a loader exit 0 on a failure it noticed. An empty or partial file with a zero exit code is cached and served, and the build passes. Framework warns about an empty file in the build log but cannot tell a partial file from a complete one.

Related how-tos

• Call a keyed API from a browser without shipping the key

• Loop over every page of a REST collection

• Add a bearer token to fetch without a client library

Last verified

Verified 2026-10-21 against Node 22.22.0 and @observablehq/framework 1.13.4. Every output block is what the command preceding it printed, against the fake orders API in the code directory. The build resolves the versions of Framework’s own npm: imports over the network, and that is the only traffic that leaves the machine. The Notebook Kit row is read from its documentation and was not run, because its cells execute in a browser.