# How to move guides to a new docs toolchain without breaking URLs > Build the redirect map from your sitemap and your logs, crawl every old URL against the new build before the switch, and audit the anchors a redirect cannot carry. Source: https://voxgig.com/howto/migrate-a-guides-site-without-breaking-urls - Audience: platform-team - Level: advanced - Verified: 2026-10-21 - Published: 2026-10-19 ## Short answer Build the list of old URLs from the sitemap and the edge log, never from memory, and generate the redirect map from one table of page moves. Spell every destination exactly as the new sitemap does, trailing slash included, or the host adds a second redirect of its own. Crawl every old path against the new build before the switch, and audit the anchors separately, because the fragment after `#` never reaches the server. --- ## You will need A sitemap for the current site, ninety days of edge or server logs with the request path and its status, the page list of the new build, and Node 22 or later. Verified 2026-10-21 against Node 22.22.0 and github-slugger 2.0.0. The sitemap protocol lists what a site publishes, not what is requested,[^1] which is why the log is the second source. Google's [site move guidance](https://developers.google.com/search/docs/crawling-indexing/site-move-with-url-changes) asks for a mapping of old URLs to new ones, redirects with no chains, and the redirects kept for at least a year. ## Approaches compared | Approach | When it fits | What it costs you | When to pick something else | | --- | --- | --- | --- | | [Astro `redirects` config](https://docs.astro.build/en/guides/routing/#redirects) | A Starlight build on a host with no redirect file of its own, or an adapter that writes the host's file for you | A static build writes one HTML page per rule with a meta refresh, so a real 301 exists only when an adapter translates the config | The host reads a redirect file, which answers faster and is a real 301 | | [Cloudflare Pages `_redirects`](https://developers.cloudflare.com/pages/configuration/redirects/) | The new site is on Pages and the map fits in 2,000 static and 100 dynamic rules | One file per site, exact-match sources with a single splat, and a rule order that decides which rule wins without telling you | More than 2,100 rules, or a match on a header or a cookie | | [Docusaurus `plugin-client-redirects`](https://docusaurus.io/docs/api/plugins/@docusaurus/plugin-client-redirects) | You are moving between Docusaurus sites, or into one, and the host offers no redirects | An extra HTML page per old path that redirects with JavaScript, which the plugin's own page says to avoid when the host can do it | The host has a redirect file, or crawlers matter more than readers | | [In-page canonical tags](https://developers.google.com/search/docs/crawling-indexing/consolidate-duplicate-urls) | Both sites stay up for a while and you want search to prefer the new URL | A reader still lands on the old page, and Google treats the tag as a strong signal rather than an instruction | The old site is going away, which is the case on this page | | [mkdocs-redirects](https://github.com/mkdocs/mkdocs-redirects) | A Material for MkDocs build, with old pages named by their Markdown path | An HTML stub per old page, with a meta refresh and a script, written into the build, so the host answers 200 and the redirect happens in the browser | The host has a redirect file | | [nginx `return 301`](https://nginx.org/en/docs/http/ngx_http_rewrite_module.html#return) | You run the server, and the map can live in a `map` block regenerated with the site | A server to operate, and a config reload on every change to the map | A static host, where the redirect file is the only control you have | Edge rules win on speed and on what a crawler sees. The `_redirects` file answers before any asset is looked up, and costs you a 2,100-rule ceiling and an order you have to get right. The build-time stubs from Docusaurus and mkdocs-redirects run on any host and cost a page per old path that answers 200 before it redirects. A canonical tag costs the least to add, but it leaves the reader on the old page. ## Inventory the paths from the sitemap and the log The sitemap lists the pages you meant to publish, and the log lists the paths people still request. Four kinds of entry appear only in the log. A trailing-slash variant of a listed page, an `.html` suffix the host already redirects, a retired version, and a deleted page that a popular post still links to. `inventory.mjs` merges the two and keeps the hit count, so the map can be checked busiest page first. ```js title="inventory.mjs" export function inventory({ sitemap, log }) { const byPath = new Map() for (const path of sitemapPaths(sitemap)) { byPath.set(path, { path, inSitemap: true, inLog: false, status: null, hits: 0 }) } for (const row of logRows(log)) { const entry = byPath.get(row.path) ?? { path: row.path, inSitemap: false, inLog: false, status: null, hits: 0 } entry.inLog = true entry.status = row.status entry.hits += row.hits byPath.set(row.path, entry) } return [...byPath.values()].sort((a, b) => b.hits - a.hits || a.path.localeCompare(b.path)) } ``` ```bash node inventory.mjs ``` ```text output path source status hits /docs/auth/api-keys sitemap+log 200 18422 /docs/quickstart sitemap+log 200 9310 /docs/intro sitemap+log 200 7745 /docs/meters/record-a-reading sitemap+log 200 4120 /docs/invoices/webhooks sitemap+log 200 3908 /docs/getting-started log only 404 1208 answers 404 today and is still linked /docs/auth/oauth sitemap+log 200 1177 /docs/auth/api-keys/ log only 200 412 a trailing-slash variant of a sitemap URL /docs/reference/errors sitemap+log 200 390 /docs/meters/list-meters sitemap+log 200 366 /docs/invoices/overview sitemap+log 200 241 /docs/1.2/auth/api-keys log only 404 96 answers 404 today and is still linked /docs/intro.html log only 301 61 an .html suffix the host already redirects /docs/1.4/auth/api-keys sitemap+log 200 44 /docs/1.4/meters/list-meters sitemap+log 200 12 15 old paths: 11 from the sitemap, 4 only in the log ``` Four of the fifteen paths are in the log alone, and the deleted page among them took 1,208 requests in ninety days, more than six sitemap pages. Script-based analytics sees a visit only when the page runs its script, so count missing pages from the edge or server log instead. ## Generate the map from one table of moves Write down where each old page goes, once, as a page rather than a URL. The variants come from rules: strip the version prefix, drop the `.html` suffix, remove the trailing slash, and look the result up. A page renamed in the table is renamed for every URL that ever pointed at it. ```js title="redirects.mjs" export const MOVES = { '/docs/intro': '/guides/introduction/', '/docs/quickstart': '/guides/quickstart/', '/docs/getting-started': '/guides/quickstart/', '/docs/auth/api-keys': '/guides/authentication/api-keys/', '/docs/auth/oauth': '/guides/authentication/oauth/', '/docs/meters/list-meters': '/guides/meters/list-meters/', '/docs/meters/record-a-reading': '/guides/meters/record-a-reading/', '/docs/invoices/overview': '/guides/invoices/overview/', '/docs/invoices/webhooks': '/guides/invoices/webhooks/', '/docs/reference/errors': '/reference/errors/', } // Versions the new site does not carry. Their pages go to the current page. export const RETIRED_VERSIONS = ['1.2', '1.4'] // The old page an old URL names, with the version, the suffix and the slash // removed: /docs/1.4/auth/api-keys/ and /docs/auth/api-keys.html are both // /docs/auth/api-keys. export function canonicalOldPath(path) { let p = path.replace(/\.html$/, '').replace(/\/$/, '') for (const v of RETIRED_VERSIONS) p = p.replace(`/docs/${v}/`, '/docs/') return p || '/' } export const destinationFor = (oldPath) => MOVES[canonicalOldPath(oldPath)] ``` The destinations end in a slash because the new sitemap does. Starlight writes [`/guides/quickstart/index.html`](https://docs.astro.build/en/reference/configuration-reference/#buildformat) by default and so does MkDocs with [`use_directory_urls`](https://www.mkdocs.org/user-guide/configuration/#use_directory_urls), while Docusaurus wrote the same file and [left the slash off](https://docusaurus.io/docs/api/docusaurus-config#trailingSlash) the URL. Cloudflare Pages [adds the slash itself](https://github.com/cloudflare/workers-sdk/blob/main/packages/pages-shared/asset-server/handler.ts) when the file is there, with a 308 of its own, so a destination without it costs every reader a second hop. ```bash node map.mjs ``` ```text output mirror: 2 static and 2 dynamic rules (limits 2000 and 100) every old path reaches a page in one hop restructure: 15 static and 0 dynamic rules (limits 2000 and 100) every old path reaches a page in one hop wrote _redirects and redirects.nginx.conf, 15 rules each ``` The two lines are the trade-off in numbers. A mirror move keeps the old paths under the new tool and needs four rules, two splats for the retired versions and two static ones. The host's slash handling does the rest. The restructure needs a rule per old URL, fifteen here. Every future rename lengthens that file for as long as the old links exist, which Google puts at a year or more. The same pairs come out as an nginx `map` block for a server you run. ```text title="_redirects" /docs/auth/api-keys /guides/authentication/api-keys/ 301 /docs/quickstart /guides/quickstart/ 301 /docs/intro /guides/introduction/ 301 ``` ## Crawl the old paths against the new build The crawl asks the new site for every old path and follows what it answers, the way a browser would, counting the hops. Before the deployment exists, `server.mjs` stands in for it. The `_redirects` rules come first, because Cloudflare evaluates the file before it looks for an asset; then the built pages, then the host's handling of a slash or an `.html` suffix, then a 404. ```js title="server.mjs" const m = matchRule(path, rules) if (m) { res.writeHead(m.rule.code, { location: m.to }) return res.end() } if (pageSet.has(path)) { res.writeHead(200, { 'content-type': 'text/html; charset=utf-8' }) return res.end(renderPage(path, pages[path])) } const fixed = hostNormalize(path, pageSet) if (fixed) { res.writeHead(308, { location: fixed }) return res.end() } res.writeHead(404, { 'content-type': 'text/plain' }) res.end('not found\n') ``` ```bash node crawl.mjs ``` ```text output /docs/1.2/auth/api-keys 301 /guides/authentication/api-keys/ => 200 /docs/intro.html 301 /guides/introduction/ => 200 /docs/1.4/auth/api-keys 301 /guides/authentication/api-keys/ => 200 /docs/1.4/meters/list-meters 301 /guides/meters/list-meters/ => 200 _redirects: 15 old paths, 15 reach a page (15 in one hop, 0 chained), 0 end in 404 ``` The same script takes `--base https://preview.example` and crawls a deployment instead. It exits 1 when a path ends in a 404 or takes a second hop, so a CI job can hold the switch. ## Audit the anchors the redirect cannot carry A redirect moves a path. The fragment after `#` is never sent, because [RFC 3986](https://www.rfc-editor.org/rfc/rfc3986.html#section-3.5) separates it from the URI before the request is made, so no server rule can rewrite it. The browser does carry it across a redirect, by a rule in [RFC 9110](https://www.rfc-editor.org/rfc/rfc9110.html#section-10.2.2),[^2] and then looks for a heading with that id on the new page. Astro, and so Starlight, [derives heading ids](https://docs.astro.build/en/guides/markdown-content/#heading-ids) with github-slugger, and Docusaurus [wraps the same package](https://github.com/facebook/docusaurus/blob/main/packages/docusaurus-utils/src/slugger.ts). MkDocs uses Python-Markdown's [`toc` extension](https://python-markdown.github.io/extensions/toc/), which collapses the hyphens an ampersand leaves behind and numbers a repeated heading `_1` rather than `-1`. ```js title="anchors.mjs" export function pythonMarkdownSlug(text, separator = '-') { const ascii = text.normalize('NFKD').replace(/[^\x00-\x7f]/g, '') const cleaned = ascii.replace(/[^\w\s-]/g, '').trim().toLowerCase() return cleaned.replace(new RegExp(`[${separator}\\s]+`, 'g'), separator) } export function pythonMarkdownIds(headings) { const seen = new Set() return headings.map((h) => { let id = pythonMarkdownSlug(h) while (seen.has(id) || !id) { const m = /^(.*)_(\d+)$/.exec(id) id = m ? `${m[1]}_${Number(m[2]) + 1}` : `${id}_1` } seen.add(id) return id }) } ``` ```bash node anchors.mjs ``` ```text output anchor lands on Starlight MkDocs /docs/auth/api-keys#rotate-a-key /guides/authentication/api-keys/ missing missing /docs/auth/api-keys#scopes--permissions /guides/authentication/api-keys/ ok missing (scopes-permissions) /docs/1.4/auth/api-keys#create-a-key /guides/authentication/api-keys/ ok ok /docs/meters/record-a-reading#example-1 /guides/meters/record-a-reading/ ok missing (example_1) /docs/invoices/webhooks#retry-schedule /guides/invoices/webhooks/ ok ok /docs/getting-started#install-the-cli /guides/quickstart/ ok ok 6 inbound anchors: Starlight keeps 5, MkDocs keeps 3 ``` Starlight loses one anchor, because Rotate a key became Rotating a key, and MkDocs loses two more on text that did not change.[^3] The fix is the same in every toolchain: keep the heading text, or pin the id. Docusaurus takes [`{#rotate-a-key}`](https://docusaurus.io/docs/markdown-features/toc#heading-ids) after a heading and ships a `write-heading-ids` command that adds one to every heading before you move. Python-Markdown's [`attr_list`](https://python-markdown.github.io/extensions/attr_list/) extension takes `{: #rotate-a-key }`. Astro needs a `rehype` plugin for an explicit id. ## Check it worked ```bash node --test migrate.test.mjs ``` ```text output 1..14 # tests 14 # suites 0 # pass 14 # fail 0 # cancelled 0 # skipped 0 # todo 0 # duration_ms 756.462745 ``` Fourteen tests pin the shapes. The fifth feeds the first draft through the validator and asserts eleven shadowed rules and eleven chains, so a validator that stops catching either goes red. The crawl test asserts `oneHop: 15` against the stand-in, and the committed `_redirects` has to match what the generator writes. Another runs `map.mjs --check` and `crawl.mjs` on the first draft and expects exit status 1 from both. ## When it goes wrong Every path answers 404 or takes two hops. The first draft of this page's map put a catch-all splat on line one and spelled its destinations without the slash. ```bash node map.mjs --check drafts/first-draft_redirects ``` ```text output static rule for /docs/1.4/meters/list-meters is shadowed by /docs/* above it /docs/intro -> /guides/introduction chains: the page is at /guides/introduction/ /docs/quickstart -> /guides/quickstart chains: the page is at /guides/quickstart/ ``` Cloudflare applies the top-most matching rule, so the splat answered every request and sent `/docs/auth/api-keys` to `/guides/auth/api-keys`, which is not a page. Put static rules first, or leave the catch-all out. The chains are the slash: the host sends `/guides/quickstart` on to `/guides/quickstart/` with a redirect of its own, so the reader makes two round trips. A reader arrives at the top of the right page instead of the section they were sent to. The fragment went with them, and the heading id changed. ```bash node pitfall.mjs ``` ```text output link followed: /docs/auth/api-keys#rotate-a-key server received: /docs/auth/api-keys server answered: 301 /guides/authentication/api-keys/ browser lands on: /guides/authentication/api-keys/#rotate-a-key ids on that page: create-a-key scopes--permissions rotating-a-key revoke-a-key #rotate-a-key: not on the page, so the reader lands at the top ``` The server saw a path with no fragment, so no rule could have mapped `rotate-a-key` to `rotating-a-key`. Only the audit sees the loss, and only an explicit id or the old heading text prevents it. The crawl passes against the stand-in but fails against the deployment. Each static host treats `/x`, `/x/`, and `/x.html` differently, and the [trailing slash guide](https://github.com/slorber/trailing-slash-guide) tabulates them, and GitHub Pages answers 404 where Cloudflare Pages redirects. Crawl a preview deployment with `--base` before you trust the stand-in. ## When not to do this Do not restructure in the same release as the toolchain move unless you will own the longer table. The mirror map here is four rules, and the host does the rest. The restructure is fifteen rules that stay in the file for as long as anyone links to the old paths, and every future rename lengthens it. Do not add a catch-all splat to the guides index to bring the 404 count to zero. A reader who followed a link to a specific page lands on an index with no idea why. You also lose the 404 that would have told you a path still needs a rule. Canonical tags are not a migration. They tell search which of two live URLs to prefer, and Google calls the tag a strong signal rather than an instruction. Once the old site stops serving, the tag goes with it. Do not trust the sitemap alone, or the log alone. The sitemap misses the paths nobody meant to publish. The log misses the pages nobody visited this quarter, whose links sit in a book or a slide deck you will never see. ## Related how-tos - [Make a Docusaurus site agent-ready](/howto/make-a-docusaurus-site-agent-ready) - [Publish an OpenAPI document at a stable URL for agents](/howto/publish-openapi-at-a-stable-url) - [Re-index docs on release without serving stale answers](/howto/reindex-docs-on-every-release) - [Turn a breaking release changelog into a migration guide](/howto/changelog-to-a-migration-guide) ## Last verified Verified 2026-10-21 against Node 22.22.0 and github-slugger 2.0.0. Every output block is what the command preceding it printed, run in the page's code directory against an in-process stand-in server. The host's handling of a trailing slash and an `.html` suffix is modeled on Cloudflare's documentation and the source of its Pages asset server, not measured against a deployment. The MkDocs ids come from a port of Python-Markdown's slug function, checked against a run of Python-Markdown 3.11.1 rather than against an MkDocs build. No Docusaurus, Starlight, or MkDocs site was built. [^1]: The [protocol](https://www.sitemaps.org/protocol.html) allows a file 50,000 URLs and 50 MB uncompressed, and its one date field, `lastmod`, records when a page last changed. No field records that a URL has stopped answering, so a sitemap can be valid in every field while naming pages that have been gone for a year. [^2]: The browser's habit of carrying the fragment across a redirect entered HTTP in 2014, in [RFC 7231](https://www.rfc-editor.org/rfc/rfc7231.html#section-7.1.2). Fifteen years earlier, [RFC 2616](https://www.rfc-editor.org/rfc/rfc2616.html#section-14.30) had described `Location` as a single absolute URI and left fragments unmentioned. RFC 9110 kept the rule in 2022 and changed one term in it: the request target became the target URI. [^3]: GitHub [documents](https://docs.github.com/en/get-started/writing-on-github/getting-started-with-writing-and-formatting-on-github/basic-writing-and-formatting-syntax#section-links) its anchor rules: letters made lowercase, spaces replaced by hyphens, other punctuation removed, and a repeated heading given a hyphen and a counter. Removing the ampersand from Scopes & permissions leaves two spaces, so the id is `scopes--permissions`, and github-slugger reproduces the double hyphen on purpose. Python-Markdown collapses the run to one hyphen and counts repeats with an underscore, which is the whole of the disagreement.