How to Fix Non-200 URLs in Your XML Sitemap

Your XML sitemap lists URLs that do not return a clean 200 OK, such as redirects (301/302), missing pages (404/410), server errors (5xx) or pages blocked by a firewall (403). This guide shows you how to find each bad URL, fix or remove it at the source, and confirm that every sampled sitemap URL returns 200.

By Updated ⏱ 20-45 minutes🎯 Beginner● Medium impactTechnical
Illustration: Non-200 URLs in sitemap fix guide
TECHNICAL · FIX GUIDE Non-200 URLs in sitemap IMPACTEFFORT TIME20-45 minutes LEVELBeginner

Why it matters

A sitemap tells Google and Bing which URLs are canonical and worth crawling. When a meaningful share of them redirect or error, crawlers spend budget on dead ends, discovery of real pages slows, and Google's Search Console reports start flagging the sitemap as unreliable. AI search crawlers that use sitemaps for discovery hit the same broken or redirected URLs, so your current content is picked up later or not at all.

Our audit samples URLs from your sitemap and fails the check when 10% or more of them return anything other than 200.

✕Beforehttps://example.com/sitemap.xml listshttp://example.com/blog/old-post (301→ https://example.com/blog/new-post/)✓Afterhttps://example.com/sitemap.xml listsonly final, indexable URLs likehttps://example.com/blog/new-post/,

What you need

Your sitemap URL (usually https://example.com/sitemap.xml or listed in /robots.txt)
A terminal with curl, or a crawler such as Screaming Frog SEO Spider in List mode
Admin access to your CMS or sitemap generator
Access to Google Search Console for resubmission

Step-by-step fix

1

Locate every sitemap file

Open https://example.com/robots.txt and note each Sitemap: line. If the file is a sitemap index (it contains <sitemapindex> and <sitemap> entries), open each child sitemap, because the URLs you need to check live there. Common locations are /sitemap.xml, /sitemap_index.xml (Yoast, Rank Math) and /wp-sitemap.xml (WordPress core).

text
curl -s https://example.com/robots.txt | grep -i '^sitemap:'
Pro tip: If robots.txt has no Sitemap line, add one. It costs nothing and helps every crawler find the file.
2

Check the HTTP status of every sitemap URL

Request each <loc> URL without following redirects, so a 301 is reported as 301 rather than the 200 of its destination. The command below prints only URLs that are not 200. For large sitemaps, use Screaming Frog: Mode > List, then Upload > Download XML Sitemap, and set Configuration > Spider > Advanced so redirects are not followed, then filter the Response Codes tab by non-200.

text
curl -s https://example.com/sitemap.xml | grep -oE '<loc>[^<]+' | sed 's/<loc>//' | while read u; do code=$(curl -s -o /dev/null -w '%{http_code}' -A 'Mozilla/5.0 (compatible; Googlebot/2.1)' "$u"); [ "$code" != "200" ] && echo "$code $u"; done
Pro tip: Group the results by status code. Redirects, 404s and 403s each have a different fix, and they usually come from one or two root causes.
3

Replace redirected URLs with their final destination

A 301/302/307/308 in a sitemap almost always means the sitemap lists a non-canonical version. The usual culprits are:

  • http:// instead of https://
  • www vs non-www mismatch
  • a missing or extra trailing slash
  • old slugs that were renamed

Set your CMS Site Address / base URL to the exact canonical form so the generator outputs final URLs. For renamed pages, make sure the sitemap lists the new slug, not the old one.

html
<!-- Wrong: redirects -->
<url><loc>http://example.com/pricing</loc></url>
<!-- Right: final 200 URL -->
<url><loc>https://example.com/pricing/</loc></url>
Pro tip: Run `curl -sIL https://example.com/pricing` to see the full redirect chain. The last Location header is the URL that belongs in the sitemap.
4

Remove or restore 404 and 410 URLs

For each 404 or 410, decide whether the page should exist. If it was deleted on purpose, unpublish it fully (including from trash, drafts or hand-edited sitemap files) so the generator drops it. If it was deleted by mistake, restore it.

If the content moved, add a 301 redirect for visitors and list only the new URL in the sitemap.

Pro tip: Static or hand-maintained sitemaps are the most common source of 404s. Switch to a generated sitemap whenever possible.
5

Fix 403, 429 and 5xx responses

A 403 or 429 usually means a WAF, bot-protection or rate-limit rule is challenging crawlers. A 5xx points to a server or plugin error on that template. Check your firewall logs for blocked requests to those paths and allowlist verified search engine bots.

Then load the failing URLs yourself to reproduce any server error.

text
curl -s -o /dev/null -w '%{http_code}\n' -A 'Mozilla/5.0 (compatible; bingbot/2.0)' https://example.com/products/blue-widget/
Pro tip: Compare the status code with a browser user agent and with a bot user agent. If they differ, the problem is your bot protection, not the page.
6

Exclude noindex and non-canonical pages from the generator

Configure the sitemap generator so it only outputs indexable, self-canonical URLs. Exclude:

  • tag and author archives you set to noindex
  • thank-you and cart pages
  • paginated or parameter URLs
  • any page whose canonical points elsewhere

This stops new bad URLs from entering the sitemap the next time content changes.

Pro tip: Rule of thumb: if a URL would not be the one you want shown in Google, it does not belong in the sitemap.
7

Regenerate, resubmit and re-test

Clear any page or CDN cache for the sitemap files so the new version is served. Then re-run the status check from step 2 and confirm it prints nothing. In Google Search Console, go to Indexing > Sitemaps, enter the sitemap URL and click Submit.

In Bing Webmaster Tools, go to Sitemaps > Submit sitemap.

text
curl -sI https://example.com/sitemap.xml | grep -iE 'HTTP/|cache|age'
Pro tip: Re-run our audit after the cache clears. It samples URLs, so a clean full crawl of the sitemap is the best guarantee of a pass.

Platform-specific notes

WordPress

First check **Settings > General**: WordPress Address and Site Address must use the exact canonical scheme and host (e.g. https://example.com). In Yoast, use **Yoast SEO > Settings > Content types / Taxonomies** and turn off "Show in search results" for types that should not be in the sitemap. In Rank Math, use **Rank Math > Sitemap Settings**. Noindexed posts are excluded automatically. If both a plugin and WordPress core (/wp-sitemap.xml) produce sitemaps, keep only one.

Shopify

Shopify generates /sitemap.xml automatically, and you cannot edit it directly. Non-200 entries usually come from products set to unavailable on the Online Store sales channel, or from a primary domain mismatch. Set the primary domain under **Settings > Domains**. Manage redirects by searching "URL redirects" in the admin. To remove a product or page, unpublish it from the Online Store channel. Apps that add `seo.hidden` metafields also remove items from the sitemap.

Next.js

With the App Router, the sitemap is built in `app/sitemap.ts`. Build each URL from one canonical base constant, and match the `trailingSlash` setting in next.config.js so entries don't 308-redirect. Filter out unpublished or deleted CMS records before returning the array. Libraries like next-sitemap need `siteUrl` set to the https canonical host.

Cloudflare

Bot Fight Mode, Super Bot Fight Mode or custom WAF rules can return 403 or challenge pages to crawlers even though the URLs work in a browser. Under **Security > Events**, filter by the affected paths. Make sure verified bots are allowed, for example with a skip rule using `cf.client.bot`. After regenerating, purge the cached sitemap under **Caching > Configuration > Purge Cache**.

Wix

Wix builds the sitemap automatically. To remove a page, open **Pages & Menu**, click the page's **...** menu, choose **SEO basics**, and turn off "Let search engines index this page"; noindexed pages are excluded. Set up 301s for renamed pages in the **URL Redirect Manager**, found under **Marketing & SEO > SEO** (the exact location varies by dashboard version).

Check that it worked

✅ How to verify

  • Re-run the curl loop from step 2 against every child sitemap and confirm it outputs no lines.
  • In Google Search Console, under **Indexing > Sitemaps**, the sitemap shows status "Success". Over the following weeks, Page indexing should show fewer "Page with redirect" and "Not found (404)" entries attributed to the sitemap.
  • Spot-check five random sitemap URLs with `curl -sI URL` and confirm the first line is `HTTP/2 200` with no Location header.
  • Re-run our audit and confirm that fewer than 10% of sampled sitemap URLs are flagged, ideally 0.

⚠️ Common mistakes

  • Testing with redirects followed (`curl -L` or a browser), which hides 301s because the final page returns 200.
  • Fixing the redirect rule but leaving the old URL in a static or cached sitemap file.
  • Running two sitemap generators (e.g. WordPress core plus a plugin) where only one is maintained, so the stale one keeps listing old URLs.
  • Allowing bot protection to block search engine crawlers, so URLs return 403 to bots while looking fine to you.

Is this still an issue on your site? Run a free audit — ZORO checks this and 294 other SEO, AI-visibility, accessibility and security signals, with fixes for your platform.

Check my site free

FAQ

Should redirected URLs be in my sitemap?

No. A sitemap should list only final, canonical URLs that return 200. If a URL redirects, list its destination instead. Keep the redirect in place for visitors and old backlinks.

Do 404s in a sitemap hurt my rankings?

They don't trigger a penalty, but they waste crawl budget and make Google trust your sitemap less as a signal. On large sites this slows down how quickly new and updated pages are discovered.

Should noindex pages be in the XML sitemap?

No. Listing a noindex page sends conflicting signals: the sitemap says "index this" while the page says "don't". Configure your generator to exclude noindex pages and pages whose canonical points to a different URL.

How often should I check my sitemap for broken URLs?

Check after any migration, domain or HTTPS change, URL structure change, or bulk content deletion. Otherwise, a monthly crawl of the sitemap is enough for most sites.