Crawl and analyze XML sitemaps with regex filtering via sitemap-mcp-server (npx)

Recipe: Crawl and analyze XML sitemaps via sitemap-mcp-server (npx)

Package: sitemap-mcp-server v1.0.1 (npm) Transport: stdio (NDJSON framing) Launch: npx -y sitemap-mcp-server Auth: none (LLM filtering mode optional, requires PROVIDER/API_KEY/MODEL env vars) Cold-start: ~2s (npx install)

Server identity

serverInfo: { name: "sitemap-mcp-server", version: "1.0.0" }
capabilities: { tools: {} }

Tool inventory (3 tools)

Verified trace (3 tool calls across 2 sitemaps)

1. analyze_sitemap — structure analysis on docs.anthropic.com (3,123 URLs)

→ {"jsonrpc":"2.0","id":10,"method":"tools/call","params":{"name":"analyze_sitemap","arguments":{"url":"https://docs.anthropic.com/sitemap.xml"}}}
← {
  "total_urls": 3123,
  "by_degree_level": { "postgraduate": 696, "undergraduate": 188, "unknown": 2239 },
  "by_pattern": { "/docs/": 3122 },
  "priority_stats": {
    "avg": "0.81",
    "distribution": { "0.5": 1, "0.7": 1331, "0.8": 122, "0.9": 1637, "1.0": 32 }
  }
}

2. fetch_sitemap — regex-filtered URL extraction

→ {"jsonrpc":"2.0","id":11,"method":"tools/call","params":{"name":"fetch_sitemap","arguments":{"url":"https://docs.anthropic.com/sitemap.xml","include_patterns":[".*tool-use.*"]}}}
← {
  "total": 300,
  "pg_count": 24, "ug_count": 12, "unknown_count": 264,
  "urls": [
    { "loc": "https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview", "priority": 1 },
    { "loc": "https://platform.claude.com/docs/en/agents-and-tools/tool-use/how-tool-use-works", "priority": 0.8 },
    { "loc": "https://platform.claude.com/docs/en/agents-and-tools/tool-use/define-tools", "priority": 0.8 },
    // ... 300 URLs across en/ja/ko locales
  ]
}

Notes

• NDJSON framing (newline-delimited)
• The degree_level field and by_degree_level breakdown appear university-oriented (pg/ug categorization heuristic built into the server) — these fields are present but not meaningful for non-university sitemaps
• fetch_sitemap supports both include_patterns and exclude_patterns as arrays of regex strings
• detect_changes requires a university_key for state persistence between runs — useful for monitoring crawls
• Works on any public sitemap.xml including sitemap index files
• The LLM filtering mode (use_llm: true + user_intent) is powerful for semantic filtering but requires API credentials in environment variables