# ═══════════════════════════════════════════════ # Baznek — robots.txt # Domain: baznek.com # ═══════════════════════════════════════════════ # ── Google AdSense & Search Crawlers ── User-agent: Mediapartners-Google Allow: / User-agent: Googlebot Allow: / Allow: /blog Allow: /blog/ Allow: /compare/ Allow: /privacy Allow: /terms Allow: /changelog Allow: /about Allow: /contact Allow: /developer-api.html Allow: /llms.txt Allow: /api/v1/openapi.json # ── General Web Crawlers ── User-agent: * Allow: / Allow: /blog Allow: /blog/ Allow: /compare/ Allow: /privacy Allow: /terms Allow: /changelog Allow: /about Allow: /contact Allow: /developer-api.html Allow: /llms.txt Allow: /api/v1/openapi.json # ── Disallow Private & Internal Endpoints ── Disallow: /admin Disallow: /metrics Disallow: /api/auth Disallow: /api/journal Disallow: /api/ab-test Disallow: /api/waitlist Disallow: /api/v1/calculate Disallow: /api/v1/rates Disallow: /api/v1/tax-estimate Disallow: /api/v1/limits Disallow: /api/v1/keys # ── Block Dev / Internal Files ── Disallow: /service-worker.js Disallow: /manifest.json # ── Explicit AI & LLM Search Crawler Permissions (GEO / LLMO) ── User-agent: GPTBot Allow: / Allow: /blog Allow: /compare/ Allow: /developer-api.html Allow: /llms.txt Allow: /api/v1/openapi.json Disallow: /admin Disallow: /metrics Disallow: /api/auth User-agent: OAI-SearchBot Allow: / Allow: /blog Allow: /compare/ Allow: /developer-api.html Allow: /llms.txt Allow: /api/v1/openapi.json Disallow: /admin Disallow: /metrics Disallow: /api/auth User-agent: PerplexityBot Allow: / Allow: /blog Allow: /compare/ Allow: /developer-api.html Allow: /llms.txt Allow: /api/v1/openapi.json Disallow: /admin Disallow: /metrics Disallow: /api/auth User-agent: Perplexity-User Allow: / Allow: /blog Allow: /compare/ Allow: /developer-api.html Allow: /llms.txt Allow: /api/v1/openapi.json Disallow: /admin Disallow: /metrics Disallow: /api/auth User-agent: ClaudeBot Allow: / Allow: /blog Allow: /compare/ Allow: /developer-api.html Allow: /llms.txt Allow: /api/v1/openapi.json Disallow: /admin Disallow: /metrics Disallow: /api/auth User-agent: anthropic-ai Allow: / Allow: /blog Allow: /compare/ Allow: /developer-api.html Allow: /llms.txt Allow: /api/v1/openapi.json Disallow: /admin Disallow: /metrics Disallow: /api/auth User-agent: Google-Extended Allow: / Allow: /blog Allow: /compare/ Allow: /developer-api.html Allow: /llms.txt Allow: /api/v1/openapi.json Disallow: /admin Disallow: /metrics Disallow: /api/auth User-agent: Bingbot Allow: / Allow: /blog Allow: /compare/ Allow: /developer-api.html Allow: /llms.txt Allow: /api/v1/openapi.json Disallow: /admin Disallow: /metrics Disallow: /api/auth # ── Sitemap Location ── Sitemap: https://www.baznek.com/sitemap.xml