User-agent: * Disallow: Sitemap: https://surfmaroc.com/sitemap.xml # Explicitly Allow Known AI Crawlers User-agent: GPTBot Allow: / User-agent: ClaudeBot Allow: / User-agent: CCBot Allow: / User-agent: Google-Extended Allow: / # Allow Search Engine Crawlers User-agent: Googlebot Allow: / User-agent: Bingbot Allow: / User-agent: Yandex Allow: / User-agent: Baiduspider Allow: / User-agent: DuckDuckBot Allow: / User-agent: Applebot Allow: /