🔍 What Does 'Not Globally Crawlable' Mean?
When Google flags your site as not globally crawlable, it means Googlebot is being blocked from accessing your pages from one or more geographic regions. This flag appears in Google Search Console and signals that your site may not be indexed consistently worldwide.
This flag has two possible root causes that require very different solutions. Understanding which applies to your situation will save you significant troubleshooting time.
⚠️ Two Root Causes — One Flag
Before taking any action, it is important to distinguish between the two categories below:
- Category 1 — Search Atlas / OTTO actionable: Issues related to how your site is configured within the Search Atlas ecosystem, such as incorrect crawl settings or OTTO directives that unintentionally restrict access.
- Category 2 — External configuration (requires your developer or hosting provider): Issues that exist at the server, CDN, firewall, or hosting level and are entirely outside the scope of Search Atlas or OTTO. These require intervention from your technical team or hosting provider.
The vast majority of not globally crawlable flags reported through Google Search Console fall into Category 2. Search Atlas does not control your server infrastructure, firewall rules, or CDN access policies.
✅ What Search Atlas Can Help You Check
If you believe the issue may be related to your Search Atlas or OTTO configuration, verify the following before escalating externally:
- Confirm that OTTO has not applied any unintended robots.txt directives that block Googlebot. Review your active OTTO tasks and look for any robots.txt changes that may have been deployed.
- Check that no noindex or nofollow tags have been pushed site-wide by mistake through OTTO optimisations.
- Ensure your sitemap submitted in Google Search Console is current and reflects the correct URLs for your site.
If none of the above apply, or if OTTO has not made changes to your robots.txt or meta directives, the root cause is almost certainly external to Search Atlas.
🛠️ External Causes That Require Developer or Hosting Intervention
The following issues are outside the scope of Search Atlas and OTTO. Your developer or hosting provider must resolve them:
- Geo-blocking or IP filtering: Your server, firewall, or security software is blocking requests from certain countries or IP ranges, including Google's crawl infrastructure.
- CDN misconfiguration: Your content delivery network (for example, Cloudflare, Fastly, or AWS CloudFront) may be restricting access based on region or flagging Googlebot as suspicious traffic.
- Hosting provider restrictions: Some hosting plans restrict traffic from outside specific regions by default. Your hosting provider's control panel or support team can confirm and adjust this.
- DDoS protection or bot mitigation tools: Security services that block automated bots may inadvertently block Googlebot. Your developer needs to whitelist Google's verified crawlers in your security settings.
- Server-side redirects or authentication walls: If your server returns different responses based on the visitor's geographic location, Google may be served a blocked or redirect response in certain regions.
📋 How to Diagnose the Issue Quickly
Follow these steps to identify whether the problem is internal or external:
- Open Google Search Console and navigate to Settings → Crawl Stats. Look for errors tied to specific response codes (403, 429, or connection timeouts) that indicate server-level blocking.
- Use Google's URL Inspection Tool inside Search Console to test a sample URL. If Google reports it cannot reach the page, the block is at the server or network level.
- Use a third-party tool such as GeoPerf or Uptrends to test your site's availability from multiple global locations. If the site fails to load from certain regions, confirm a server or CDN-level block is in place.
- Ask your hosting provider or developer to review your firewall logs for blocked requests from Googlebot IP ranges, which are published in Google's official documentation.
🚀 Recommended Next Steps Based on Your Situation
- If OTTO or Search Atlas changes are the suspected cause: Review your OTTO task history and revert any robots.txt or meta tag changes. Contact our support team via the chat widget for guided assistance.
- If the issue is external (most common scenario): Share the diagnostic information from Google Search Console and your geo-availability test results with your developer or hosting provider. Ask them specifically to review firewall rules, CDN geo-restrictions, and bot mitigation settings.
- If you are unsure which category applies: Start with the OTTO review steps above, then run the URL Inspection Tool in Google Search Console. If Google still cannot reach the page and OTTO has made no relevant changes, escalate externally.
If you need further assistance, open the chat widget in the bottom-right corner of the platform and type human teammate to be connected with a member of our team.