🔍 What Is the Nginx 444 Error?

When you try to add a site to an OTTO project and see a "No sites found" error, the root cause may be your web server silently dropping the connection before Search Atlas can crawl the site. A 444 status code is specific to Nginx — it instructs the server to close the connection immediately without sending any response. Unlike a standard 403 or 429 error, a 444 response produces no error message, making it harder to diagnose.

This is not a problem with your OTTO configuration. It means the Nginx server hosting your site is actively blocking the Search Atlas crawler before the crawl can begin.

⚙️ Why Nginx Returns a 444 to Search Atlas

Nginx servers can be configured to return a 444 for requests that match certain conditions, including:

• Requests from unrecognised or unlisted user-agent strings
• Requests from IP address ranges not explicitly permitted in the server configuration
• Security rules that treat automated crawlers as malicious traffic by default
• Rate-limiting or bot-mitigation rules that trigger on crawl patterns

Because the connection is dropped silently, the Search Atlas platform receives no usable response and cannot confirm the site exists — which is why OTTO reports "No sites found" instead of a clear access-denied message.

✅ How to Whitelist the Search Atlas Crawler

To allow the Search Atlas crawler through your Nginx server, you need to permit its user-agent and/or IP ranges in your Nginx configuration. Follow the steps below or share them with your hosting provider or developer.

1. Identify your Nginx configuration file. This is typically located at /etc/nginx/nginx.conf or inside /etc/nginx/sites-available/ for your specific domain.
2. Whitelist the Search Atlas user-agent. Add a rule that allows requests from the Search Atlas crawler user-agent. Your developer should add a condition that checks for the Search Atlas bot user-agent string and ensures it is not matched by any return 444 or deny directive. Contact support via the chat widget below to obtain the exact current user-agent string for the Search Atlas crawler.
3. Whitelist Search Atlas IP ranges. Add the Search Atlas crawler IP addresses to your allowlist before any deny-all or connection-drop rules. Contact support via the chat widget below to obtain the current list of Search Atlas crawler IP addresses.
4. Check the order of your Nginx rules. Nginx processes rules sequentially. Ensure your whitelist rules appear before any blanket bot-blocking or rate-limiting rules, otherwise the allow rule will never be reached.
5. Reload Nginx to apply changes. Run sudo nginx -t to validate the configuration syntax, then run sudo systemctl reload nginx to apply the changes without downtime.
6. Re-add your site in OTTO. Navigate to OTTO SEO in the left sidebar and open All Sites (SEO Automation), open your OTTO project, and attempt to add the site again. The crawler should now be able to reach the site and the "No sites found" error should be resolved.

🛡️ If You Use a Third-Party Firewall or CDN Alongside Nginx

If your Nginx server sits behind an additional layer such as a CDN or a web application firewall, the 444 block may be applied at the Nginx level even if your CDN settings appear to allow crawlers. Check both layers:

• Nginx configuration — review all return 444 and deny directives, particularly inside if blocks that evaluate the user-agent or remote address.
• WAF or CDN rules — confirm that bot-fight or challenge modes are not re-blocking the crawler after Nginx has been updated. Refer to your CDN provider's documentation for steps to whitelist specific user-agents or IP ranges at the CDN level.

Both layers must permit the Search Atlas crawler for the connection to succeed.

🔎 How to Confirm the Block Is Resolved

After updating your configuration, you can verify the fix before re-adding the site in OTTO:

• Use a tool such as curl -A "[Search Atlas user-agent]" https://yourdomain.com from a terminal to simulate a crawler request and confirm the server returns a 200 response instead of dropping the connection.
• Check your Nginx access logs (/var/log/nginx/access.log) to confirm requests from the Search Atlas user-agent are no longer returning 444.
• Once confirmed, navigate to OTTO SEO in the left sidebar and open All Sites (SEO Automation), then retry adding the site to your OTTO project.

💬 Need Help?

If you need further assistance, open the chat widget in the bottom-right corner of the platform and type human teammate to be connected with a member of our team.