🕷️ Fix OTTO Crawler Failures on Homepage Redirects

Camilo Aponte

Camilo Aponte

Last updated on Sep 30, 2026

🔍 Overview

When you create an OTTO SEO project, OTTO crawls your homepage to verify the pixel installation and index your site. If your domain automatically redirects to a subdirectory — for example, example.com redirecting to example.com/news — OTTO may follow that redirect and land on a page that breaks the crawl. This can cause project creation to stall, return zero pages crawled, or prevent pixel detection from completing successfully.

⚠️ Why This Happens

OTTO's crawler starts at your root domain and follows any redirects it encounters. Redirects to subdirectories can cause two specific problems:

  • Pixel not detected: If the subdirectory page does not carry the OTTO pixel in its HTML, the crawler cannot confirm a successful installation, and project creation fails.
  • Crawl returns zero pages: In some configurations, the redirect strips internal parameters OTTO needs to complete the crawl, resulting in no pages being indexed even though the site is reachable.

✅ Step-by-Step: Resolve the Redirect Issue

  1. Confirm where your domain redirects. Open a browser and type your bare domain (e.g., example.com). Note the final URL in the address bar. If it ends in a subdirectory such as /news or /en, a redirect is in place.
  2. Place the OTTO pixel on the redirect destination. Make sure the OTTO tracking pixel is installed on the page your domain actually lands on after the redirect — not just on the root-level template. If you are using a CMS, add the pixel to the header template of that specific section.
  3. Use the exact landing URL when creating your project. In Search Atlas, go to Left sidebar → OTTO SEO → All Sites (SEO Automation) (URL: /seo-automation-v3). When prompted to enter your domain during project creation, enter the full subdirectory URL that is the true homepage — for example, example.com/news — rather than the bare domain. This gives OTTO a stable starting point that does not redirect.
  4. Verify the pixel using your browser. Before retrying project creation, open the landing page, right-click, and select View Page Source. Search for your OTTO pixel code to confirm it is present. If it is missing, reinstall it and save your changes before proceeding.
  5. Create or re-create the OTTO project. Return to Left sidebar → OTTO SEO → All Sites (SEO Automation) and select Create. Enter the subdirectory URL from step 3 and complete the setup wizard. OTTO will now crawl from that stable endpoint instead of chasing a redirect.
  6. Monitor the crawl. After creation, check that the project status moves past the initial setup phase and that pages are being indexed. If the project shows 0 pages crawled after several minutes, proceed to the troubleshooting section below.

🛠️ Additional Troubleshooting

If the steps above do not resolve the issue, consider the following:

  • Check for redirect chains. A single domain can have multiple hops — for example, example.com → www.example.com → example.com/news. Each hop increases the chance of parameter loss. Use a redirect-checker tool to map every hop and ensure the final destination carries the pixel.
  • Confirm there is no redirect loop. Misconfigured redirects can cause a page to redirect back to itself. If your site went offline or slowed down after adding OTTO, this may be the cause. Contact your hosting provider or developer to audit your redirect rules.
  • Check that the subdirectory is publicly reachable. Some subdirectories are gated behind a login or return a non-200 HTTP status for bots. OTTO requires a publicly accessible 200 OK response to crawl successfully.
  • Allow up to 15 minutes after pixel installation before triggering project creation. Caching layers on your server or CDN may serve an older version of the page without the pixel for a short period after you save changes.

❓ Frequently Asked Questions

Can OTTO crawl a subdirectory like /news instead of the root domain?
Yes. When creating your OTTO project, enter the subdirectory URL as your project domain. OTTO will treat that URL as the starting point for the crawl and will not require the bare root domain.

Why did my project domain change automatically after creation?
If you entered a root domain that immediately redirects, OTTO may have auto-resolved it to the redirect destination. This is expected behavior. Verify the project now points to the correct subdirectory and that the pixel is installed there.

Will using a subdirectory as my project URL affect SEO recommendations?
OTTO will scope its recommendations to the pages it can reach from your starting URL. If your entire site lives under /news, entering that subdirectory will give OTTO full access. If your site spans multiple subdirectories, contact support to discuss the best configuration for your setup.

💬 Need More Help?

If you need further assistance, open the chat widget in the bottom-right corner of the platform and type human teammate to be connected with a member of our team.