Overview
When you add a large number of pages to OTTO at once, you may notice that only a portion of them appear in your crawl results immediately. This is expected behavior, not a bug. OTTO's crawl function has built-in processing limits for bulk additions, and full visibility across all newly added pages may not happen instantly.
How OTTO Processes Newly Added Pages
OTTO separates two distinct processes that customers often expect to happen simultaneously:
- Page addition: The act of submitting new URLs to OTTO for tracking and optimization.
- Crawl visibility: The appearance of those pages within OTTO's crawl results.
Adding pages successfully does not mean they will all be visible in the crawl results right away. OTTO queues pages for analysis and processes them over time. When a large number of pages are submitted at once, only a portion may appear immediately — the rest are queued and will surface as OTTO continues to process them.
Why Only Some Pages Are Appearing
If you added a large batch of pages and only a fraction are showing in your crawl results, this reflects OTTO's processing behavior for bulk submissions. The pages that do not appear immediately are still queued and tracked — none are lost. They will become visible as OTTO works through the queue. This behavior is by design to maintain platform performance across all users.
What You Can Do
If you are waiting for pages to appear in OTTO after a bulk addition, the following steps can help manage your workflow:
- Wait for processing to complete: Pages submitted in bulk will appear progressively as OTTO processes the queue. Check back periodically to see newly visible pages.
- Submit smaller batches when possible: If you need certain pages to be processed sooner, consider submitting your highest-priority pages in a smaller, separate batch before submitting the rest. Smaller submissions are generally processed more quickly.
- Confirm pages are queued: Verify within OTTO that your submitted URLs are listed and tracked, confirming they have been accepted into the system even if not yet fully visible in crawl results.
- Contact support if pages remain missing: If pages have not appeared after an extended period and you believe there may be an issue, reach out to the support team with the specific URLs and the date you submitted them so the team can investigate.
If you need further assistance, open the chat widget in the bottom-right corner of the platform and type human teammate to be connected with a member of our team.