🔍 What You May Be Seeing
You ran a recrawl on your site and the new pages now appear in Page Explorer. However, when you open OTTO Automation, those same pages are not listed. This is a common question, and it does not mean the crawl failed.
🧠 Why This Happens
Crawling and OTTO processing are two separate steps:
- Crawling discovers the pages on your site and adds them to Page Explorer.
- OTTO processing analyzes those pages so OTTO can generate and manage optimizations for them in Automation.
A recrawl updates your page list, but OTTO may not automatically pick up the new pages. In that case, the project needs to be reprocessed so OTTO can include them.
✅ Quick Checks Before Requesting a Reprocess
Run through these checks first. They resolve many cases on their own.
- Confirm the crawl finished. Make sure the recrawl shows as complete and is not still running.
- Verify the pages in Page Explorer. Search for the exact page URLs to confirm they were captured.
- Check the correct site. If you manage several sites, confirm you are viewing the same project in both Page Explorer and OTTO Automation.
- Clear filters. Remove any active filters or search terms in OTTO Automation that could hide the new pages.
- Refresh the page. Reload your browser or log out and back in to make sure you are seeing the latest data.
- Allow some time. After a crawl, processing can take a while on larger sites. Check again later before taking further action.
🔁 How to Get the New Pages Into OTTO
If the pages are in Page Explorer but still missing from OTTO Automation after the checks above, the site most likely needs to be reprocessed.
- Open the project for the affected site.
- Go to OTTO and check whether a reprocess or refresh option is available for the site.
- If it is available, start it and allow time for processing to complete.
- Return to OTTO Automation and check that the new pages are listed.
If you do not see a reprocess option, our team can trigger it for you. Share the site name and a few example page URLs so we can check them quickly.
💡 Tips to Avoid This in the Future
- After every recrawl, confirm new pages appear in both Page Explorer and OTTO Automation before making further changes.
- When you add many pages at once, such as a new section or product category, expect that a reprocess may be needed.
- Keep a list of newly published URLs so you can verify them easily.
- Make sure new pages are accessible to crawlers and not blocked by settings such as noindex tags or robots rules.
🛠️ Still Not Showing?
If the pages are still missing after a reprocess, include the following details when you contact us:
- The site or project name
- Two or three example URLs that are missing from OTTO Automation
- The approximate date and time of your last recrawl
- Whether the pages appear in Page Explorer
💬 Need Help?
If you need further assistance, open the chat widget in the bottom-right corner of the platform and type human teammate to be instantly connected with a member of our team.