🔎 Overview
After triggering a recrawl on a site, you may notice it does not immediately appear — or reappear — under Active Projects in the Agent Hub. This is a known delay that occurs while Search Atlas processes the new crawl data and syncs it across the platform. This article explains why it happens and what you can do to resolve it.
⚙️ Why the Delay Happens
When a recrawl is initiated, Search Atlas goes through several background steps before the site is reflected in the Agent Hub:
- Crawl queuing: Depending on site size and platform load, your recrawl may sit in a processing queue before it begins.
- Data indexing: Once crawling is complete, the system needs time to index the updated site data before it can be surfaced in Active Projects.
- Sync lag: There is a short synchronisation window between when crawl data is processed and when the Agent Hub refreshes its project list.
In most cases, the site will appear in Active Projects within a few minutes to a few hours depending on the size of the site and current system load.
🛠️ Steps to Resolve the Issue
- Wait and refresh: After a recrawl, wait at least 10–15 minutes, then hard-refresh your browser (Ctrl + Shift + R on Windows, Cmd + Shift + R on Mac) and navigate back to the Agent Hub at Left sidebar → Home → Agent Hub (URL: /home).
- Confirm the recrawl completed: Go to Left sidebar → Site Metrics (Site Explorer) (URL: /site-explorer/list) and locate your site. Check whether the crawl status shows as completed. If it still shows as in progress or queued, the Agent Hub will not update until crawling finishes.
- Check that the project is active: Inside the Agent Hub, confirm the project has not been paused or deactivated. A project in a paused state will not surface in the Active Projects list even after a successful recrawl.
- Re-enter the Agent Hub: Navigate away from the Agent Hub page and return to it via Left sidebar → Home → Agent Hub. This forces the page to re-fetch the latest project data.
- Clear browser cache: Cached data can sometimes prevent updated project status from loading. Clear your browser cache, then log back in and check Active Projects again.
📊 What to Expect After a Successful Recrawl
Once the recrawl has fully completed and data has been indexed, your site will appear in Active Projects in the Agent Hub with refreshed metrics. Any previously flagged issues will be updated to reflect the current state of the site. If the project was visible before the recrawl, it should return to the same position in your Active Projects list.
💡 Tips to Avoid Future Delays
- Avoid triggering multiple recrawls in quick succession on the same site — this can push your crawl further back in the queue.
- Schedule recrawls during off-peak hours if your site is large, as processing times can vary based on platform load.
- After initiating a recrawl, monitor the crawl status in Site Explorer before checking the Agent Hub, so you know when to expect the update.
🆘 Still Not Showing Up?
If your site has not appeared in Active Projects after several hours and the recrawl shows as completed in Site Explorer, there may be an underlying issue that requires investigation. If you need further assistance, open the chat widget in the bottom-right corner of the platform and type human teammate to be connected with a member of our team.