🗺️ Overview
When running a Site Audit in Search Atlas, two settings have a direct impact on the quality of your results and how long the audit takes: user agent and crawl speed. Choosing the wrong user agent can reduce page capture accuracy, while an uncalibrated crawl speed may strain your server. This article also clarifies realistic timelines so you know when to expect your audit to finish.
🤖 Choosing the Right User Agent
The user agent tells the Site Auditor how to identify itself when crawling your pages. Search Atlas offers several options, and selecting the correct one affects how accurately your pages are rendered and captured.
Recommended user agent: Chrome Desktop
- Chrome Desktop — The best choice for most sites. It renders pages the way a real browser does, capturing JavaScript-dependent content, Open Graph tags, and dynamic elements accurately. Use this for the most complete and reliable audit results.
- Search Atlas Bot — A lightweight crawler that does not render JavaScript. It is faster but will miss dynamically loaded content, metadata, and tags generated client-side. This is not recommended if your site relies on JavaScript rendering.
- Googlebot — Mimics Google's crawler. Useful if you specifically want to see how Googlebot views your site, but Chrome Desktop remains the better general-purpose choice.
If your previous audits used Search Atlas Bot and returned incomplete Open Graph or metadata results, re-run the audit with Chrome Desktop selected to get a fuller picture of your site's issues.
🚀 Optimising Crawl Speed
Crawl speed controls how many pages per second the auditor requests from your server. For most sites — particularly those on shared hosting or with limited server resources — a lower speed is recommended to reduce the risk of overloading your server and to help ensure the audit completes without triggering rate limits or returning incomplete data.
- If your site is large and hosted on a dedicated or enterprise-grade server, you may use a higher crawl speed setting.
- If you notice server errors or timeouts during a crawl, reduce the speed further to a lower value.
To adjust crawl speed, open your Site Audit settings before starting or re-running a crawl and update the crawl speed to your preferred value before launching the audit.
⏱️ How Long Does a Site Audit Take?
Site Audit analysis time depends on the size of your site and the crawl speed you have selected. Larger sites and lower crawl speeds will naturally require more time to complete. If your audit appears stuck in analysis, check that your crawl speed is appropriately set for your server capacity and consider re-running the audit with Chrome Desktop as the user agent for the most reliable results.
If you need further assistance, open the chat widget in the bottom-right corner of the platform and type human teammate to be connected with a member of our team.