## **🔄 What an OTTO Smart Crawl does**

A Site Audit crawl visits your live website and refreshes the pages, content, and structure OTTO uses for SEO recommendations and automation. Running one manually is useful after a redesign, a new page launch, or major URL or content changes, so OTTO is not working from an older version of your site.

## **📋 Before you start the crawl**

Confirm these items so the crawl can see your updated site:

- **Publish the changes** on the live domain connected to your OTTO project.
- **Check the correct project** if you manage more than one website.
- **Allow crawling** in your robots.txt file and remove any temporary noindex tags from pages you want included.
- **Submit or update your sitemap** if important new URLs are not linked from existing pages.

## **🚀 Start a manual Smart Crawl**

1. Sign in to Search Atlas and open Site Audit for the project. From an OTTO project, use the project menu and choose Open Site Audit.
2. Select the website project you revamped.
3. Open the Site Audit project and go to its Website Overview page. The same control is also on the Crawl Logs page, which you can reach with "View most recent crawl log".
4. Click Recrawl Site. While a crawl is already running, the button reads "Crawling…" and cannot be clicked again.
5. Leave the crawl running. You can stay on the page or return later to check progress.
6. When the crawl finishes, the "Crawling…" button returns to "Recrawl Site" and the progress ring returns to a health label. The updated page list is available then. OTTO recommendations are rebuilt in a separate step that starts after the crawl, so give them a little longer to reflect the new content.

Run only one crawl at a time for the same project. Starting another crawl while one is in progress can delay results.

## **⏱️ What to expect after it finishes**

A completed crawl does not change your live site. It only refreshes Search Atlas's understanding of your pages, and OTTO recommendations are rebuilt from that afterward. Pages the crawl can reach — by following links on your site or by reading your sitemap — should then appear in the project, unless a crawl exclusion rule skips them. Large sites take longer than small sites. If a page is missing, confirm it is live, linked from your site or listed in your sitemap, and not blocked from crawlers or excluded by a crawl rule.

## **🛠️ If the new site still does not appear**

- Clear any maintenance mode, password protection, or firewall rules that block the crawler.
- Make sure redirects from old URLs point to the correct new pages.
- Wait until the current crawl shows complete, then run one new Smart Crawl.
- Check that you selected the project for the redesigned domain, not an older property.

## **💬 Need more help**

If you need further assistance, open the chat widget in the bottom-right corner of the platform and send your message. A member of our support team will reply in the chat.