## **🧭 Overview**

If your Site Audit crawl has been running for an extended period with little or no progress, your site is likely **stalled**, not just slow. On Shopify stores, a common cause is **Cloudflare blocking Search Atlas crawl requests** before they reach your site. This article helps you confirm the root cause and take the right next steps.

## **⚠️ Slow Crawl vs. Stalled Crawl: Know the Difference**

- **Slow crawl (normal):** Pages are being discovered and audited gradually. Progress increments over time. Large sites with many pages can take several days.
- **Stalled crawl (problem):** The crawl shows little or no progress over an extended period. The page count stays frozen or advances by only a handful of pages per day. This is a blocking issue, not a size issue.

If your crawl has been running for a prolonged period with minimal progress, proceed through the steps below in order. **Do not skip to sitemap fixes** — if Cloudflare is blocking the crawler, sitemap changes will have no effect.

## **🔍 Step 1: Confirm Cloudflare Is Active on Your Shopify Store**

1. Visit your Shopify store URL and open your browser's developer tools (**F12** → Network tab).
2. Look for response headers containing **cf-ray** or **server: cloudflare**. If present, Cloudflare is active on your domain.
3. Alternatively, check your DNS or hosting settings to confirm whether your domain is proxied through Cloudflare.

If Cloudflare is confirmed active, continue to Step 2 before doing anything else.

## **🛡️ Step 2: Configure Cloudflare to Allow Search Atlas Crawler Requests**

Cloudflare's bot protection and firewall rules can silently block automated crawlers, returning errors or empty responses that stall your audit. You will need to configure Cloudflare to allow the Search Atlas crawler through.

1. Log in to your **Cloudflare account** and select your domain.
2. Review your active security and bot-protection rules to identify any that may be blocking or challenging automated crawl requests.
3. Contact our support team to obtain the current list of Search Atlas crawler IPs and guidance on how to create the appropriate allow rules for your specific Cloudflare configuration.
4. Once the rules are in place, return to Search Atlas and restart your Site Audit crawl to verify that progress resumes.

## **📋 What to Have Ready When Contacting Support**

If you are unsure which settings to change, or if the crawl remains stalled after reviewing your Cloudflare configuration, please have the following information ready so our team can assist you efficiently:

- Your **project name** in Search Atlas
- Your **Shopify store URL**
- Confirmation that Cloudflare is active on the domain (e.g., cf-ray header present)
- A description of how long the crawl has been stalled and approximately how many pages have been audited
- Any error messages or unusual response codes you have observed

If you need further assistance, open the chat widget in the bottom-right corner of the platform and type **human teammate** to be connected with a member of our team.