## **🔍 Why This Error Appears in Search Atlas**

When Search Atlas crawls your site, it flags a **crawl-delay** directive in your **robots.txt** file as a critical error. This directive instructs search engine bots to pause between requests, which can significantly slow down how quickly Google and other search engines index your pages.

If your store runs on Shopify, you have likely already discovered that you cannot simply open and edit your robots.txt file the way you would on a self-hosted site. This is expected behavior — it is a platform-level restriction, not a problem with your account or Search Atlas.

## **🛍️ Why Shopify Locks Your robots.txt File**

Shopify automatically generates a **robots.txt** file for every store. On most plans, this file is managed by Shopify's infrastructure and is not directly editable through the Shopify admin panel or theme editor by default. The **crawl-delay** directive you see is typically injected by Shopify itself or by a third-party app installed on your store.

Because of this, standard advice such as "open your robots.txt file and delete the line" does not apply to Shopify stores. You need to use one of the platform-specific methods described below.

## **✅ Method 1: Check for Apps Injecting the Directive**

The most common cause of a **crawl-delay** directive on Shopify is a third-party SEO or security app that modifies your robots.txt output. Follow these steps to identify and remove the source:

1. Log in to your **Shopify Admin** panel.
2. Go to **Apps** in the left sidebar.
3. Review any installed SEO, site speed, or security apps such as SEO Manager, Plug in SEO, or bot-protection tools.
4. Open each relevant app's settings and look for a **robots.txt** or **crawl settings** section.
5. If you find a crawl-delay setting, disable it or set the value to **0** and save.
6. Wait 24–48 hours, then re-run a crawl in Search Atlas to confirm the error is resolved.

If you are unsure which app is responsible, temporarily disable SEO-related apps one at a time and check your live robots.txt file at **yourdomain.com/robots.txt** after each change to identify the source.

## **✅ Method 2: Use a Custom robots.txt Template (Shopify 2.0 Themes)**

If your store uses a **Shopify Online Store 2.0** theme, Shopify allows you to override the default robots.txt file using a Liquid template. This method gives you full control over the file's content.

1. In your **Shopify Admin**, go to **Online Store → Themes**.
2. Next to your active theme, click **Actions → Edit code**.
3. In the left file panel, look for a folder called **Templates**.
4. If a file named **robots.txt.liquid** already exists, click it to open it. If it does not exist, click **Add a new template**, select **robots.txt** from the dropdown, and click **Create template**.
5. Review the template content and remove any line that contains **crawl-delay**.
6. Click **Save**.
7. Visit **yourdomain.com/robots.txt** to confirm the directive is no longer present.

**Important:** When creating a custom robots.txt.liquid template, Shopify replaces its automatically generated file entirely with your template. Make sure the template retains all necessary **Disallow** and **Sitemap** directives from the original. You can copy the current content of your live robots.txt as a starting point before making edits.

## **✅ Method 3: Contact Shopify Support**

If your theme is not a 2.0 theme, the robots.txt.liquid template option is unavailable, and no installed app appears to be the source of the directive, the crawl-delay may be hard-coded by Shopify at the infrastructure level. In this case, direct editing is not possible without Shopify's involvement.

Contact **Shopify Support** directly and provide the following information:

- Your store URL and the current content of your robots.txt file.
- The specific **crawl-delay** line you need removed.
- A note that the directive is causing indexation issues flagged by your SEO tooling.

Shopify support can advise whether an upgrade to a 2.0-compatible theme or a platform-level change is required.

## **📊 Verifying the Fix in Search Atlas**

After applying any of the methods above, confirm the error has been resolved in the Search Atlas dashboard:

1. In the left sidebar, navigate to **OTTO SEO V3**.
2. Open your active project and locate the site audit or technical SEO section.
3. Trigger a new crawl of your site.
4. Once the crawl completes, confirm the **crawl-delay critical error** no longer appears in your issue list.

If the error persists after the crawl, revisit your robots.txt file directly at **yourdomain.com/robots.txt** to verify the directive has been fully removed before re-crawling.

## **💬 Need More Help?**

If you need further assistance, open the chat widget in the bottom-right corner of the platform and type **human teammate** to be connected with a member of our team.