# **🔍 What Are Stale Crawl Sources and Phantom Records?**

When your website undergoes a major change — such as a domain migration, a site rebuild, or a hack cleanup — your Brand Vault may continue to reference URLs and crawl data from the old version of your site. These are called **stale crawl sources**.

Alongside stale sources, you may also encounter **phantom processing records**: leftover data entries that appear in your Brand Vault's queue even though the underlying pages no longer exist or have changed significantly. These phantom records can cause issues such as:

- Your knowledge base reflecting outdated or incorrect site content
- Timeouts or errors when generating content in bulk with Content Genius
- OTTO SEO referencing old page data during automation tasks
- Slower Brand Vault performance overall

# **🗂️ When Should You Take Action?**

You should clean up your Brand Vault after any of the following events:

- A domain or subdomain migration
- A full site rebuild or CMS change
- A security incident that required bulk URL removal or restructuring
- A large-scale content audit that removed or redirected many pages
- You notice Content Genius bulk generation timing out or producing irrelevant output

# **⚙️ Step 1 — Identify Your Stale Crawl Source**

Before removing anything, confirm which crawl source is outdated.

1. Access your Brand Vault: top-right **More Features** menu (grid icon, next to your account menu) → **Brand Vault**, (Brand Vault is accessed via the **More Features** menu.)
2. Once inside, select the relevant workspace or project.
3. Locate the **Crawl Sources** section. Each source is listed with a unique identifier (for example, a UUID such as **e213e8ca-cc02-492c-be44-8f8fdee44479**).
4. Review the source URL and the **last crawled** date. If the URL belongs to a pre-migration version of your site or returns errors, it is stale and safe to remove.

# **🗑️ Step 2 — Delete the Stale Crawl Source**

1. In the **Crawl Sources** section, click the stale source you identified.
2. Select **Delete** or the trash icon next to the source entry.
3. Confirm the deletion when prompted. This removes the source and queues all associated crawl data — including page snapshots, media references, and indexed chunks — for cleanup.

**Note:** Deleting a crawl source is permanent. Make sure the source is genuinely outdated before proceeding. If you are unsure, add your new site URL as a fresh crawl source first so your Brand Vault has up-to-date data before you remove the old one.

# **🧹 Step 3 — Clear Phantom Processing Records**

After deleting the stale crawl source, phantom records tied to that source will be marked for removal. In most cases, the platform clears these automatically within a short processing window. To verify the cleanup has completed:

1. Return to your Brand Vault and check the **Processing Queue** or **Indexed Pages** count.
2. Wait up to **30 minutes** for the system to finish purging orphaned records. The exact time depends on how many records exist — larger sets (such as 1,000+ entries) may take slightly longer.
3. Refresh the Brand Vault page. The processing count should return to zero or reflect only your current, active crawl sources.

If phantom records remain after 30 minutes, do not delete and re-add your source repeatedly — this can create duplicate entries. Instead, wait a full hour and refresh again before escalating.

# **🚀 Step 4 — Add Your Updated Site as a New Crawl Source**

Once the stale data is cleared, add your post-migration site to give Brand Vault fresh, accurate content to work with.

1. In Brand Vault, go to **Crawl Sources** and select **Add Source**.
2. Enter your current site URL (the live, post-migration domain).
3. Choose your preferred crawl depth and scope settings.
4. Click **Start Crawl**. The platform will index your updated pages and populate the Brand Vault with current data.

Once the crawl is complete, Content Genius bulk generation and OTTO SEO automation will reference your updated knowledge base.

# **💡 Tips to Prevent This Issue in the Future**

- **Before a migration:** Note your existing crawl source IDs so you can identify and remove them quickly after go-live.
- **After a migration:** Add your new domain as a crawl source immediately and delete the old one within the same session.
- **After a security incident:** Remove any crawl sources that indexed spam or injected URLs before running Content Genius or OTTO SEO tasks.
- **Routine audits:** Periodically review your Brand Vault crawl sources to ensure all entries point to live, relevant URLs.

# **💬 Need More Help?**

If you need further assistance, open the chat widget in the bottom-right corner of the platform and type **human teammate** to be connected with a member of our team.