Free Generator

Robots.txt Generator

Generate a robots.txt file optimized for your ecommerce platform. Select your platform and options, then copy the output to your site root.

PLATFORM

OPTIONS

Block AI training crawlers

Blocks GPTBot, CCBot, Claude, Perplexity, and other AI scrapers

Block Bingbot

Prevents Bing from indexing your store (~3% of search traffic)

Explicit allow for search engines

Adds explicit User-agent blocks for Google, Bing, Yahoo, DuckDuckGo

Include Sitemap URL

Adds a Sitemap: directive pointing to your sitemap.xml

Sitemap URL

HOW TO DEPLOY

Copy the generated robots.txt content
In Shopify admin: Online Store → Themes → Edit Code
Open config/robots.txt.liquid (Shopify 2.0)
Replace or customize the content and save

robots.txt

# Shopify-optimized robots.txt
# Note: Shopify 2.0 uses robots.txt.liquid — paste this there

User-agent: *
Allow: /

# Block Shopify admin and internal paths
Disallow: /admin
Disallow: /cart
Disallow: /orders
Disallow: /checkouts
Disallow: /checkout
Disallow: /account
Disallow: /cgi-bin/
Disallow: /cdn/wpm/

# Block search/filter parameter combinations that create duplicate content
Disallow: /*?sort=
Disallow: /*?q=
Disallow: /*?view=
Disallow: /search

Sitemap: https://yourstore.com/sitemap.xml

After deployingRun a Robots.txt Analyzer scan to verify your new file is correct, or run a full health scan to check 20 categories of store health including crawlability.

Want a full health report?

Our full scan checks 20 categories across SEO, security, performance, accessibility, and content quality.

Run Full Scan Free