Skip to content
Ecom

The Shopify Search Console Setup Most Agencies Get Wrong

Property type, domain vs URL prefix, /collections vs /products indexing, and the robots.txt.liquid patterns we use to keep Shopify from indexing every filter combination.

By WitsCode10 min read

Most Shopify stores that come to us for an SEO audit have Search Console set up. They have a property verified, a sitemap submitted, and a dashboard with coverage data in it. What they almost never have is a setup that actually reflects what Google is doing with their store. We have run this audit across more than two hundred and fifty Shopify builds at WitsCode, and the same handful of Search Console mistakes show up every single time. Wrong property type. Wrong verification method. Unfiltered faceted URLs burying the useful coverage data. No plan for the collections duplication problem. Hreflang and international markets missing from the property tree entirely.

None of this is obscure. Google and Shopify both document the fix. The issue is that the defaults look fine, and nobody audits the defaults until organic traffic plateaus and someone asks why the Performance report shows fewer queries than the store has collections. This article walks through the setup we now do on every Shopify engagement, why each decision matters, and the robots.txt.liquid patterns we use to stop Shopify from feeding Google a catalog that is fifty times larger than the one the merchant actually sells.

Domain property first, URL-prefix properties second

The single biggest setup mistake is using a URL-prefix property as the primary Search Console view. A URL-prefix property only covers the exact protocol, host, and path you verify. If you verify https://www.brand.com, you see nothing from https://brand.com, nothing from the myshopify.com mirror, nothing from the m. subdomain if you ever had one, and nothing from a checkout subdomain. On a Shopify store that is in the middle of a domain move, or that still has myshopify traffic ranking during a migration, or that has legacy redirects pointing at the apex, a URL-prefix property will silently underreport.

A Domain property covers every subdomain and both protocols under one verified zone. You verify it once, with a DNS TXT record, and you get https, http, www, apex, m, shop, and myshopify traffic rolled into a single set of reports. For a Shopify store this is the property type you want as your source of truth. DNS verification is the only method Domain properties support, which means you need access to the DNS host. On Shopify-managed domains this is under Settings, Domains, the domain name, then DNS settings. On external DNS hosts it is a standard TXT record add. Five minutes, one propagation wait, done.

URL-prefix properties still have a role. Once the Domain property is verified, add URL-prefix properties for any subfolder you want to isolate. The common case is international markets. If you run Shopify Markets with subfolders, create a URL-prefix property per market: https://brand.com/en-us/, https://brand.com/en-gb/, https://brand.com/fr/. Each one gives you market-level query data, click-through rates, and coverage numbers that the Domain property blurs together. During a domain migration, also add a URL-prefix property for the outgoing domain and the myshopify.com mirror so you can watch traffic decay and catch any URL that is still receiving impressions after the 301 should have moved it.

Verification method matters more than people think

Shopify exposes a field under Online Store, Preferences, Google Search Console where you can paste a meta-tag verification string. It works. It is also the most fragile option available. A theme update, a theme swap, or a developer rebuild can wipe that field without anyone noticing, and Search Console will eventually de-verify the property and stop collecting data.

DNS verification, which Domain properties require anyway, avoids all of this. The TXT record lives in the DNS zone, not in the theme. Theme changes cannot break it. Developer handoffs cannot break it. The only way to lose verification is to change DNS, which tends to be a deliberate act. We verify every Shopify store via DNS even when the merchant has a working meta tag already, and we remove the meta tag from the Shopify Preferences field once DNS verification is confirmed. Two verification methods on one property is not extra insurance, it is two places something can go wrong.

The faceted URL explosion nobody is looking at

Here is the pattern we see on almost every unaudited Shopify store. The Coverage report shows a few thousand valid pages and several hundred thousand excluded pages. The merchant sees the valid number, assumes everything is fine, and never clicks into the excluded bucket. Inside the excluded bucket is a parade of URLs like this one.

/collections/mens-jackets?sort_by=price-ascending&filter.v.option.size=medium&filter.v.price.gte=50&filter.p.product_type=parka&page=3

Shopify collection pages accept a long list of URL parameters. sort_by controls the sort order. filter.v and filter.p prefixes drive faceted filters across variant options and product properties. pb controls paginated product block layouts on some themes. q drives internal search. page handles pagination. Every combination is a crawlable URL. A catalog with five filter dimensions and four or five options per dimension produces tens of thousands of unique URLs from a single collection, and a store with twenty collections produces hundreds of thousands.

Google does not index most of these. It crawls them, spends crawl budget on them, evaluates them, and then puts them in one of the exclusion buckets: Crawled currently not indexed, Duplicate without user-selected canonical, or Alternate page with proper canonical tag. The store ranks fine on the clean URLs. Crawl budget, though, is being burned on filter combinations nobody searches for, and the signal to Google is that the store has an enormous amount of thin, near-duplicate content.

Search Console does not fix this. robots.txt does.

The robots.txt.liquid override pattern

Shopify used to hardcode robots.txt. You could not edit it. In June 2021 Shopify released the robots.txt.liquid template, which lives under templates in the theme code editor and lets you customize the generated robots file using Liquid. This is the supported mechanism for Shopify robots customization. Apps that promise to edit robots.txt for you are almost always just writing to this template.

The template starts by iterating over robots.default_groups, which produces Shopify's default rules. You append your own Disallow and Allow directives inside the user-agent block. The rules we add on every Shopify SEO engagement look like this.

Disallow: /*?*sort_by=
Disallow: /*?*filter.
Disallow: /*?*pb=
Disallow: /*?*q=
Disallow: /search
Disallow: /*?variant=
Disallow: /a/downloads/
Disallow: /apps/

The first three lines kill the faceted URL explosion. Anything on the site with a sort_by, filter., or pb parameter becomes uncrawlable. Google stops spending crawl budget on filter permutations and the excluded-URL number in Coverage drops by one or two orders of magnitude within a few weeks. The canonical collection URL still ranks, because that one has no parameters and is not disallowed.

The q and /search lines stop Shopify's internal search results pages from being crawled. Internal search result pages are low value and generate infinite URLs as queries vary. The variant line is optional and depends on whether variant-specific URLs have any SEO value to the merchant. For most apparel and homeware stores they do not, and blocking them removes another duplication vector. The /a/downloads and /apps lines target two places where Shopify apps typically generate crawlable junk, digital download links and app proxy endpoints, neither of which should be in the index.

One thing to check before shipping these rules. If the store is running a proper app like a wishlist or a loyalty program behind /apps, the Disallow on /apps can affect it. Review the proxy paths in Apps settings and carve out Allow rules for any app proxy that needs to be crawlable. The Shopify community forum has a long thread of cases where merchants blanket-disallowed /apps and then wondered why their Judge.me reviews stopped rendering on search results. Scoped rules, not blanket bans.

The /collections/all problem

Every Shopify store has a URL at /collections/all that lists every product in the catalog. Shopify generates it automatically. Nothing on the storefront links to it prominently, but Google finds it anyway through sitemap.xml and internal linking from product pages. If the store has category-level collections, and most stores do, /collections/all becomes a duplicate superset of every other collection page. A product that lives in /collections/mens-jackets also appears at /collections/all, with similar but not identical product ordering.

Shopify's default canonical for /collections/all points to itself, not to any parent. It is its own page, as far as the canonical signal is concerned, which means Google treats it as a distinct URL to evaluate. We handle this one of three ways depending on the store.

The cleanest fix is a 301 redirect from /collections/all to the homepage or to the primary hub collection. This is set up under Navigation, URL Redirects. It removes the duplication entirely and consolidates any backlinks that happened to point at /collections/all into the homepage's link equity.

If the merchant wants /collections/all to exist as a working page for some reason, for example because an app or a theme component links to it, the second option is a noindex meta robots tag added conditionally in theme.liquid, checking whether template contains 'collection' and handle is 'all'. The page stays crawlable and functional, but drops out of the index.

The third option is a Disallow: /collections/all line in robots.txt.liquid. This works, but it is slightly less clean than the noindex approach because disallowed pages can still appear in the index as URL-only entries if they have enough external links pointing at them. Noindex strips them out completely. We use Disallow only when the theme cannot be edited.

International properties and the hreflang trap

Shopify Markets outputs hreflang link tags automatically when you have multiple markets configured with subfolder routing. You can see them in the page source of any storefront URL, a block of link rel="alternate" hreflang entries pointing to each market's version of the current URL. This mostly works. The trap is that many theme developers, especially on older themes, strip or override the default head content and lose the hreflang output in the process.

We audit hreflang on every international Shopify store before trusting what Search Console reports. View source on a product URL, confirm that every market has a corresponding hreflang link, confirm that the x-default hreflang points to the correct market, and confirm that the URLs in those hreflang tags return a 200 and not a redirect. Google's International Targeting report, now folded into Search Console's Legacy tools, used to surface hreflang errors cleanly. That report is gone, so the monitoring has to be done manually or through a third-party crawler.

The Search Console property structure for an international Shopify store looks like this. One Domain property on brand.com as the master view. One URL-prefix property per market subfolder: brand.com/en-us/, brand.com/en-gb/, brand.com/fr/, brand.com/de/. Each URL-prefix property submits a separate sitemap, which Shopify Markets generates per market at /sitemap.xml with market-scoped content. Together the properties let you see per-market query data without losing the full-site picture.

If the store is using a multi-store setup on separate top-level domains, for example brand.com and brand.co.uk as independent Shopify stores rather than subfolders of one, then each domain gets its own Domain property, each with its own DNS verification, and hreflang has to be configured manually through theme code because Shopify Markets does not span stores. This is the harder setup and it is where most international SEO issues live.

Sitemap submission cleanup

Shopify auto-generates /sitemap.xml as an index sitemap that points to sitemap_products_1.xml, sitemap_collections_1.xml, sitemap_pages_1.xml, and sitemap_blogs_1.xml. Most agencies submit only the top-level /sitemap.xml, which works but hides per-type coverage data. In Search Console, submit the child sitemaps individually as well. The Sitemaps report will show you how many products Google discovered versus how many collections versus how many blog posts, which is the fastest way to spot that, for example, your product sitemap has four thousand URLs but your indexed product count is twelve hundred, and something is wrong in the product template.

On Shopify's Basic plan and above the sitemap is exposed by default. On trial and development stores it is blocked along with the rest of the storefront, so do not waste time submitting sitemaps until the store is on a paid plan with the password removed.

The setup checklist we actually use

On every Shopify SEO engagement we run the same Search Console setup pass before any on-page work starts. Verify the Domain property via DNS. Remove any existing meta-tag verification from Shopify Preferences once DNS is confirmed. Add URL-prefix properties for each market subfolder if the store is international. Submit /sitemap.xml and each child sitemap individually. Edit robots.txt.liquid to disallow sort_by, filter., pb, q, search, and variant parameter patterns, plus any app-generated paths that do not need indexing. Decide the /collections/all strategy, redirect or noindex, and implement it. Confirm hreflang is present and correct in page source on a sample of URLs. Pull the Coverage report baseline so the drop in excluded URLs can be measured over the following month.

That is the setup. It is not complicated and it takes an afternoon on most stores. The reason it is worth writing down is that almost no Shopify store we have audited has all seven items right on the first pass. Usually it is one or two. Sometimes it is zero. The gap between a store that is doing this correctly and a store that is not is thousands of wasted crawl requests, a coverage report that nobody can read, and organic traffic that looks flat because Google is spending its attention on URLs nobody should be indexing.

If you want the same audit run across your Shopify store before you spend another quarter on content or backlinks, -> the WitsCode SEO audit is where we start every engagement, and Search Console setup is always in the first ten findings.

Get weekly field notes.

Practical writing on shipping products, straight to your inbox. No spam.

Need help with this?

Shopify Development

We design and build web apps, MVPs, and SaaS products. Talk to us about what you are working on.

Talk to us

Want to discuss ecom for your business?

Start a project and we'll talk through where you are, what's working, and the highest-leverage moves for the next 90 days.