Fix Duplicate Content: Consolidate URLs With 301 or Canonical

What's happening

Duplicate content is the same or near-identical content served at multiple URLs on a domain. Sources include URL parameters (?utm_*,?sort=,?ref=), trailing slash variants, www vs apex domain, http vs https, faceted filter pages, archive pagination, AMP variants, mobile m. subdomains, and country TLDs without hreflang.

Google does not penalize duplicate content algorithmically — there is no 'duplicate content penalty' — but it does consolidate signals to a single canonical URL, picking which one based on internal links, redirects, sitemap inclusion, and rel=canonical hints. When the consolidation picks the wrong URL, your preferred URL drops out of the index and the wrong variant ranks.

Search Console's Page indexing report flags duplicates as 'Duplicate, Google chose different canonical than user', 'Duplicate without user-selected canonical', or 'Alternate page with proper canonical tag'. Each indicates a different consolidation outcome and a different fix path.

Why it matters

Link equity dilution. External links pointing to /page and /page?ref=twitter are treated as separate URLs by Google when no canonicalization exists. Each accumulates its own signals, splitting the consolidation.

Wrong URL ranks. When Google's canonical picker chooses a parameter URL or a trailing-slash variant, your preferred URL drops out of the index. Direct traffic still works, but search traffic goes to the wrong canonical.

Crawl budget waste. Google spends crawl budget on duplicate variants instead of new content. On large sites with thousands of parameter URLs this is material — a 30-50% crawl-budget waste is common before consolidation is fixed.

Common causes

• URL parameters (utm_*, gclid, fbclid, ref) creating endless variants of the same page.
• Trailing slash inconsistency — both /page and /page/ serving same content with no redirect.
• www vs apex domain not consolidated via 301.
• http vs https not redirected to https.
• Faceted filter URLs (?color=red&size=10) not canonicalized to the parent.
• AMP version not canonicalized to the canonical HTML version.
• Tag/category archives serving duplicate excerpts of the same articles.

Detect this on your site

Run a quick scan with the SEO Auditor. The tool surfaces this exact issue with the records and context needed to apply the fix below.

How to fix it

1. 1

   Identify duplicate URL patterns

   Crawl the site and group URLs by content fingerprint (HTML hash or main-content excerpt). The SEO Auditor surfaces parameter-driven duplicates, www/apex variants, http/https variants, and trailing-slash variants. Search Console's Page indexing > Excluded report also surfaces 'Duplicate' classifications.

2. 2

   Pick one canonical URL form

   Decide: https vs http (https), www vs apex (pick one), trailing slash on or off (pick one). Apply the choice consistently across server redirects, sitemap, internal links, and rel=canonical. The framework's middleware should redirect all variants to the canonical form.

3. 3

   301 redirect non-canonical variants

   http/https, www/apex, and trailing-slash variants should 301 redirect to the canonical at the server or CDN layer. 301 is permanent and passes link equity; 302 is temporary and does not. Set up the redirects in middleware or.htaccess / nginx.conf.

4. 4

   Use rel=canonical for parameter URLs

   URLs with tracking parameters (utm_*, gclid) should not redirect — they need to preserve the parameter for analytics — but should declare rel=canonical pointing to the parameter-stripped version. Same for sort/filter URLs that should consolidate to the parent.

5. 5

   Configure URL parameter handling in Search Console

   The legacy URL Parameters tool was deprecated in 2022 — Google now infers parameter behavior automatically. The replacement is to canonicalize correctly via rel=canonical and let Google do the rest. Audit Search Console > Page indexing for residual issues.

6. 6

   Use noindex for true duplicates that must stay accessible

   Some duplicates serve real user purposes (filtered views, sort orders) but should not appear in search. Add noindex via meta tag or X-Robots-Tag header on those URLs. Do not block in robots.txt — Google needs to crawl them to see the noindex.

Related fixes