A Practical Guide to Crawl Traps in Filtered Navigation
The crawling and indexing problems created by filter and sort combinations on category pages.
Samir Haddad
The key figures
- URL explosion
- each filter combination can create a new URL
- Crawl budget
- crawlers spend time on low-value combinations
- Canonical tags
- can point filtered pages at the main category
- Robots rules
- some parameter paths can be blocked from crawling
Why this is worth getting right
Filters can generate millions of URLs from a small catalog, wasting crawl budget and creating near-duplicate pages.
Do this, not that
Do
- Decide which filter pages deserve indexing
- Canonicalize low-value combinations
- Keep indexable filters in clean URLs
- Block infinite combinations from crawling
- Monitor crawl stats after changes
Don’t
- Indexing every filter combination
- Blocking pages that earn traffic
- Filters generating session-based URLs
- Leaving parameter handling to chance
When to bring in help
Our advice Bring in help when large catalogs create crawl problems.
Where this comes from
- Google Search Central — Faceted navigation best practices
- Google Search Central — Crawl budget management
The figures and practices above come from the sources listed.
Working on something like this?
We take on SEO Services work for teams who want it done once, properly. Tell us what you are building and we will tell you honestly whether we are the right studio for it. Start a project.
Where to go next
Spotted something wrong? Report an error on this page. We correct on the page and say what changed.
