Crawling and Indexation Control
Indexation Problems the GSC Pages Report Decoded
Read the Search Console Pages report line by line and turn each exclusion reason into a fix.
Want to earn a Rankar Academy certificate?
This lesson is free to read in full, with nothing to sign up for. Join the Academy to track your learning, complete courses and sit the certification assessment.
Join the Academy →Performance model
Technical SEO Excellence · Crawling and Indexation Control
Stage 1
Indexed versus not indexed
Stage 2
Reading the common exclusion reasons
Stage 3
Prioritising fixes
Indexed versus not indexed
The Pages report splits URLs into Indexed and Not indexed, then subdivides the latter into specific reasons. Treat this report as diagnostic, not just a scoreboard: a healthy site still has excluded URLs for legitimate reasons like pagination or redirects, so the goal is matching each reason to intent, not driving the excluded count to zero.
Reading the common exclusion reasons
'Crawled - currently not indexed' means Google fetched the page but chose not to index it, usually due to thin or duplicate content, and is the reason worth the most attention. 'Discovered - currently not indexed' means Google knows the URL exists but has not crawled it yet, often a crawl budget or internal linking signal. 'Alternate page with proper canonical tag' and 'Duplicate without user-selected canonical' both point to duplication, but the second means Google overrode your signals entirely.
- Crawled - currently not indexed: quality issue, improve or merge content
- Discovered - currently not indexed: strengthen internal links, check crawl budget
- Duplicate without user-selected canonical: add explicit rel=canonical
- Soft 404: page returns 200 but reads as empty or error-like to Google
- Blocked by robots.txt: intentional or accidental exclusion
Prioritising fixes
Sort excluded URLs by estimated traffic potential, using internal search volume data or historical rankings before the exclusion appeared, and fix the highest-value pages first. For 'crawled - currently not indexed' at scale, look for a shared template weakness, such as thin product descriptions, rather than fixing one URL at a time.
Requesting indexing responsibly
URL Inspection's 'Request indexing' is useful for a handful of high-priority pages after a genuine fix, but it has a low daily quota and does not override an underlying quality problem. Resubmitting the same thin page repeatedly wastes quota and rarely changes the outcome.
Tracking exclusions over time in RankAudit
RankAudit's indexation tracker snapshots your Pages report categories weekly and highlights URLs that moved from Indexed to an exclusion reason, so regressions are caught within days rather than discovered months later in a traffic drop.
Run a site crawl, read the health score, prioritise issues and hand fixes to the team.
Why this lesson matters
Ignoring indexation issues identified in GSC directly translates to lost organic visibility and revenue. Imagine an e-commerce site with thousands of product pages. If a significant percentage of these pages are excluded due to "duplicate without user-selected canonical" or "blocked by robots.txt" errors, those products effectively do not exist to search engines. Each unindexed page represents a missed opportunity for a potential customer to find and purchase that product, impacting sales conversion rates and overall business growth.
Conversely, mastering the GSC Pages report allows you to ensure all valuable content is indexable and discoverable. Consider a SaaS company that publishes extensive documentation or case studies. If these critical resources are not indexed, their authority and utility diminish significantly, hindering lead generation and customer support deflection. Proactive diagnosis and resolution of indexation problems ensure that your SEO efforts, from keyword research to content creation, culminate in tangible organic traffic and business value, rather than being wasted on invisible pages.
Prioritising Exclusions: Impact vs. Effort Matrix
Effective indexation management isn't about fixing every exclusion immediately, but rather applying an 'Impact vs. Effort' matrix to prioritise. High-impact exclusions, like critical product categories or key service pages marked as 'Noindexed by ‘noindex’ tag,' demand immediate attention due to their direct potential to generate revenue or leads. Conversely, low-impact exclusions, such as historical blog comment pages deemed 'Crawled - currently not indexed' (and which genuinely offer little user value), might be deprioritised or even accepted as a natural part of site management, preventing resource drain.
The GSC report doesn't inherently prioritise by business value, so it's up to the SEO practitioner to overlay this context. This involves understanding your site's architecture, its most valuable content, and the potential traffic implications of each exclusion type. For instance, a "Page with redirect" for an old, non-canonical URL carries much lower urgency than "Discovered - currently not indexed" for a newly launched product page. This strategic approach ensures resources are allocated to fixes that yield the greatest return on investment for organic performance.
- Identify revenue-critical pages first.
- Assess potential organic traffic loss per exclusion.
- Estimate developer or content team effort for each fix.
- Focus on high-impact, low-effort resolutions initially.
Do it now
Immediately open your GSC property and navigate to the 'Pages' report. Focus on the 'Why pages aren't indexed' section. Select the exclusion reason with the highest number of affected URLs that you suspect holds high business value. Do not request validation yet. Instead, open RankAudit and navigate to the 'GSC Exclusions' section, ensuring your account is connected. This allows you to track and manage the specific exclusion type you've identified, providing historical context and impact over time.
Here are your next steps:
- Filter your GSC 'Pages' report by the chosen exclusion reason.
- Export the list of affected URLs.
- Cross-reference a sample of these URLs with your site's structure and business objectives.
- Determine if the exclusion is an error to fix or an intended state.
Key takeaways
- ✓Match each exclusion reason to a specific root cause, not a generic fix
- ✓Prioritise 'crawled - not indexed' pages by traffic potential
- ✓Look for shared template issues behind repeated exclusions
- ✓Only request indexing after making a real content change
Why this lesson matters
This lesson belongs to Crawling and Indexation Control, the part of Technical SEO Excellence where the goal is: give googlebot precise instructions and confirm every page you care about is actually indexed.
Read it once, then do it straight away on a real site inside RankAudit. Nothing here is theory for its own sake — every step produces something you can show a client.
Do it now
- 1Open RankAudit with sample data already loaded, so you are not stuck on setup.
- 2Crawl a site and triage the issue list.
- 3Run a crawl in RankAudit and cross-check the robots.txt, sitemap and index coverage reports it produces.
