Diagnose a page missing from Google

Estimated reading time : 3 min · Published October 5, 2026

Define what is actually missing

Record the URL, publication date, search query and intended destination. In a Search Console property you are authorised to access, compare URL inspection with the live test. Inspection describes Google’s known state; the live test checks the page currently accessible.

A site: query or title search alone is not a reliable index inventory. Separate four questions: does Google know the URL, can it crawl it, has it selected it for indexing and does it appear for useful searches? These require different checks.

Check technical obstacles

Open the URL without signing in, examine its HTTP status and redirect chain, then look for noindex in HTML and X-Robots-Tag in headers. Check robots.txt and server refusals. Test essential resources too if the content depends on JavaScript.

robots.txt controls crawling; it is not access protection and does not guarantee removal of a URL from results. A crawler must be able to access a noindex instruction to read it. Private pages need access control. Clarify whether the page should be public before removing protection.

Check URL and content consistency

Compare the declared canonical with Google’s selected URL where available. A canonical pointing elsewhere may explain exclusion of a variant. Menus, internal links, sitemaps and redirects should support the intended destination rather than signal contradictory versions.

If the page is accessible and properly signalled, assess its distinct value: does it answer a real question, provide specific information and belong to a useful internal journey? Adding repetitive paragraphs to nearly identical pages does not solve insufficient coverage. Consolidate equivalents where that helps the reader and use a relevant destination.

Correct and observe without promising timing

Retain the diagnosed problem, correction, date and successful checks. Update links and the sitemap if the URL changes. An indexing request can signal a correction but does not guarantee an inclusion date or ranking.

Recheck a representative set of pages rather than republishing the whole site after every fluctuation. If exclusion is intentional, document that decision. The useful result is a verifiable action: restore access, improve, consolidate, retain outside the index or protect with authentication.

Reference documents

Content updated on October 5, 2026

Record checks and findings in the acceptance log

Acceptance matrix to adapt to your project

These proposed checks use synthetic cases. Decide the required behaviour with the team, record the result and assign unresolved gaps before release.

Test cases, expected outcomes and useful evidence
Case	Expected outcome	Evidence to retain
Public access	The intended page works without login or error.	Status and live-test result.
Directives	Crawling and indexing match the intention.	robots.txt, meta robots and headers.
Canonical	The destination agrees with links and sitemap.	Declared URL and observed selection.
Correction	An identified problem has a verifiable response.	Date, test and state follow-up.

Frequently asked questions

Will every sitemap URL be indexed?

No. A sitemap helps discovery of proposed URLs. Crawling and indexing remain subject to technical checks and search-engine selection.

Should robots.txt be removed to fix the problem?

First check the relevant rule and whether the page should be public. Deleting the whole file does not fix noindex, server errors or conflicting canonicals and may open unnecessary crawl paths.

Should an indexed page with low visibility be rebuilt?

Start with intended queries, answer relevance and observed performance. Indexing and ranking are different issues; low visibility does not automatically justify a redesign.