Estimated reading time : 8 min · Published
Define what is actually missing
Record the URL, publication date, search query and intended destination. In a Search Console property you are authorised to access, compare URL inspection with the live test. Inspection describes Google’s known state; the live test checks the page currently accessible.
A site: query or title search alone is not a reliable index inventory. Separate four questions: does Google know the URL, can it crawl it, has it selected it for indexing and does it appear for useful searches? These require different checks.
Scroll the table horizontally to read every column. With a keyboard, focus the table area and use the arrow keys.
| Situation or setting | What it means | Useful check |
|---|---|---|
| Indexed-version data | Describes the version Google knew when it collected the page. | Compare collection and fix dates before inferring failure. |
| Live test succeeded | The page appears accessible to the tool at test time. | This guarantees neither indexing nor appearance for a particular query. |
| Declared and selected canonicals differ | The declared preference differs from Google’s known choice. | Check content, redirects and links; the live test does not predict canonical selection. |
Check technical obstacles
Open the URL without signing in, examine its HTTP status and redirect chain, then look for noindex in HTML and X-Robots-Tag in headers. Check robots.txt and server refusals. Test essential resources too if the content depends on JavaScript.
robots.txt controls crawling; it is not access protection and does not guarantee removal of a URL from results. A crawler must be able to access a noindex instruction to read it. Private pages need access control. Clarify whether the page should be public before removing protection.
Look for the instruction in the response that actually serves the content. An HTML page can deliver noindex in a meta tag; a PDF can deliver it through the X-Robots-Tag HTTP header without an HTML tag. A 200 response with noindex may therefore be an intentional exclusion. Google does not support a noindex rule written in robots.txt.
For Google to read noindex, its crawler must be able to access the resource. Simultaneously blocking crawling can prevent that instruction from being read. After a correction, record the current test separately from the last known crawl date: a successful live test does not instantly update the index or predict the selected canonical. Confidential resources remain protected by authentication independently of these search signals.
Scroll the table horizontally to read every column. With a keyboard, focus the table area and use the arrow keys.
| Observed situation | What Google can read | Useful check |
|---|---|---|
| Accessible HTML with meta noindex | The page instruction when the crawler fetches it. | Confirm that exclusion is intended before changing the tag. |
| Accessible PDF with X-Robots-Tag noindex | The instruction in the file’s HTTP header. | Check the PDF response itself, not just its linking page. |
| Crawling blocked and noindex on the resource | The block may prevent reading noindex. | Compare publication intent, robots.txt and the response accessible to the crawler. |
| Successful live test after correction | An accessible state at the time of testing. | Then compare the known crawl; do not infer guaranteed indexing. |
- Google Search Central : noindex
Consulted on 22 October 2026; tag, HTTP header and the need for the crawler to access the instruction.
- Google Search Console : URL Inspection
Consulted on 22 October 2026; indexed data versus live testing and limitations of the result.
Check URL and content consistency
Compare the declared canonical with Google’s selected URL where available. A canonical pointing elsewhere may explain exclusion of a variant. Menus, internal links, sitemaps and redirects should support the intended destination rather than signal contradictory versions.
If the page is accessible and properly signalled, assess its distinct value: does it answer a real question, provide specific information and belong to a useful internal journey? Adding repetitive paragraphs to nearly identical pages does not solve insufficient coverage. Consolidate equivalents where that helps the reader and use a relevant destination.
Follow the address chain before interpreting an inspection verdict. Fictional example: https://example.com/ancienne/ redirects over HTTP to https://example.com/nouvelle/. Google says live testing follows redirects before examining the page without necessarily indicating this or displaying the final tested URL. A positive result therefore does not prove that the old address itself serves the content. Record its status, destination header, intermediate steps and final response separately.
The entered URL’s indexed state and live result do not always describe the same version or step in that chain. Also inspect the exact destination URL in an authorised property covering it. Compare crawl date, declared canonical and Google’s known choice when available. The live test does not predict canonical selection. Retain both observations instead of treating one green screen as validation of a redirect, indexing or ranking.
Scroll the table horizontally to read every column. With a keyboard, focus the table area and use the arrow keys.
| Observation | Scope | Check to retain |
|---|---|---|
| HTTP response from /ancienne/ | Starting address | Redirect status and stated destination |
| Live test started on /ancienne/ | May examine /nouvelle/ after redirect | Check chain and final URL separately |
| Indexed state of /ancienne/ | Known index state of entered address | Do not treat it as the destination’s indexed state |
| Separate inspection of /nouvelle/ | Exact destination in a suitable property | Compare crawl and canonical; no ranking guarantee |
Correct and observe without promising timing
Retain the diagnosed problem, correction, date and successful checks. Update links and the sitemap if the URL changes. An indexing request can signal a correction but does not guarantee an inclusion date or ranking.
Recheck a representative set of pages rather than republishing the whole site after every fluctuation. If exclusion is intentional, document that decision. The useful result is a verifiable action: restore access, improve, consolidate, retain outside the index or protect with authentication.
Check Google controls and measure AI impressions
Google documentation consulted on 9 October 2026 now distinguishes an inclusion control for generative Search features and a dedicated performance report. In Search Console, check the relevant property’s setting and any inheritance before interpreting missing visibility. Inclusion establishes eligibility, not a guaranteed citation. This control does not replace indexing, snippet controls or controls concerning model training.
The generative AI report describes impressions in the covered features. Compare consistent periods, final pages, devices and countries. An absent report may reflect insufficient volume; it does not by itself establish exclusion. Separate impressions from visits and actual enquiries. Without property access, both the setting and results remain unverified.
Reference documents
- Google: crawling errors
- Google: using noindex · Source consulted
- Google: consolidating duplicate URLs · Source consulted
- Google: the role of robots.txt
- Google Search Console : URL Inspection · Source consulted
- Google Search Central : Generative AI optimization · Source consulted
- Google Search Console : Search generative AI control · Source consulted
- Google Search Console : Generative AI performance report · Source consulted
Content updated on
Documented reference points for this topic
| Reference | Value or syntax | Scope |
|---|---|---|
| Google Search · Canonical | rel="canonical" | A preference signal for Google; neither a redirect nor a selection guarantee. |
| Google Search · Sitemap | 50 000 URL · 50 MB | Limits per uncompressed file; larger sets require several sitemaps. |
Dates indicate when references were consulted. These facts are neither a site audit nor an overall compliance statement.
