AI systems · SEO

We audited our own indexing last month. We missed two of our own product pages.

Publishing a fix and confirming it worked are two different steps. Last month we only did the first one.

WritingBy Landon LittleSeptember 5, 20267 min read

We went back to check our own fix, not to look for something new

Last month we wrote about a sitemap that gave a search engine no reason to recrawl 88 of our 103 pages, fixed it, and published the diagnosis. That post shipped, the fix shipped, and the natural next move is to consider the problem closed and turn to something else.

We went back instead. Not because anything looked wrong, but because a fix that looks complete and a fix that has actually been verified against the live site are two different claims, and we had only made the first one. Opening the exact same Search Console report a month later, expecting a quiet confirmation, surfaced three separate problems the original pass never caught.

A second, older version of the redirect bug we thought we had already fixed

The August fix redirected a retired network of URLs, one folder deep under a /local/ path, to their current equivalents. It worked, and a spot check confirmed it. What that spot check did not surface was an even older version of the same URL pattern, one folder shallower, predating the one we had already caught, covering the same trades and the same cities.

The fix was mechanical once found: the same trade-to-page mapping already proven correct for the newer network, applied to the older one, verified end to end this time rather than by status code alone.

A page quietly declaring the wrong domain as its own canonical address

One of our own trade pages showed up flagged as crawled but not indexed. Inspecting it directly in Search Console showed why: as of its last crawl, the page declared its own canonical URL on our bare domain, while every other page on the site, and the domain's own redirect rules, treat the www address as canonical. A page contradicting the site's own stated structure is exactly the kind of signal that makes a crawler hesitate.

Checking the live page directly found the contradiction was already gone. Somewhere between that last crawl and today, the canonical tag had been corrected to match every other page on the site. Nothing needed fixing in the code. What was missing was simply a nudge back to the crawler that the page was worth looking at again.

The one that mattered most: two of our own product pages, never crawled

The largest section of that report is pages a search engine has discovered through our sitemap but never visited at all. Thirty-four of them, as of this check. Most were pages that made sense to still be waiting their turn. Two were not: our own pages for roofing and for restaurants, live, correctly built, linked internally, present in the sitemap since launch, and never crawled a single time.

We are not going to pretend a manual nudge to recrawl a handful of pages fixes that ceiling. It does not. What it does is keep the pages that are actually finished from sitting invisible while the underlying constraint, which is a separate and slower problem, gets worked on.

What we actually changed, and what we are not claiming

We are not claiming any of this moved our search position. Nothing here has had time to show that yet, and we said the same thing last month for the same honest reason: isolating the effect of one fix takes longer than a same-day check allows.

What changed is the process, not just the fix. Verifying that a fix landed is now its own explicit step, done by testing the live site directly rather than by trusting a report that may be describing a version of the site that no longer exists. A report that says something is broken is a claim about the past. The only way to know the present state is to go and look.

Questions this post answers

How do you verify that a Search Console fix actually worked?
By live-testing the actual current behavior of the site, not by trusting the report. Search Console's indexing data reflects the last time a crawler visited a URL, which can be weeks or months old. A page can already be fixed in production while the report still shows the old, broken state.
What is the difference between Discovered and Crawled, currently not indexed?
Discovered means a search engine knows a URL exists, usually from a sitemap, but has not visited it yet. Crawled means it visited and chose not to index what it found. Discovered is a queue problem. Crawled is closer to a quality or trust judgment, though it can also just mean the page changed since that one visit.
Can a real, correctly built page simply never get crawled?
Yes. A page can be live, linked internally, present in the sitemap, and completely correct, and still sit for months with zero crawls if a search engine has not allocated the budget to reach it. That allocation tracks a site's overall authority, not any single page's own quality.
Why does a canonical tag pointing at the wrong domain matter?
A canonical tag tells a search engine which URL is the authoritative version when more than one URL could serve the same content. If a page declares itself canonical at an address that immediately redirects elsewhere, that is a direct contradiction a crawler has to resolve, and the safest resolution for it is often to index neither version until the confusion clears.

Want this working for your business?

We build the automation your team keeps meaning to build, then hand it over running. Book a call and we will map the first working slice.

Book a 20-minute call

Pick a time that works. Twenty minutes on video, no pitch. You leave knowing whether this is worth doing.