In more detail
After crawling a page, Google analyses its text, images, video, structured data and other signals. It decides whether the page is a duplicate of another (and, if so, which URL is canonical), and whether it's worth storing. Google's guide to how Search works notes that "indexing isn't guaranteed; not every page that Google processes will be indexed".
Pages may not be indexed because:
- They carry a noindex directive, in a meta tag or HTTP header
- They're blocked by robots.txt, so Google can't see their content
- They're treated as duplicates and another URL was chosen as canonical
- They return errors, redirect or look like soft 404s
- Google judged them low value, for example thin or near-empty pages
- Google hasn't got round to them yet
Search Console's Page indexing report lists indexed and non-indexed pages with a reason for each, such as "Discovered – currently not indexed" or "Crawled – currently not indexed". The URL Inspection tool shows the status of a single URL and lets you request indexing after fixing a problem.
Why it matters when hiring an agency
If an important page isn't indexed, nothing else an agency does for it can help. Indexing checks belong at the start of any engagement. Ask an agency to show you how many of your important pages are indexed, the main reasons others aren't, and what it plans to fix first.
Be sceptical of services that promise to "force" or "instantly" index pages. There's no paid fast lane into Google's index. For a quick check of any page, our Google Index Checker tests robots.txt, noindex directives, canonical tags and status codes, and explains what to fix.