In more detail

Google's crawl budget documentation describes it as a combination of two things. Crawl capacity is how much Googlebot can crawl without overloading your server. Crawl demand is how much Google wants to crawl, based on how popular and how fresh your pages are.

Google is clear about who needs to worry. Its guide is aimed at large sites (roughly a million or more unique pages with content that changes moderately often) and medium or larger sites with very rapidly changing content (around 10,000 or more pages changing daily). For most small and medium sites, Google says, crawl budget isn't something to worry about; if pages are crawled the day they're published, the site is fine.

Common causes of wasted crawling include faceted navigation that creates endless filter combinations, session IDs and tracking parameters in URLs, infinite calendars, soft 404 pages, long redirect chains and large numbers of near-duplicate pages. Fixes include tightening robots.txt rules, consolidating duplicates, keeping XML sitemaps clean and improving server speed.

Why it matters when hiring an agency

If your site has a few hundred pages and an agency spends time on "crawl budget optimisation", ask why. It's usually not where the value is.

For large ecommerce, marketplace, classifieds or publishing sites, crawl efficiency can be a real constraint, and experience with it is worth paying for. Ask how the agency would measure it: Search Console's Crawl Stats report and server log analysis are the main sources. A good agency will show you which URL patterns waste crawling and what fixing them should change.