Skip to content
Magnetic Keys
SEO & AEO

Crawl budget

Definition

Crawl budget is the number of pages a search engine will crawl on a site within a given period, determined by the site's authority, server responsiveness and the perceived value of its URLs.

It only becomes a practical constraint on large sites — typically tens of thousands of URLs. Below that, crawl budget problems are usually index bloat problems in disguise: faceted navigation, session parameters and thin generated pages consuming crawls that should go to commercial pages.

The fix is rarely to request more crawling. It is to reduce the number of low-value URLs through robots directives, canonicalisation, parameter handling and noindex, so the crawler spends its allocation on pages that matter.

More in SEO & AEO

AEO (Answer Engine Optimisation)

AEO is the practice of structuring content and data so AI assistants such as ChatGPT, Perplexity and Google AI Overviews cite a business as a source when answering questions, including when no click follows.

AI visibility

How often a brand is named or cited in answers from AI assistants for the prompts its buyers actually use. The emerging equivalent of rank tracking, measured by prompting at scale and counting mentions.

Backlink

A backlink is a hyperlink from another website to yours, treated by search engines as a signal of credibility. Value depends on the linking site's authority and topical relevance, not on link count.

Canonical tag

A canonical tag tells search engines which URL is the preferred version of a page when duplicate or near-duplicate content exists at multiple addresses, consolidating ranking signals onto one URL.

Content cluster

A content cluster is a comprehensive pillar page on a broad topic surrounded by interlinked supporting pages covering specific subtopics, structured so internal links consolidate relevance signals.

Core Web Vitals

Core Web Vitals are Google's user-experience metrics: Largest Contentful Paint (loading), Interaction to Next Paint (responsiveness) and Cumulative Layout Shift (visual stability), measured on real user traffic.

All 87 terms
Put it to work

Definitions are free.So is the cost review.

A short call, then a written review inside 72 hours: what your marketing costs, what it produces, and what is worth fixing first.

Browse the glossary
hello@magnetickeys.comWhatsApp +971 52 529 5577Al Khawaneej, Dubai · United Arab Emirates